Running with Kubernetes (k8s)

This guide explains how to deploy and run the Rated Log Indexer in a Kubernetes environment.

Prerequisites

• Access to a Kubernetes cluster

• kubectl CLI tool installed and configured

• Basic understanding of Kubernetes concepts (ConfigMaps, Deployments)

• Rated Log Indexer credentials and configuration details

Setup Instructions

1. Prepare Configuration

1. Navigate to the config directory where you'll find rated-configmap.example.yaml

2. Create your configuration file:

cp config/rated-configmap.example.yaml config/rated-configmap.yaml

1. Modify the configuration file with your specific settings

The example file includes both the ConfigMap for your indexer configuration and the Deployment specification needed for Kubernetes.

2. Deploy to Kubernetes

Apply the configurations to your cluster:

kubectl apply -f config/rated-configmap.yaml

3. Verify Deployment

Check the status of your deployment:

# Check deployment status
kubectl get deployments rated-log-indexer

# Check pod status
kubectl get pods -l app=rated-log-indexer

# View logs
kubectl logs -l app=rated-log-indexer

Configuration Reference

ConfigMap Settings

The ConfigMap contains your Rated Log Indexer configuration in YAML format. Key sections include:

• inputs: Configure your data sources

  • integration: Specify the integration type (e.g., cloudwatch)

  • filters: Define how to process and transform logs

  • offset: Configure ingestion tracking and start points

• output: Configure where processed data should be sent

• secrets: Manage sensitive information

Configuration yaml

Deployment Settings

The Deployment configuration defines how the indexer runs in your cluster:

• replicas: Number of indexer instances (default: 1)

• image: Docker image to use (default: ratedlabs/rated-log-indexer:latest)

• volumeMounts: Configuration file mounting

• volumes: ConfigMap volume definition

ratedlabs/rated-log-indexer - Docker Imagehub.docker.com

Security Considerations

1. Sensitive Data: Consider using Kubernetes Secrets instead of ConfigMap for sensitive values

2. RBAC: Ensure appropriate RBAC policies are in place for the indexer pod

Troubleshooting

Common Issues

1. Pod Startup Failures

kubectl describe pod -l app=rated-log-indexer

1. Configuration Issues

# View ConfigMap
kubectl get configmap indexer-config -o yaml

# Check container logs
kubectl logs -l app=rated-log-indexer

1. Resource Constraints

kubectl top pod -l app=rated-log-indexer

Maintenance

Updating Configuration

1. Update the ConfigMap:

kubectl apply -f config/rated-configmap.yaml

1. Restart the pods:

kubectl rollout restart deployment rated-log-indexer

Version Updates

To update the indexer version:

kubectl set image deployment/rated-log-indexer indexer=ratedlabs/rated-log-indexer:new-version

Best Practices

1. Always use version tags for the Docker image instead of latest

2. Implement appropriate resource requests and limits

3. Set up monitoring and alerting

4. Regularly backup your configuration

5. Use namespaces to isolate the indexer deployment

6. Implement liveness and readiness probes