Skip to main content

​
Steps to ensure leader election

1

Check for the active leader pod

Run the following command to confirm that only one collector manager pod is active as the leader:
kubectl get pods -A -l collector-manager/leader=true
Example output:
NAME                                         READY   STATUS    RESTARTS   AGE
plerion-collector-manager-ccbc55c5d-dr27w    1/1     Running   0          10m
Only one pod should appear in the output. If multiple pods are listed, continue with the next steps.
2

Validate configuration

Ensure that leader election is enabled in the collector manager configmap:
kubectl get configmap plerion-collector-manager -n plerion-system \
  -o go-template='{{.data.LEADER_ELECTION}}'
The value should be true.See Validate Helm release for steps to confirm your configuration.
3

Check pod logs for leader activity

If multiple pods appear as leaders, verify that only one pod is active and the others are waiting to acquire the lease or shutting down.View logs for a specific pod:
kubectl logs -f plerion-collector-manager-ccbc55c5d-dr27w
Example logs for a pod waiting to acquire the lease:
{"level":"info","ts":"2023-11-21T03:33:01Z","logger":"setup","msg":"initializing controller"}
{"level":"info","ts":"2023-11-21T03:33:01Z","logger":"setup","msg":"fetching tenant config"}
{"level":"info","ts":"2023-11-21T03:33:08Z","logger":"controller-runtime.metrics","msg":"Metrics server is starting to listen","addr":":8080"}
{"level":"info","ts":"2023-11-21T03:33:08Z","logger":"setup","msg":"starting manager"}
{"level":"info","ts":"2023-11-21T03:33:08Z","logger":"setup","msg":"Starting server","kind":"health probe","addr":"[::]:8081"}
{"level":"info","ts":"2023-11-21T03:33:08Z","logger":"setup","msg":"starting server","path":"/metrics","kind":"metrics","addr":":8080"}
{"level":"info","ts":"2023-11-21T03:33:08Z","msg":"attempting to acquire leader lease plerion-system/ecaf1259.collector.plerion.com...\\n"}
Example logs for a pod that has acquired the lease:
{"level":"info","ts":"2023-11-21T03:33:01Z","logger":"setup","msg":"initializing controller"}
{"level":"info","ts":"2023-11-21T03:33:01Z","logger":"setup","msg":"fetching tenant config"}
{"level":"info","ts":"2023-11-21T03:33:08Z","logger":"controller-runtime.metrics","msg":"Metrics server is starting to listen","addr":":8080"}
{"level":"info","ts":"2023-11-21T03:33:08Z","logger":"setup","msg":"starting manager"}
{"level":"info","ts":"2023-11-21T03:33:08Z","logger":"setup","msg":"Starting server","kind":"health probe","addr":"[::]:8081"}
{"level":"info","ts":"2023-11-21T03:33:08Z","logger":"setup","msg":"starting server","path":"/metrics","kind":"metrics","addr":":8080"}
{"level":"info","ts":"2023-11-21T03:33:08Z","msg":"attempting to acquire leader lease plerion-system/ecaf1259.collector.plerion.com...\\n"}
{"level":"info","ts":"2023-11-21T03:33:25Z","msg":"successfully acquired lease plerion-system/ecaf1259.collector.plerion.com\\n"}
{"level":"info","ts":"2023-11-21T03:33:25Z","logger":"setup","msg":"Starting EventSource","controller":"resourcecollector","source":"<redacted>"}
{"level":"info","ts":"2023-11-21T03:33:25Z","logger":"setup","msg":"Starting Controller","controller":"resourcecollector"}
{"level":"info","ts":"2023-11-21T03:33:25Z","logger":"setup","msg":"Starting workers","controller":"resourcecollector","worker count":1}
If leader election cannot be ensured in the current release, reinstall the collector manager.Follow the uninstall and install guides to reset the deployment.
⌘I