Troubleshooting

This document describes where can you look for information when you encounter problems with Carthago Operator for Jenkins.

Operator logs

To get logs from Operator pod, run:

$ kubectl -n <operator-namespace> logs <operator-pod-name> -f 

In the logs look for WARNING, ERROR and SEVERE keywords.

Jenkins logs

To learn about errors with Jenkins, the state of Jenkins pod:

$ kubectl -n <jenkins-namespace> get pod <name-of-the-jenkins-pod> -w

or

$ kubectl -n <jenkins-namespace> describe pod <name-of-the-jenkins-pod>
  • If is in a CrashLoopBackoff the fault is in the Jenkins itself.

  • If Jenkins pod is in a ImagePullBackOff, it means that Operator can’t download the image, check the name of Jenkins Custom Resource (CR) and make sure you have the access to the repo.

  • If the Operator is constantly terminating our pod with missing-plugins messages, that means the plugins lost compatibility with the Jenkins image and their versions need to be updated.

You can also check the logs from jenkins-controller container:

$ kubectl -n <jenkins-namespace> logs <jenkins-pod> jenkins-controller -f 

If you encounter problems with Jenkins during its initial configuration, check initial-config container:

$ kubectl -n <jenkins-namespace> logs <jenkins-pod>  -f 

Kubernetes Events

You can also check Kubernetes events from namespaces where Operator and Jenkins are deployed in.

Sometimes it might be worth it to check events from default namespace too, especially in case of running out of resources on Node with Operator and Jenkins.

$ kubectl -n <namespace> get events --sort-by='{.lastTimestamp}'

Quick soft reset

In case of problems with Jenkins, you can reset its pod. To do that, delete it and wait for it to come up again. Operator will re-create the pod according to config in Jenkins CR.

$ kubectl -n <jenkins-namespace> delete pod <jenkins-pod>

Running Operator in debug mode

Running Operator in debug mode makes it log DEBUG level logs too, and those can sometimes help understand what’s going on with Operator. To enable debug mode, add --debug=true to Operator container’s args.

  • in Operator Helm chart values.yaml file, you can find it under .operator.args
  • in Operator Deployment, you can find it under .spec.template.spec.containers[0].args