Debugging Pods

Kubernetes pods and deployments can fail due to misconfigurations, resource constraints, or runtime errors. Effective debugging requires leveraging kubectl commands to inspect pod states, retrieve logs, and interact with containers. This section covers tools and workflows to diagnose and resolve common issues.

Checking Pod Status and Logs¶

Use kubectl describe and kubectl logs to identify root causes of pod failures.

1. Describe the Pod¶

kubectl describe pod <pod-name>

Look for events under the "Events" section, which often reveal errors like:
- "Failed to pull image" (image pull issues)
- "CrashLoopBackOff" (container exited unexpectedly)
- "InvalidImageName" (syntax errors in image names)

2. View Container Logs¶

kubectl logs <pod-name> -c <container-name>

If the container isn’t running, use --previous to check logs from the last terminated instance:

kubectl logs <pod-name> -c <container-name> --previous

3. Check Pod Phase and Conditions¶

kubectl get pod <pod-name> --output=jsonpath='{.status.phase}'

A phase of Error or Failed indicates a critical issue. Use kubectl get pod <pod-name> --output=jsonpath='{.status.conditions}' to inspect detailed conditions like Ready, Initialized, and ContainersReady.

Debugging Running or Stuck Pods¶

1. Exec into a Running Pod¶

If the pod is in Running state but not responding, interact with the container:

kubectl exec -it <pod-name> -- <command>

For example, to open a shell:

kubectl exec -it <pod-name> -- sh

Use this to check file systems, verify configurations, or run diagnostic commands like curl or netstat.

2. Debug a Non-Running Pod¶

If the pod is in Error or CrashLoopBackOff state, use kubectl debug to add a temporary shell container:

kubectl debug <pod-name> -it --image=busybox --command="sh"

This allows inspecting the pod’s filesystem without restarting it.

Troubleshooting Deployments¶

1. Check Deployment Status¶

kubectl describe deployment <deployment-name>

Look for rollout status under "Events" and "Replicas" in the "Status" section. A mismatch between Desired and Current replicas may indicate deployment failures.

2. Rollback Failed Deployments¶

If a deployment fails, roll back to a previous version:

kubectl rollout undo deployment/<deployment-name>

View rollout history to identify which revision to revert:

kubectl rollout history deployment/<deployment-name>

3. Check Pod Template Configuration¶

Ensure the deployment’s pod template is correctly configured:

kubectl get deployment <deployment-name> -o jsonpath='{.spec.template.spec.containers}'

Look for missing environment variables, incorrect image names, or resource limits.

Common Issues and Workarounds¶

Network Issues: Use kubectl get endpoints to verify service endpoints. Test connectivity with curl or telnet from within a pod.
Resource Limits: Check if pods are terminated due to OOM (Out-Of-Memory) errors:
```
kubectl describe pod <pod-name> | grep -i "oom"
```

Configuration Errors: Validate ConfigMaps and Secrets:

kubectl get configmap <configmap-name> -o yaml
kubectl get secret <secret-name> -o jsonpath='{.data}'

Key takeaways¶

Use kubectl describe and kubectl logs to diagnose pod failures quickly.
kubectl exec and kubectl debug allow interactive troubleshooting of running or stuck pods.
Monitor deployment rollouts and use rollbacks to recover from failed updates.
Validate configurations for ConfigMaps, Secrets, and resource limits to prevent common errors.
Combine log analysis with network and resource checks for comprehensive debugging.