Identifying and Analysing Kubernetes Pods with Multiple Restarts

In a Kubernetes environment, monitoring the health of your pods is crucial for maintaining application stability and performance. One key indicator of potential issues is the restart count of pods. If a pod restarts frequently, it could signify underlying problems such as resource constraints, crashes, or misconfigurations.

In this post, we'll create a bash script to identify pods that have restarted more than two times. This script leverages the output of the `kubectl get pods` command to filter and display relevant pod details.

Step-by-Step Guide

Prerequisites:
- Ensure you have kubectl installed and configured to access your Kubernetes cluster.
- Basic knowledge of bash scripting.

Understanding the Command:

The command kubectl get pods provides a list of pods along with their statuses. Here’s a sample output:

   NAME                      READY   STATUS    RESTARTS   AGE
   my-app-5d7fcb5f89-2j5x6   1/1     Running   1          10d
   my-app-5d7fcb5f89-7g2h3   1/1     Running   3          10d
   my-app-5d7fcb5f89-k9l8m   1/1     Running   0          10d

Bash Script:
Let’s write a bash script to extract pods that have restarted more than twice.

#!/bin/bash

# Get the list of pods with their details
pods=$(kubectl get pods --no-headers)
echo "Pods with more than 2 restarts:"

# Loop through each line of the pod details
echo "$pods" | while read -r line; do

# Extract the restart count
restarts=$(echo $line | awk '{print $4}')  
   
# Check if the restart count is greater than 2
if [ "$restarts" -gt 2 ]; then
  echo $line
fi
done

Script Explanation:
- Line 1: The shebang (#!/bin/bash) tells the system to use the bash shell to interpret the script.
- Line 3: Executes kubectl get pods --no-headers to fetch the pod details without the header row.
- Line 7: Iterates over each line of the pod details.
- Line 9: Uses awk to extract the restart count (the fourth column in the kubectl get pods output).
- Line 12: Checks if the restart count is greater than 2 and prints the line if true.

Example Usage:
Save the script to a file, for example, check_pod_restarts.sh, and make it executable:

$ bash
$ chmod +x check_pod_restarts.sh

Run the script:

$ bash
$ ./check_pod_restarts.sh

The output will display pods that have restarted more than twice:

Pods with more than 2 restarts:
my-app-5d7fcb5f89-7g2h3 1/1 Running 3 10d

By using this simple bash script, you can quickly identify pods in your Kubernetes cluster that might be experiencing issues due to frequent restarts. This proactive monitoring can help you address potential problems early, ensuring the stability and reliability of your applications.

Feel free to adapt this script to fit your specific needs, and let me know if you have any questions or improvements!

---

This blog post outlines the problem, provides a clear solution with a bash script, and includes an example to demonstrate the script's usage. Feel free to tweak the content to match your style and add any additional insights or tips you might have!

— -

Author Bio:

With over a decade of experience in DevOps and SRE, I specialize in optimizing system performance and automating deployment processes. My expertise lies in CI/CD, configuration management, and cloud migrations, and I am passionate about integrating tools like Jenkins, Git, Terraform, and Ansible to drive efficiency and reliability. Follow me for more insights on enhancing application reliability and performance.

— -

Feel free to share your thoughts or ask questions in the comments below! If you found this post helpful, don’t forget to like and share it with your network.