Simplified EKS Monitoring with Amazon Managed Service for Prometheus and Grafana

Monitoring Kubernetes clusters effectively is critical for understanding performance, identifying bottlenecks, and ensuring high availability. Amazon Managed Service for Prometheus (AMP) simplifies monitoring for Amazon EKS (Elastic Kubernetes Service) clusters by allowing users to collect and analyze metrics without managing the underlying infrastructure. Combined with Amazon Managed Grafana for visualizations, this provides a complete monitoring solution.

In this post, we’ll explore how to set up a simplified monitoring solution for Amazon EKS using Amazon Managed Service for Prometheus. By the end, you will have a clear understanding of setting up this environment, visualizing metrics, and best practices for cleanup.

Introduction to Amazon Managed Service for Prometheus and Its Role in EKS Monitoring

Amazon Managed Service for Prometheus (AMP) is a fully managed service compatible with the open-source Prometheus monitoring system. It’s designed to collect, store, and query EKS cluster metrics at scale without the operational overhead of managing Prometheus. AMP enables Kubernetes-native monitoring by scraping and storing metrics, which are crucial for tracking the health and performance of your applications.

AMP’s key role in EKS monitoring is to provide insights into node health, pod resource usage, network traffic, and other essential metrics—all without the need to manage agents or complex configurations.

Setting Up the Demo Environment: Prerequisites and Initial Steps

Before we dive into setting up the environment, ensure you have the following prerequisites:

AWS Account: Ensure you have an active AWS account.
AWS CLI: Install and configure the AWS Command Line Interface (CLI).
kubectl: Install the Kubernetes command-line tool for managing your EKS cluster.
eksctl: Use this tool to create and manage EKS clusters easily.
IAM Permissions: Ensure you have sufficient permissions to create and manage EKS clusters, roles, policies, and Amazon Managed Service for Prometheus.

To start, log in to the AWS Management Console, navigate to the IAM roles, and create a service-linked role that grants EKS access to Amazon Managed Service for Prometheus.

Creating an Amazon EKS Cluster: The Foundation for Our Monitoring Setup

Next, we will create the Amazon EKS cluster, which will serve as the foundation for our monitoring solution. Use eksctl to set up the cluster quickly:

eksctl create cluster \

–name demo-cluster \

–region us-west-2 \

–nodes 3

This command creates a basic 3-node EKS cluster. Once the cluster is up, configure kubectl to interact with it:

aws eks –region us-west-2 update-kubeconfig –name demo-cluster

You can verify the cluster is up and running by checking the status of the nodes:

kubectl get nodes

Establishing Amazon Managed Service for Prometheus: Collecting Metrics Without Agents

Once your cluster is running, it’s time to set up AMP to collect metrics. The beauty of AMP is that it doesn’t require manual deployment of Prometheus instances in your EKS cluster.

Step 1: Create an AMP Workspace

Go to the Amazon Managed Service for Prometheus console and create a new workspace. This workspace will act as the storage and query backend for the Prometheus metrics collected from your EKS cluster.

Step 2: Set Up Prometheus Scrape Configurations

You’ll need to configure how AMP scrapes metrics from your EKS cluster. This is done by setting up a ServiceMonitor in your Kubernetes cluster, which directs Prometheus on what to scrape:

apiVersion: monitoring.coreos.com/v1

kind: ServiceMonitor

metadata:

name: eks-metrics

labels:

release: prometheus

spec:

selector:

matchLabels:

app: eks-app

endpoints:

– port: http

namespaceSelector:

matchNames:

– monitoring

This ServiceMonitor configuration ensures AMP can scrape the desired metrics from your Kubernetes applications.

Integrating Amazon Managed Grafana for Visualizing EKS Metrics

With metrics flowing into AMP, you can use Amazon Managed Grafana to visualize the data. Managed Grafana provides pre-built dashboards for Kubernetes, simplifying the visualization process.

Step 1: Set Up Amazon Managed Grafana

In the AWS Management Console, navigate to Amazon Managed Grafana and create a new workspace. Ensure that AMP is enabled as a data source for Grafana.

Step 2: Add AMP as a Data Source in Grafana

After creating the Grafana workspace, configure it to use the Prometheus workspace created in the previous step as a data source. This allows Grafana to pull metrics from AMP.

Step 3: Use Pre-Built Dashboards

Amazon Managed Grafana comes with pre-built dashboards for Kubernetes clusters. You can import a dashboard specific to EKS or create your own based on your monitoring needs.

Configuring Amazon EKS for Metric Scraping: Ensuring Collector Access

To enable Prometheus metric scraping, we need to deploy Prometheus node exporters on the EKS cluster. These exporters will gather node-level metrics, which Prometheus will scrape and store.

Deploy a Prometheus node exporter using Helm:

helm install prometheus prometheus-community/prometheus-node-exporter –namespace monitoring

Ensure that the necessary permissions are set so Prometheus can scrape the node exporters without any issues.

Cleanup Procedures: Maintaining Best Practices Post-Demo

Once you’ve finished exploring the demo environment, it’s essential to clean up resources to avoid incurring unnecessary costs.

Step 1: Delete the EKS Cluster

eksctl delete cluster –name demo-cluster

This command removes the EKS cluster and associated resources.

Step 2: Delete AMP Workspace and Grafana Instance

Head back to the AWS Management Console and delete the AMP workspace and the Amazon Managed Grafana instance.

Step 3: Remove Helm Resources

Ensure you clean up any Prometheus exporters you deployed:

helm uninstall prometheus –namespace monitoring

By following these cleanup steps, you ensure a cost-efficient environment while maintaining best practices.

Conclusion

Amazon Managed Service for Prometheus offers a simplified and scalable solution for monitoring Amazon EKS clusters. When integrated with Amazon Managed Grafana, it provides a powerful visualization tool, allowing teams to monitor and react to key metrics efficiently. This fully managed approach eliminates operational overhead, enabling teams to focus on application performance.