Monitoring the memory usage of your Amazon EC2 instances is crucial for maintaining the health and performance of your applications. Unfortunately, by default, Amazon CloudWatch only provides metrics for CPU utilization, disk I/O, and network I/O. Memory usage metrics aren’t offered out of the box. However, you can monitor memory usage using a custom script to collect and send memory metrics to CloudWatch.
Here’s a step-by-step guide on how to set up memory monitoring for your EC2 instances and make it available via CloudWatch Logs.
Step 1: Install the CloudWatch Agent
First, you need to install the CloudWatch agent on your EC2 instances. The CloudWatch agent can collect system-level metrics, including memory usage.
- Connect to your EC2 instance:
ssh -i your-key-pair.pem ec2-user@your-ec2-instance-public-dns
- Download the CloudWatch agent:
wget https://s3.amazonaws.com/amazoncloudwatch-agent/amazon_linux/amd64/latest/amazon-cloudwatch-agent.rpm
- Install the CloudWatch agent:
sudo rpm -U ./amazon-cloudwatch-agent.rpm
Step 2: Configure the CloudWatch Agent
Next, configure the CloudWatch agent to collect memory metrics.
- Create a configuration file:
sudo vi /opt/aws/amazon-cloudwatch-agent/bin/config.json
- Add the following configuration:
{
“metrics”: {
“metrics_collected”: {
“mem”: {
“measurement”: [
“mem_used_percent”
],
“metrics_collection_interval”: 60
}
}
}
}
Step 3: Start the CloudWatch Agent
- Start the CloudWatch agent:
sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json -s
- Verify the CloudWatch agent is running:
sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -m ec2 -a status
Step 4: Verify Memory Metrics in CloudWatch
- Log in to the AWS Management Console.
- Navigate to CloudWatch.
- Select Metrics.
- Choose the CWAgent namespace.
- Find the mem_used_percent metric.
Step 5: Send Memory Metrics to CloudWatch Logs
To send these metrics to CloudWatch Logs, you need to configure a CloudWatch Logs agent or use a Lambda function to push the logs.
- Create a CloudWatch Log Group:
aws logs create-log-group –log-group-name EC2MemoryMetrics - Create a Lambda function to send metrics from CloudWatch to CloudWatch Logs:
import boto3
import json
import os
cloudwatch = boto3.client(‘cloudwatch’)
logs = boto3.client(‘logs’)
def lambda_handler(event, context):
metrics = cloudwatch.get_metric_data(
MetricDataQueries=[
{
‘Id’: ‘memoryUsage’,
‘MetricStat’: {
‘Metric’: {
‘Namespace’: ‘CWAgent’,
‘MetricName’: ‘mem_used_percent’,
‘Dimensions’: [
{
‘Name’: ‘InstanceId’,
‘Value’: event[‘instanceId’]
}
]
},
‘Period’: 60,
‘Stat’: ‘Average’,
},
‘ReturnData’: True,
},
],
StartTime=event[‘startTime’],
EndTime=event[‘endTime’],
)
for result in metrics[‘MetricDataResults’][0][‘Values’]:
logs.put_log_events(
logGroupName=’EC2MemoryMetrics’,
logStreamName=event[‘instanceId’],
logEvents=[
{
‘timestamp’: int(event[‘startTime’].timestamp() * 1000),
‘message’: json.dumps(result)
},
],
)
return {
‘statusCode’: 200,
‘body’: json.dumps(‘Memory usage sent to CloudWatch Logs’)
}
3. Deploy the Lambda function and schedule it to run regularly using CloudWatch Events or EventBridge.
By following these steps, you can effectively monitor the memory usage of your EC2 instances and make this information available via CloudWatch Logs, ensuring you have the necessary insights to maintain optimal performance.