Amazon Relational Database Service (RDS) event notifications provide critical insights into database health, performance, and activities. However, managing and sifting through a flood of notifications can be overwhelming. By leveraging Amazon SNS Subscription Filter Policies, you can efficiently filter RDS event notifications and ensure the right messages reach the right audience.
This guide will explore setting up a filtered notification system for RDS events using Amazon SNS.
Understanding the Need for Two SNS Topics: Why a Single Topic Isn’t Enough for Filtering
Amazon RDS allows you to subscribe to its event notifications using Amazon SNS. However, a single SNS topic can become a bottleneck when filtering is needed for different types of events. For example:
- Scenario A: Operations teams need alerts for critical database errors.
- Scenario B: Developers want updates on database modifications for debugging.
While Amazon SNS supports filter policies, using two separate topics allows for better organization and granularity:
- Topic A: Receives all RDS events without filtering.
- Topic B: Applies a Subscription Filter Policy to refine the events delivered.
This separation ensures you can send filtered notifications to specific recipients while maintaining a complete event log.
Creating a Lambda Function to Parse and Add Attributes: Preparing Messages for Filtering
For SNS filtering to work, messages must contain attributes that can be used for policy matching. RDS event notifications don’t include these attributes by default, so you must use a Lambda function to parse and enrich messages with attributes.
Steps to Create the Lambda Function:
- Set up the Lambda Function:
- Create a function in AWS Lambda.
- Use a runtime like Python or Node.js.
- Parse RDS Events: Extract information such as eventCategory, dbInstanceIdentifier, and severity from the event payload.
- Add Attributes: Use the MessageAttributes parameter to include custom attributes when re-publishing to the filtered SNS topic.
Example Code (Python):
import boto3
import json
sns = boto3.client(‘sns’)
def lambda_handler(event, context):
message = json.loads(event[‘Records’][0][‘Sns’][‘Message’])
attributes = {
‘eventCategory’: {
‘DataType’: ‘String’,
‘StringValue’: message.get(‘EventCategory’, ‘Unknown’)
},
‘severity’: {
‘DataType’: ‘String’,
‘StringValue’: message.get(‘Severity’, ‘Unknown’)
}
}
sns.publish(
TopicArn=’arn:aws:sns:region:account-id:FilteredTopic’,
Message=json.dumps(message),
MessageAttributes=attributes
)
return {‘statusCode’: 200, ‘body’: ‘Message processed and re-published’}
Crafting an SNS Subscription Filter Policy: Defining Rules for Event Notification Delivery
A Subscription Filter Policy allows you to define criteria for message delivery. For example, you can filter notifications based on eventCategory or severity.
Example Filter Policy:
{
“eventCategory”: [“configuration change”, “failure”],
“severity”: [“critical”]
}
This policy ensures that only critical events related to configuration changes or failures are delivered to the subscriber.
Steps to Apply the Filter Policy:
- Navigate to the SNS console.
- Select the subscription associated with your filtered topic.
- Add the filter policy under the Subscription Filter Policy section.
Configuring the RDS Event Subscription: Connecting RDS Events to the SNS Topic
To ensure RDS event notifications flow into your SNS topic:
- Open the Amazon RDS Console.
- Navigate to Event Subscriptions.
- Create or modify a subscription:
- Select the source (e.g., a DB instance or cluster).
- Choose the types of events (e.g., configuration changes, availability issues).
- Select your primary SNS topic for raw notifications.
- Save the subscription.
Verifying the Filtering Setup: Testing and Troubleshooting Your Configuration
Once everything is set up, verify that filtered notifications are delivered as expected:
- Trigger an RDS Event:
- Perform actions like modifying an instance or simulating a failure.
- Check the Lambda Function:
- Ensure the Lambda function is parsing messages correctly and adding attributes.
- Validate Filtered Notifications:
- Confirm that the filtered topic only delivers notifications that match the policy.
- Debugging Issues:
- Review CloudWatch logs for Lambda execution errors.
- Check the SNS topic’s Delivery Status Logs for insights into message delivery.
Conclusion
Combining Amazon RDS, SNS, Lambda, and Subscription Filter Policies allows you to streamline event notifications, ensuring that only relevant messages reach the right audience. This approach reduces noise, enhances productivity, and provides a scalable solution for managing database notifications.