Automating the provisioning and management of EC2 infrastructure on AWS is critical for achieving high availability, consistency, and scalability. By leveraging Infrastructure as Code (IaC) with Terraform and integrating it with a CI/CD pipeline using GitHub Actions, you can streamline the deployment process, enhance security, and minimize human error. This blog post will explore the steps to automate high-availability EC2 infrastructure on AWS.
Leveraging Infrastructure as Code (IaC) and CI/CD Pipeline for Automating EC2 Infrastructure Provisioning on AWS
Infrastructure as Code (IaC) allows you to define your infrastructure using code, making it easy to manage, version, and deploy. Terraform is a popular IaC tool that supports AWS and other cloud providers. By combining Terraform with GitHub Actions, you can automate the provisioning and management of your EC2 infrastructure, ensuring consistency and repeatability.
Using OpenID Connect to Enhance Security and Avoid Credential Exposure
OpenID Connect (OIDC) provides a secure way to authenticate GitHub Actions with AWS without exposing long-term credentials. By configuring AWS to trust GitHub as an identity provider, you can use short-lived tokens to access AWS resources, reducing the risk of credential leakage.
Defining Infrastructure with Terraform Configuration Files
To start, you must define your infrastructure using Terraform configuration files. These files describe the resources you want to create, such as VPCs, subnets, EC2 instances, Auto Scaling Groups, and load balancers.
Creating a VPC with Public Subnets
A Virtual Private Cloud (VPC) is essential for isolating your infrastructure. In your Terraform configuration, define a VPC with public subnets to ensure your EC2 instances can communicate with the Internet.
resource “aws_vpc” “main” {
cidr_block = “10.0.0.0/16”
}
resource “aws_subnet” “public” {
count = 2
vpc_id = aws_vpc.main.id
cidr_block = cidrsubnet(aws_vpc.main.cidr_block, 8, count.index)
map_public_ip_on_launch = true
availability_zone = data.aws_availability_zones.available.names[count.index]
}
Launching an Auto Scaling Group with a Launch Template
Auto Scaling Groups ensure your application can handle varying traffic levels by automatically adjusting the number of EC2 instances. Use a launch template to define the instance configuration.
resource “aws_launch_template” “app” {
name_prefix = “app”
image_id = “ami-0c55b159cbfafe1f0”
instance_type = “t2.micro”
}
resource “aws_autoscaling_group” “app” {
desired_capacity = 2
max_size = 3
min_size = 1
vpc_zone_identifier = [for subnet in aws_subnet.public : subnet.id]
launch_template {
id = aws_launch_template.app.id
version = “$Latest”
}
}
Setting Up an Application Load Balancer for Traffic Distribution
An Application Load Balancer (ALB) distributes incoming traffic across multiple EC2 instances, improving availability and fault tolerance.
resource “aws_lb” “app” {
name = “app-alb”
internal = false
load_balancer_type = “application”
security_groups = [aws_security_group.lb.id]
subnets = [for subnet in aws_subnet.public : subnet.id]
}
resource “aws_lb_listener” “app” {
load_balancer_arn = aws_lb.app.arn
port = 80
protocol = “HTTP”
default_action {
type = “forward”
target_group_arn = aws_lb_target_group.app.arn
}
}
resource “aws_lb_target_group” “app” {
name = “app-tg”
port = 80
protocol = “HTTP”
vpc_id = aws_vpc.main.id
}
Implementing Security Groups for VPC and Load Balancer
Security groups act as virtual firewalls, controlling inbound and outbound traffic. Define security groups for your VPC and ALB to restrict access.
resource “aws_security_group” “vpc” {
name = “vpc-sg”
vpc_id = aws_vpc.main.id
ingress {
from_port = 0
to_port = 0
protocol = “-1”
cidr_blocks = [“0.0.0.0/0”]
}
egress {
from_port = 0
to_port = 0
protocol = “-1”
cidr_blocks = [“0.0.0.0/0”]
}
}
resource “aws_security_group” “lb” {
name = “lb-sg”
vpc_id = aws_vpc.main.id
ingress {
from_port = 80
to_port = 80
protocol = “tcp”
cidr_blocks = [“0.0.0.0/0”]
}
egress {
from_port = 0
to_port = 0
protocol = “-1”
cidr_blocks = [“0.0.0.0/0”]
}
}
Encapsulating Infrastructure Configuration with Terraform Modules
Terraform modules allow you to encapsulate and reuse configurations. To simplify your main configuration, create modules for standard components like VPCs, EC2 instances, and load balancers.
Automating Infrastructure Provisioning with GitHub Actions
GitHub Actions can automate the deployment of your Terraform configurations. To define the CI/CD pipeline, create a workflow file in your GitHub repository.
Configuring AWS Credentials and Terraform Backend
Use AWS credentials securely by configuring the OIDC provider in AWS and storing secrets in GitHub. Also, configure the Terraform backend to store the state file securely.
Executing Terraform Commands in a Workflow
Define steps in your GitHub Actions workflow to initialize Terraform, plan the changes, and apply the configuration.
name: Terraform
on:
push:
branches:
– main
jobs:
deploy:
runs-on: ubuntu-latest
steps:
– name: Checkout code
uses: actions/checkout@v2
– name: Setup Terraform
uses: hashicorp/setup-terraform@v1
– name: Terraform Init
run: terraform init
– name: Terraform Plan
run: terraform plan
– name: Terraform Apply
run: terraform apply -auto-approve
Benefits of Infrastructure Automation: Consistency, Repeatability, and Reduced Errors
Automating infrastructure provisioning ensures consistency and repeatability and reduces the likelihood of human errors. It also allows for rapid scaling and updating of infrastructure while maintaining a reliable and secure environment.
Challenges Encountered: IAM Permissions and User Data Script Encoding
Some common challenges include managing IAM permissions to ensure access without over-provisioning and encoding user data scripts correctly to avoid errors during instance initialization.
Conclusion: Secure and Scalable High Availability EC2 Infrastructure on AWS
You can achieve a secure, scalable, and resilient environment by automating the provision of high-availability EC2 infrastructure on AWS using Terraform and GitHub Actions. This approach enhances productivity and ensures a consistent and reliable deployment process.
References
Integrating with GitHub Actions – CI/CD pipeline to deploy a Web App to Amazon EC2
Scaling IaC and CI/CD pipelines with Terraform, GitHub Actions, and AWS Proton