Jenkins is a powerful open-source automation server widely used for continuous integration and continuous delivery (CI/CD) in software development. To maximize Jenkins’ efficiency in a production environment, deploying it on a scalable master-slave architecture on AWS EC2 is a common approach. This guide walks you through setting up Jenkins on AWS EC2 with a master-slave architecture, ensuring your system is ready for high-demand workflows.

Introduction to Jenkins on AWS EC2

Deploying Jenkins on AWS EC2 provides scalability, reliability, and flexibility. By setting up a master-slave architecture, you can distribute workloads across multiple slave nodes, improving performance and efficiency. This architecture allows you to handle larger builds and execute jobs in parallel, making your CI/CD pipeline more robust.

Preparing the Environment

Before diving into the Jenkins setup, ensure your AWS environment is configured correctly:

  1. AWS Account: Create or log in to your AWS account.

  2. VPC and Subnets: Ensure you have a VPC with appropriate subnets where your EC2 instances will be deployed.

  3. Security Groups: Set up security groups allowing SSH (port 22) and HTTP (port 8080) access to your instances.

Creating Multiple EC2 Instances for Jenkins Setup

To start, you’ll need to create EC2 instances that will serve as your Jenkins master and slave nodes:

  1. Master Instance: Launch an EC2 instance (e.g., t2.medium) to host the Jenkins master.

  2. Slave Instances: Launch additional EC2 instances for the Jenkins slave nodes. Choose instance types based on the anticipated workload.

Essential Links for EC2 Creation and Initial Jenkins Setup

Here are some essential links to guide you through creating EC2 instances and setting up Jenkins:

Initializing Jenkins on the Master Instance

After your EC2 instance is up and running:

  1. SSH into the Master Instance: Access your instance using SSH.

  2. Install Jenkins: Follow the Jenkins installation steps for your OS (Ubuntu, Amazon Linux, etc.).

  3. Start Jenkins: Run Jenkins and ensure it is accessible on port 8080.

Accessing Jenkins through the Browser

Once Jenkins is up and running on your master instance:

  1. Browser Access: Open your browser and navigate to http://<your-ec2-public-ip>:8080.

  2. Unlock Jenkins: Use the initial administrator password to unlock Jenkins.

Logging into Jenkins and Navigating to Node Management

After logging in:

  1. Navigate to Node Management: Go to Manage Jenkins > Manage Nodes and Clouds.

  2. Add New Nodes: Begin configuring your slave nodes by adding new nodes.

Configuring the First Slave Node

To configure a slave node:

  1. Add a New Node: Click on New Node, name it, and choose Permanent Agent.

  2. Customize Node Settings: Specify labels, remote root directory, and the number of executors.

Adding a New Node as a Permanent Agent

In the node configuration:

  1. Permanent Agent Setup: Choose Permanent Agent when adding the new node.

  2. Configure Node Details: Enter specific details, such as labels for job restrictions and execution limits.

Customizing Node Settings for Specific Labels and Execution Limits

Further, customize the node settings:

  1. Label Assignment: Assign labels to nodes to control which jobs run on which nodes.

  2. Execution Limits: Set the maximum number of concurrent jobs the node can handle.

Connecting the Slave Node to the Master

To connect the slave node:

  1. Slave Agent Setup: Download the agent.jar file from Jenkins to your slave node.

  2. Agent Execution: Execute the agent on the slave node using the command provided by Jenkins.

Executing Commands on the Slave Node for Connection

On your slave node:

  1. Run Connection Command: Use the command provided by Jenkins to establish a connection between the slave and the master.

Verifying the Online Status of the Slave Node

Back in Jenkins:

  1. Check Node Status: Ensure the slave node shows as “Online” under Manage Nodes.

Repeating the Process for Additional Slave Nodes

For additional nodes:

  1. Add More Slave Nodes: Follow the same process to add and configure more slave nodes.

  2. Verify Each Connection: Ensure each slave node is properly connected and ready.

Adding More Slave Nodes Following the Same Procedure

Continue adding nodes:

  1. Node Configuration: Use consistent configurations for each new node.

  2. Labeling and Limits: Assign appropriate labels and execution limits.

Ensuring Each Slave Node is Properly Connected and Ready

Double-check:

  1. Status Verification: Confirm that all slave nodes are online and connected to the master.

Creating a Sample Jenkins Project

To test the setup:

  1. New Project Setup: Create a new Jenkins project.

  2. Label Assignment: Restrict the project to specific slave nodes using labels.

Initiating a New Jenkins Project

In Jenkins:

  1. Project Configuration: Add basic build steps (e.g., shell commands) for demonstration.

  2. Label Restrictions: Use labels to control where the project runs.

Configuring Project Restrictions Based on Labels

Ensure that:

  1. Node Restrictions: Jobs are only executed on specified slave nodes using labels.

Adding Basic Build Steps for Demonstration

For the project:

  1. Build Steps: Add simple steps to test node execution (e.g., echo “Hello, Jenkins”).

Testing the Master-Slave Workflow and Triggering a Build on the Jenkins Dashboard

Trigger the build:

  1. Build Execution: Start the build and observe how the job is delegated from the master to a slave node.

Observing Job Delegation from Master to Slave Nodes

Watch the process:

  1. Job Delegation: Verify that the master node delegates the job to a connected slave node based on the project labels.

Evaluating the Efficiency of the Master-Slave Architecture

Finally, assess the architecture:

  1. Performance Review: Assess the performance and efficiency improvements the master-slave setup offers.

  2. Scalability: Consider the scalability benefits of adding more nodes as needed.

Conclusion

Deploying Jenkins on AWS EC2 with a master-slave architecture significantly improves your CI/CD pipeline’s efficiency and scalability. Following the steps outlined in this guide, you can create a robust environment capable of handling complex builds and large-scale projects.

References

Set Up a Jenkins Build Server

Cost Optimize your Jenkins CI/CD pipelines using EC2 Spot Instances