Table of contents
Jenkins Install
Controller and Agent
The controller is the main Jenkins instance that will co-ordinate the connections and tooling for the agents.
Agents are the workhorse instances used to run jobs and pipelines. It can have various tools and unique tags to build different kinds of projects.
Let's set up a Jenkins controller on AWS EC2. It is not recommended to run jobs on the controller for security reasons. But this is a single instance setup and will run jobs... I will also set up a Caddy reverse proxy with SSL to direct traffic to Jenkins. Caddy is chosen for its simplicity, performance, and security. Read more about Caddy.
EC2 Instance settings:
Instance type: t3.small
AMI: Ubuntu Server 22.04 LTS
Select / Create an "SSH Key pair"
Select / Create a "Security Group" with the following inbound rules:
Port 22, SSH - Your IP
Port 80 - Anywhere
Port 443 - Anywhere
Storage: increase it by a few GB, I used 15 GB
Advanced details:
Spot instances (Optional, to save costs)
IAM instance profile (Optional)
User data: Enter this script to install Jenkins automatically
Add to Route 53
Add a DNS A record to Route 53 for your Jenkins domain to point to your IP address.
Log in to Jenkins
Set the hostname
sudo hostnamectl set-hostname jenkins-svr
sudo vim /etc/hosts
bash
Check the cloud-init log to confirm that the script execution was a success.
tail -f /var/log/cloud-init-output.log
Start the caddy server
sudo systemctl start caddy
journalctl -u caddy -f
---
"msg":"certificate obtained successfully"
Access Jenkins in the browser
Eg: https://jenkins.aws.melvincv.com/
Unlock Jenkins
sudo cat /var/lib/jenkins/secrets/initialAdminPassword
Install Suggested Plugins.
Get the JDK and Maven paths and add them to Manage Jenkins > Tools
$ mvn --version
Apache Maven 3.9.6 (bc0240f3c744dd6b6ec2920b3cd08dcc295161ae)
Maven home: /opt/apache-maven
Java version: 17.0.10, vendor: Private Build, runtime: /usr/lib/jvm/java-17-openjdk-amd64
Name: openjdk17
Name: maven3.9
sudo usermod -aG docker jenkins
sudo systemctl restart jenkins
Install Jenkins Plugins
Manage > Plugins > Available
sonarqube scanner
AWS Credentials
Amazon ECR
docker
docker pipeline
ansible
Sonarqube Install
Create a new EC2 instance for Sonarqube.
EC2 Instance settings:
Instance type: t3.medium
AMI: Ubuntu Server 22.04 LTS
Select / Create an "SSH Key pair"
Select / Create a "Security Group" with the following inbound rules:
Port 22, SSH - Your IP
Port 80 - Anywhere
Port 443 - Anywhere
Storage: increase it by a few GB, I used 15 GB
Advanced details:
Spot instances (Optional, to save costs)
IAM instance profile (Optional)
User data: Enter this script to install prerequisites automatically
Log in to sonarqube
Set hostname
sudo hostnamectl set-hostname sonarqube
sudo vim /etc/hosts
bash
Log in to the instance and create a folder and the docker-compose file
mkdir sonar; cd sonar
cat > compose.yml
Paste the contents of this file and press Ctrl+D
Create a .env file in the same folder with the Postgres password:
POSTGRES_PASSWORD=xxxxxxxxxxxxxxxxxxx
Start the compose stack:
docker compose pull
docker compose up -d
Wait for a minute for it to initialize...
ubuntu@sonarqube:~/sonar$ docker compose ps
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
sonar-adminer-1 adminer "entrypoint.sh php -…" adminer About a minute ago Up About a minute 0.0.0.0:10000->8080/tcp, :::10000->8080/tcp
sonar-db-1 postgres:12 "docker-entrypoint.s…" db About a minute ago Up About a minute 5432/tcp
sonar-sonarqube-1 sonarqube:lts-community "/opt/sonarqube/dock…" sonarqube About a minute ago Up About a minute 0.0.0.0:9000->9000/tcp, :::9000->9000/tcp
In AWS Route 53, add an A record to point to your instance IP Address.
Wait for a few minutes for the DNS entries to take effect... Then start the Caddy server and check its logs to see if the Let's Encrypt certificate got issued:
sudo systemctl start caddy
journalctl -u caddy -f
---
"msg":"certificate obtained successfully"
Log in with the default credentials and change your password to a strong one:
user: admin pass: admin
In the Sonarqube dashboard, create a token for Jenkins:
In Jenkins, add a credential of type "Secret text" with ID "sonar-token"
Go to "Manage Jenkins > System > Sonarqube installations" and "Add a Sonarqube server."
Name: sonar
URL: https://sonar.aws.melvincv.com/
Token: sonar-token
In Sonarqube, create a webhook that allows it to ping Jenkins after the code passes the quality gate. Administration > Config > Webhooks
https://jenkins.aws.melvincv.com/sonarqube-webhook/
Make sure you replace jenkins.aws.melvincv.com with your jenkins URL. Slash at the end of the URL is important.
To login and push to AWS ECR, we need Access Key ID and Secret Key of a user with AWS ECR Full Access permissions.
Create "AWS Credentials" with ID jenkins-ecr-login-credentials
Add your Access Key ID and Secret Key.
AWS ECR
Create a new Private repository called springbootapp
Create an IAM user with permission to read and write to the repo. (AmazonEC2ContainerRegistryFullAccess)
Create a Jenkins Job
Create a Job of type "Pipeline"
Tick the option "GitHub hook trigger for GITScm polling"
Paste the Pipeline file and save it.
Create a Github Webhook
You may create a webhook for the app repo so that Github can ping Jenkins when a commit is pushed to the repo branch.
Payload URL:https://jenkins.aws.melvincv.com/github-webhook/
👷♂️ Build Now.
Sonarqube Project will be added to the Projects page by Jenkins. ⬇️
Email Notifications
Send email notifications of the build status to your email ID using AWS SES.
Manage Jenkins > System > Scroll to end
Scroll up and add the From address
CD using Ansible
The "CD with Ansible" Step would have failed while building the job in Jenkins.
Start / Prepare a Target CD Instance with the following user data:
#!/bin/bash
# Upgrade packages
sudo apt update && sudo apt upgrade -y
# Install Docker
curl -fsSL https://get.docker.com -o install-docker.sh
sudo sh install-docker.sh
sudo usermod -aG docker $USER
# Install AWS CLI v2
sudo apt install -y unzip
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install
Update the IP address in the inventory file.
Create Jenkins Credentials of type "SSH Username with private key" with ID "cd-server-creds"
Create an Ansible Playbook (present in repo) that pulls the docker image from AWS ECR and runs a container from it.
Finally, you should have the app running as a Docker container on port 8080 of the target instance.
GitHub Repos
Scripts: https://github.com/melvincv/blog-melvincv/tree/main/jenkins/udemy-202403
App and Ansible files: https://github.com/melvincv/springboot-maven-micro