# Jenkins Pipeline with Maven, SonarQube, Docker, and AWS ECR ## Jenkins Install ### Controller and Agent **The controller** is the main Jenkins instance that will co-ordinate the connections and tooling for the agents. **Agents** are the workhorse instances used to run jobs and pipelines. It can have various tools and unique tags to build different kinds of projects. Let's set up a Jenkins controller on AWS EC2. It is not recommended to run jobs on the controller for security reasons. But this is a single instance setup and will run jobs... I will also set up a Caddy reverse proxy with SSL to direct traffic to Jenkins. Caddy is chosen for its simplicity, performance, and security. [Read more about Caddy.](https://caddyserver.com/features) EC2 Instance settings: * Instance type: t3.small * AMI: Ubuntu Server 22.04 LTS * Select / Create an "SSH Key pair" * Select / Create a "Security Group" with the following inbound rules: * Port 22, SSH - Your IP * Port 80 - Anywhere * Port 443 - Anywhere * Storage: increase it by a few GB, I used 15 GB * Advanced details: * Spot instances (Optional, to save costs) * IAM instance profile (Optional) * User data: [Enter this script to install Jenkins automatically](https://raw.githubusercontent.com/melvincv/blog-melvincv/main/jenkins/udemy-202403/jenkins/install.sh) ### Add to Route 53 Add a DNS A record to Route 53 for your Jenkins domain to point to your IP address. ### Log in to Jenkins Set the hostname ```plaintext sudo hostnamectl set-hostname jenkins-svr sudo vim /etc/hosts bash ``` Check the cloud-init log to confirm that the script execution was a success. ```plaintext tail -f /var/log/cloud-init-output.log ``` Start the caddy server ```plaintext sudo systemctl start caddy journalctl -u caddy -f --- "msg":"certificate obtained successfully" ``` Access Jenkins in the browser Eg: [https://jenkins.aws.melvincv.com/](https://jenkins.aws.melvincv.com/) Unlock Jenkins ```plaintext sudo cat /var/lib/jenkins/secrets/initialAdminPassword ``` Install Suggested Plugins. Get the **JDK** and **Maven** paths and add them to **Manage Jenkins > Tools** ```plaintext $ mvn --version Apache Maven 3.9.6 (bc0240f3c744dd6b6ec2920b3cd08dcc295161ae) Maven home: /opt/apache-maven Java version: 17.0.10, vendor: Private Build, runtime: /usr/lib/jvm/java-17-openjdk-amd64 ``` Name: openjdk17 ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1713156998647/5d743546-d657-4f3c-8d80-641601b700db.png align="center") **Name**: maven3.9 ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1713157045962/b763fc4c-054c-421d-ab44-c2333e5d4f02.png align="center") ```plaintext sudo usermod -aG docker jenkins sudo systemctl restart jenkins ``` ### Install Jenkins Plugins Manage > Plugins > Available * sonarqube scanner * AWS Credentials * Amazon ECR * docker * docker pipeline * ansible # Sonarqube Install Create a new EC2 instance for Sonarqube. EC2 Instance settings: * Instance type: t3.medium * AMI: Ubuntu Server 22.04 LTS * Select / Create an "SSH Key pair" * Select / Create a "Security Group" with the following inbound rules: * Port 22, SSH - Your IP * Port 80 - Anywhere * Port 443 - Anywhere * Storage: increase it by a few GB, I used 15 GB * Advanced details: * Spot instances (Optional, to save costs) * IAM instance profile (Optional) * User data: [Enter this script to install prerequisites automatically](https://raw.githubusercontent.com/melvincv/blog-melvincv/main/jenkins/udemy-202403/sonarqube/prereq.sh) ### Log in to sonarqube Set hostname ```plaintext sudo hostnamectl set-hostname sonarqube sudo vim /etc/hosts bash ``` Log in to the instance and create a folder and the docker-compose file ```plaintext mkdir sonar; cd sonar cat > compose.yml ``` Paste the [contents of this file](https://github.com/melvincv/blog-melvincv/blob/main/jenkins/udemy-202403/sonarqube/compose.yml) and press Ctrl+D Create a .env file in the same folder with the Postgres password: ```plaintext POSTGRES_PASSWORD=xxxxxxxxxxxxxxxxxxx ``` Start the compose stack: ```plaintext docker compose pull docker compose up -d ``` Wait for a minute for it to initialize... ```plaintext ubuntu@sonarqube:~/sonar$ docker compose ps NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS sonar-adminer-1 adminer "entrypoint.sh php -…" adminer About a minute ago Up About a minute 0.0.0.0:10000->8080/tcp, :::10000->8080/tcp sonar-db-1 postgres:12 "docker-entrypoint.s…" db About a minute ago Up About a minute 5432/tcp sonar-sonarqube-1 sonarqube:lts-community "/opt/sonarqube/dock…" sonarqube About a minute ago Up About a minute 0.0.0.0:9000->9000/tcp, :::9000->9000/tcp ``` In AWS Route 53, add an A record to point to your instance IP Address. Wait for a few minutes for the DNS entries to take effect... Then start the Caddy server and check its logs to see if the Let's Encrypt certificate got issued: ```plaintext sudo systemctl start caddy journalctl -u caddy -f --- "msg":"certificate obtained successfully" ``` Log in with the default credentials and change your password to a strong one: > user: admin pass: admin In the Sonarqube dashboard, create a token for Jenkins: ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1711373672696/7fc3b925-9ce3-4281-ac18-e92a60fd64b9.png align="center") ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1711373701449/d376abb5-5abd-4923-91f2-6ed4dd8df1c9.png align="center") In Jenkins, add a credential of type "Secret text" with ID "sonar-token" Go to "Manage Jenkins > System > Sonarqube installations" and "Add a Sonarqube server." > Name: sonar > > URL: [https://sonar.aws.melvincv.com/](https://sonar.aws.melvincv.com/) > > Token: sonar-token In Sonarqube, create a webhook that allows it to ping Jenkins after the code passes the quality gate. Administration > Config > Webhooks ```plaintext https://jenkins.aws.melvincv.com/sonarqube-webhook/ ``` Make sure you replace `jenkins.aws.melvincv.com` with your jenkins URL. Slash at the end of the URL is important. To login and push to AWS ECR, we need Access Key ID and Secret Key of a user with AWS ECR Full Access permissions. Create "AWS Credentials" with ID `jenkins-ecr-login-credentials` Add your Access Key ID and Secret Key. ## AWS ECR Create a new Private repository called `springbootapp` ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1713165929894/88c2be64-0339-4b3b-9980-fe8bd49070bf.png align="center") Create an IAM user with permission to read and write to the repo. ([AmazonEC2ContainerRegistryFullAccess](https://us-east-1.console.aws.amazon.com/iam/home?region=ap-northeast-1#/policies/details/arn%3Aaws%3Aiam%3A%3Aaws%3Apolicy%2FAmazonEC2ContainerRegistryFullAccess)) ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1713166076366/0d120314-c666-459f-a6e5-2dea43d59d16.png align="center") ## Create a Jenkins Job Create a Job of type "Pipeline" Tick the option "**GitHub hook trigger for GITScm polling**" Paste the Pipeline file and save it. ### Create a Github Webhook You may create a webhook for the app repo so that Github can ping Jenkins when a commit is pushed to the repo branch. **Payload URL:**[https://jenkins.aws.melvincv.com/github-webhook/](https://jenkins.aws.melvincv.com/github-webhook/) 👷‍♂️ Build Now. Sonarqube Project will be added to the Projects page by Jenkins. ⬇️ ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1711374542994/a63ff53d-b9fc-4f31-8f30-588474502f31.png align="center") ## Email Notifications Send email notifications of the build status to your email ID using AWS SES. Manage Jenkins > System > Scroll to end ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1711374783405/50587e88-f170-4077-a591-fc843669c6e3.png align="center") ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1711374872822/31f8c7a7-4b6c-4ae2-bdeb-00aaeed268ea.png align="center") Scroll up and add the From address ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1711374936644/ed18c204-3acd-4a81-b8ca-6ed9069c5a0a.png align="center") ## CD using Ansible The "CD with Ansible" Step would have failed while building the job in Jenkins. Start / Prepare a Target CD Instance with the following user data: ```plaintext #!/bin/bash # Upgrade packages sudo apt update && sudo apt upgrade -y # Install Docker curl -fsSL https://get.docker.com -o install-docker.sh sudo sh install-docker.sh sudo usermod -aG docker $USER # Install AWS CLI v2 sudo apt install -y unzip curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" unzip awscliv2.zip sudo ./aws/install ``` Update the IP address in the inventory file. Create Jenkins Credentials of type "SSH Username with private key" with ID "cd-server-creds" Create an [Ansible Playbook (present in repo)](https://github.com/melvincv/blog-melvincv/blob/main/jenkins/udemy-202403/ansible/playbook.yml) that pulls the docker image from AWS ECR and runs a container from it. ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1713165745425/ab653171-b126-41b3-b958-fb2113707661.png align="center") Finally, you should have the app running as a Docker container on port 8080 of the target instance. ## GitHub Repos Scripts: [https://github.com/melvincv/blog-melvincv/tree/main/jenkins/udemy-202403](https://github.com/melvincv/blog-melvincv/tree/main/jenkins/udemy-202403) App and Ansible files: [https://github.com/melvincv/springboot-maven-micro](https://github.com/melvincv/springboot-maven-micro)