Hello everyone, in this post, we will discuss how to deploy your web application (nodejs in our case) to the EC2 machine on AWS using GitLab ci/cd.
Here’s a quick diagram you can follow along.
So what we are trying to achieve here? we wanted to push our code to the GitLab (main branch) and that code will be squeezed into the zip file, and then pushed to the s3 bucket. After that, the code pipeline will be triggered when there would be a new version on s3 for our code and after that, it will be used by code deploy to deploy our code into the target group(which is EC2 in our case).
I know it might sound complicated on the first look but I will try to divide it and explain part by part.
Creating an IAM User
Before diving into the creating of our pipeline, We will need an IAM user for that and you might want to add the following permissions:
Please follow the best practices provided by AWS by scoping your IAM policies and tightening your security.
Creating EC2 Instance & Role
Create a Role on IAM and attach the following policies:
Attach that role to your EC2 Instance.
Install CodeDeploy Agent on EC2
Since we will use CodeDeploy we have to install its agent in our EC2 Instance.
Follow this guide to install CodeDeploy:
Create an S3 repository
Please follow this article to create your S3 repository. Don’t forget to enable versioning on them.
Creating an application on AWS CodeDeploy
Since the basic elements are set we will start creating our application on CodeDeploy.
- Go to the CodeDeploy > Applications then click Create Application
- Choose the application name and Compute platform (EC2 in our case).
- After that click on Create deployment group.
- Choose a deployment group name, service role, deployment type, Environment configuration, Deployment settings, Load balancer, and then click on Create deployment group.
Creating a pipeline on AWS CodePipeline
We will create a CodePipeline so whenever there is a new version of our code on the S3 bucket it will be notified and it will push the necessary changes to the CodeDeploy.
- Click on Create Pipeline
- Choose a pipeline name and service role.
- Pick your source provider (S3 in our case). Pick your S3 bucket and enter the format for your zip file (yourcode.zip). Don’t forget to pick AWS CodePipeline.
- Skip build stage.
- For the Deploy stage pick our CodeDeploy and fill the necessary fields.
- Click on Create Pipeline.
Setting up the environment on GitLab
Since sharing your credentials on GitLab-ci.yml is a security flaw, we will follow the best practices provided by GitLab and we will add our variables at GitLab > CI/CD > Variables.
Our GitLab runners will automatically sign you in for your AWS account, so you don’t have to do that manually.
CloudPipelines role in that architecture
CodePipeline detects the change in the revision of the S3 zip file
CodePipeline validates the file
CodePipeline sends a signal that the bundle for CodeDeploy is ready
CodeDeploy role on that architecture
CodeDeploy executes the deployment steps
Start – initialization of the deployment
Application Stop – Executes defined script for hook
DownloadBundle – Gets the bundle file from the S3 repository through the CodePipeline
BeforeInstall – Executes defined script for hook Install – Copies the contents to the deployment location as defined by the files section ofappspec.yml
AfterInstall – Executes defined script for hook
ApplicationStart – Executes defined script for hook
ValidateService – Executes defined script for hook End – Signals the CodePipeline that the deployment has completed successfully
Since our Infrastructure looks okay to me, time to start with gitlab-ci.yml file
image: node:latest # This folder is cached between builds # https://docs.gitlab.com/ee/ci/yaml/index.html#cache cache: paths: - node_modules/ before_script: - apt-get update -qq && apt-get install - apt-get install zip -y # You specify the stages. Those are the steps that GitLab will go through # Order matters. stages: - build Build: # Script to run for deploying an application to AWS when: manual # Manual deployments switch stage: build script: - curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" # Downloading and installing awscli - unzip awscliv2.zip - ./aws/install - npm install - zip -r zipfile.zip . # - aws s3 cp zipfile.zip s3://s3-name # Uploads the zip file to S3 and expects the AWS Code Pipeline/Code Deploy to pick up - aws deploy push --application-name CodeDeployAppName --s3-location s3://s3-name/zipfile.zip # Adding revision to s3 bucket - aws deploy create-deployment --application-name CodeDeployAppName --s3-location bucket=s3-lcoation,key=zipfile.zip,bundleType=zip --deployment-group-name CloudDeploymentGroupName --deployment-config-name CodeDeployDefault.OneAtATime --file-exists-behavior OVERWRITE # Ordering the deployment of the new revision environment: production cache: key: $CI_BUILD_NAME:$CI_BUILD_REF_NAME untracked: true paths: - build/ when: always
So I wanted to show you the two ways of doing it. You can uncomment the
- aws s3 cp zipfile.zip s3://mlm-deployments # Uploads the zip file to S3 and expects the AWS Code Pipeline/Code Deploy to pick up line and upload your zip file to the S3 and wait for CodePipeline to be triggered and send it to the CodeDeploy. But in the second approach (uncommented section) it will deploy directly to the CodeDeploy. I found both of them useful but it is up to you to choose your environment by your needs.
when: manual # Manual deployments switch this code snippet lets you run your pipeline manually. I highly recommend using that switch if you are just adopting the DevOps culture. You can remove that snippet if you wanted to automatically run your pipelines whenever a code change.
After that CodeDeploy has to know what kind of scripts does it needs to run. We will create an appspec.yml file for that.
version: 0.0 os: linux files: - source: / destination: /home/ubuntu/app hooks: BeforeInstall: - location: scripts/before_install.sh timeout: 300 runas: ubuntu AfterInstall: - location: scripts/after_install.sh timeout: 300 runas: ubuntu AppStart: - location: scripts/app_start.sh timeout: 300 runas: ubuntu
After that, we will create the scripts we defined at the appspec.yml file.
#before_install.sh #!/bin/bash HOME=/home/ubuntu APP=$HOME/app LOG=$HOME/deploy.log sudo apt-get update -qq && sudo apt-get install #add any necessary bash needs here. /bin/echo "## Before Install Completed ##" >> $LOG
#after_install.sh #!/bin/bash HOME=/home/ubuntu/ LOG=$HOME/deploy.log # verify that the application directory has the correct owner/group /usr/bin/sudo /bin/chown -R ubuntu:ubuntu /home/ubuntu/app #add any necessary bash needs here. /bin/echo " ## After Install Completed ##" >> $LOG
#app_start.sh #!/bin/bash HOME=/home/ubuntu APP=$HOME/app # CMD=$APP/app.sh LOG=$HOME/deploy.log cd $APP sleep 5 sudo chown ubuntu:ubuntu /home/ubuntu/.pm2/rpc.sock /home/ubuntu/.pm2/pub.sock sudo pm2 reload all --force --update-env /bin/echo " ## App Start Completed ##" >> $LOG
Don’t forget to add additional scripts for your needs and modify the scripts for your architecture.
Successful deployment screenshots: