Top AWS DevOps Tools for Cloud Build and Deployment

Best Amazon AWS DevOps Tools: A pipeline, source code repository, build and deployment with Amazon Web Services

When I started my software career around 20+ years back the infrastructure (Software and Hardware) for any kind of development and deployment had to be procured.

This included placing order with the vendor for servers, waiting for a certain amount of time to get the same, once delivered server space had to be reserved, installation of the server, operating system, storage configuration etc. We also had to be worried about performance, availability (24*7), maintenance, network etc.

This was too much effort involved in bringing up a server for activities of development and deployment of applications.

BEST AWS DEVOPS TOOLS

Evolution of Cloud Computing

Things had to change with the evolution of Cloud Computing which means you access all your applications and databases over the internet. So Cloud Computing providers maintain all of the hardware that is needed to run your web application.

All the resources with an appropriate configuration that you need to host your application are available within a click. Time is drastically reduced for making available the resources for developers. Most importantly you pay only for what you need to use.

The focus for developers using Cloud-based services is only on what they need to work on our projects and not to worry about infrastructure availability. I am not getting into types of cloud computing (IaaS, PaaS, SaaS) there are tons of information available over the internet to describe what they are.

There are many cloud providers. But the 3 most popular ones that I have worked with are:

  • Amazon Web Services
  • Microsoft Azure
  • Google Cloud

In this section, I will put my focus on tools for a pipeline, source code repository, build and deployment with Amazon Web Services. Not to forget that teams still use DevOps tools like Jenkins, Git, Maven, and others.

So it is imperative that while teams may want to move their assets and artifacts to cloud infrastructure we also need to maximize their existing investments in tools and data with integrations/migrations as far as possible.

Click here to learn about AWS and the various services for Architects, Developers, and SysOPS. We will use the free account for the tools mentioned but of course, in a production environment, you will need to procure the services for use.

Build and Deployment using AWS Tools

From a Build and Deployment point of view, we will look at the following AWS services

  • AWS CodePipeline
  • AWS CodeCommit
  • AWS CodeBuild
  • AWS CodeDeploy

1. AWS CodePipeline

AWS CodePipeline is similar to the Jenkins Pipeline which helps to have a visual view of the end to end delivery process.

So in a CodePipeline, you will typically configure the following

  • Source Code Repository – So your source code would need to be either in AWS CodeCommit or GitHub repository.
  • Build Service – AWS CodeBuild details will be configured as part of the pipeline.
  • Deploy – AWS CodeDeploy will be configured into the pipeline.
  • During the deploy process to different environments if any approvals are needed they could be configured as well

So if there is a code change by the developer the visual representation of Build and Deploy can be seen to be automated.

Source code repository configuration in AWS CodePipeline

Source code repository configuration

Build configuration in AWS CodePipeline which uses Maven build

Build configuration in AWS CodePipeline

Deployment configuration in AWS CodePipeline

Deployment configuration in AWS CodePipeline

Complete Execution is seen in AWS CodePipeline

Complete Execution is seen in AWS CodePipeline

2. AWS CodeCommit

AWS CodeCommit is a secure online version control service which hosts private Git repositories. A team need not maintain their own version control repository instead they use AWS CodeCommit to store their source code or even binaries like the WAR/JAR/EAR files generated out of the build.

With AWS CodeCommit you create a repository and every developer will clone it to their local machine, add files to it and push it back to the AWS CodeCommit repository. One uses the standard GIT commands with the AWS CodeCommit repository.

For E.g. once the AWS CodeCommit repository is cloned to local machine you would use commands like ‘git pull’, ‘git add’, ‘git commit’, ‘git push’ etc..

Illustrative AWS CodeCommit empty repository created

AWS CodeCommit empty repository created

Clone the repository to the local machine

Clone the repository to the local machine

Files added to AWS CodeCommit repository

Files added to AWS CodeCommit repository

3. AWS CodeBuild

As we have seen the source code and other project artifacts are stored in AWS CodeCommit repository.

To implement Continuous Integration AWS CodeBuild like Jenkins fetches the latest changes of the source code from AWS CodeCommit or GitHub repository as configured and based on the build specification YAML file (created as buildspec.yml) the commands are run based on the four phases like Install, Pre-build, Build and Post-build.

Once the build is completed the artifacts (WAR/ZIP/JAR/EAR) are stored in the AWS Storage which is an S3 bucket.

Sample buildspec.yml file

version: 0.2
phases:
install:
commands:
- echo Nothing in the install phase...
pre_build:
commands:
- echo Nothing in the pre_build phase...
build:
commands:
- echo Build started on `date`
- mvn clean install
post_build:
commands:
- echo Build completed on `date`
artifacts:
files:
- target/HelloWorld-Maven.war

Sample AWS Codebuild project

Sample AWS Codebuild project

Build Success

Build Success

Artifact (WAR file) copied to S3 bucket

Artifact (WAR file) copied to S3 bucket

4. AWS CodeDeploy

As the name suggests AWS Codedeploy is the deployment service which automates the deployment of the application (in this case WAR file) to the Amazon EC2 Linux or Windows instances.

Since we now have the artifacts stored in S3 bucket which was completed using AWS CodeBuild the artifacts are then picked up from the S3 bucket and deployed appropriately to the app server Tomcat or JBoss etc. in the AWS EC2 instance provisioning.

AWS CodeDeploy depends on a YAML file called appspec.yml which has instructions on the deployment to the EC2 instances.

Sample appspec.yml file where the index.html file is copied and deployed to the Apache server

version:10.0
os:linux
files:
-source: /opt/deploy/index.html
destination:/var/www/html/
hooks:
BeforeInstall:
-location:scripts/before_install
runas:niranjan
AfterInstall:
-location:scripts/restart_server
runas:niranjan

before_install script

before_install script

restart_server script

restart_server script

GitHub repo of all files needed to run AWS CodeDeploy

GitHub repo of all files needed to run AWS CodeDeploy

Deployment execution in AWS CodeDeploy

Deployment execution in AWS CodeDeploy

Jenkins Integration with AWS Services

As mentioned earlier, nowadays teams are using Jenkins much as the defacto CI tool and in most case,s they would not really like to move away from it but rather integrate with the AWS services which we discussed. While there are certain procedures involved and I have shown screenshots of the integration.

1. Jenkins integration with AWS CodeCommit

Jenkins integration with AWS CodeCommit

2. Jenkins integration with AWS CodeBuild

Jenkins integration with AWS CodeBuild

Jenkins integration with AWS CodeBuild

3. Jenkins integration with AWS CodeDeploy

Jenkins integration with AWS CodeDeploy

Jenkins integration with AWS CodeDeploy

Putting it All Together for AWS DevOps Stack:

The stack looks below for the AWS services that are discussed above.

AWS services

Hope this tutorial on, tools for a pipeline, source code repository, build and deployment with Amazon Web Services, was helpful to you.