Best Amazon AWS DevOps Tools: A pipeline, source code repository, build, and deployment with Amazon Web Services
When I started my software career around 20+ years back the infrastructure (Software and Hardware) for any kind of development and deployment had to be procured.
This included placing orders with the vendor for servers, waiting for a certain amount of time to get the same, once delivered server space had to be reserved, installation of the server, operating system, storage configuration, etc. We also had to be worried about performance, availability (24*7), maintenance, network, etc.
This was too much effort involved in bringing up a server for activities of development and deployment of applications.
Table of Contents:
Evolution of Cloud Computing
Things had to change with the evolution of Cloud Computing which means you access all your applications and databases over the internet. So Cloud Computing providers maintain all of the hardware that is needed to run your web application.
All the resources with an appropriate configuration that you need to host your application are available within a click. Time is drastically reduced for making available the resources for developers. Most importantly you pay only for what you need to use.
The focus for developers using Cloud-based services is only on what they need to work on our projects and not to worry about infrastructure availability. I am not getting into types of cloud computing (IaaS, PaaS, SaaS) there are tons of information available over the internet to describe what they are.
There are many cloud providers. But the 3 most popular ones that I have worked with are:
- Amazon Web Services
- Microsoft Azure
- Google Cloud
In this section, I will put my focus on tools for a pipeline, source code repository, and build, and deployment with Amazon Web Services. Not to forget that teams still use DevOps tools like Jenkins, Git, Maven, and others.
So it is imperative that while teams may want to move their assets and artifacts to cloud infrastructure we also need to maximize their existing investments in tools and data with integrations/migrations as far as possible.
Click here to learn about AWS and the various services for Architects, Developers, and SysOPS. We will use the free account for the tools mentioned but of course, in a production environment, you will need to procure the services for use.
Build and Deployment using AWS Tools
From a Build and Deployment point of view, we will look at the following AWS services
- AWS CodePipeline
- AWS CodeCommit
- AWS CodeBuild
- AWS CodeDeploy
#1) AWS CodePipeline
AWS CodePipeline is similar to the Jenkins Pipeline which helps to have a visual view of the end-to-end delivery process.
So in a CodePipeline, you will typically configure the following
- Source Code Repository: So your source code would need to be either in AWS CodeCommit or GitHub repository.
- Build Service: AWS CodeBuild details will be configured as part of the pipeline.
- Deploy: AWS CodeDeploy will be configured into the pipeline.
- During the deployment process to different environments if any approvals are needed they could be configured as well
So if there is a code change by the developer the visual representation of Build and Deploy can be seen to be automated.
Source code repository configuration in AWS CodePipeline
Build configuration in AWS CodePipeline which uses Maven build
Deployment configuration in AWS CodePipeline
Complete Execution is seen in AWS CodePipeline
#2) AWS CodeCommit
AWS CodeCommit is a secure online version control service that hosts private Git repositories. A team need not maintain their own version control repository instead they use AWS CodeCommit to store their source code or even binaries like the WAR/JAR/EAR files generated out of the build.
With AWS CodeCommit you create a repository and every developer will clone it to their local machine, add files to it and push it back to the AWS CodeCommit repository. One uses the standard GIT commands with the AWS CodeCommit repository.
For E.g. once the AWS CodeCommit repository is cloned to the local machine you would use commands like ‘git pull’, ‘git add’, ‘git commit’, ‘git push’ etc.
Illustrative AWS CodeCommit empty repository created
Clone the repository to the local machine
Files added to AWS CodeCommit repository
#3) AWS CodeBuild
As we have seen the source code and other project artifacts are stored in the AWS CodeCommit repository.
To implement Continuous Integration AWS CodeBuild like Jenkins fetches the latest changes of the source code from AWS CodeCommit or GitHub repository as configured and based on the build specification YAML file (created as buildspec.yml) the commands are run based on the four phases like Install, Pre-build, Build and Post-build.
Once the build is completed the artifacts (WAR/ZIP/JAR/EAR) are stored in the AWS Storage which is an S3 bucket.
Sample buildspec.yml file
version: 0.2 phases: install: commands: - echo Nothing in the install phase... pre_build: commands: - echo Nothing in the pre_build phase... build: commands: - echo Build started on `date` - mvn clean install post_build: commands: - echo Build completed on `date` artifacts: files: - target/HelloWorld-Maven.warSample AWS Codebuild project
Build Success
Artifact (WAR file) copied to S3 bucket
#4) AWS CodeDeploy
As the name suggests AWS Codedeploy is the deployment service that automates the deployment of the application (in this case WAR file) to the Amazon EC2 Linux or Windows instances.
Since we now have the artifacts stored in the S3 bucket which was completed using AWS CodeBuild the artifacts are then picked up from the S3 bucket and deployed appropriately to the app server Tomcat or JBoss etc. in the AWS EC2 instance provisioning.
AWS CodeDeploy depends on a YAML file called appspec.yml which has instructions on the deployment to the EC2 instances.
Sample appspec.yml file where the index.html file is copied and deployed to the Apache server
version:10.0 os:linux files: -source: /opt/deploy/index.html destination:/var/www/html/ hooks: BeforeInstall: -location:scripts/before_install runas:niranjan AfterInstall: -location:scripts/restart_server runas:niranjanbefore_install script
restart_server script
GitHub repo of all files needed to run AWS CodeDeploy
Deployment execution in AWS CodeDeploy
Jenkins Integration with AWS Services
As mentioned earlier, nowadays teams are using Jenkins much as the defacto CI tool, and in most cases,s they would not really like to move away from it but rather integrate with the AWS services which we discussed. While there are certain procedures involved and I have shown screenshots of the integration.
#1) Jenkins integration with AWS CodeCommit
#2) Jenkins integration with AWS CodeBuild
#3) Jenkins integration with AWS CodeDeploy
Putting it All Together for AWS DevOps Stack:
The stack looks below for the AWS services that are discussed above.
Hope this tutorial on, tools for a pipeline, source code repository, and build and deployment with Amazon Web Services, was helpful to you.
Was this helpful?
Thanks for your feedback!
