DevOps Testing Tutorial: How DevOps will Impact QA Testing?

DevOps Testing Tutorial:

A recent RightScale survey has found that 54% of the companies have adopted DevOps and the interest around DevOps is increasing rapidly.

In this article, we will learn how this new software development methodology will impact QA and how the QA function as a whole should evolve to embrace this change.

In this article, we will learn more about DevOps and how it will impact QA and its functions.

[RightScale survey]

What You Will Learn:

What is DevOps?

DevOps – is a combination of Development & Operations – it is a software development methodology which looks to integrate all the software development functions from development to operations within the same cycle.

This calls for higher level of coordination within the various stakeholders in the software development process (namely Development, QA & Operations)

DevOps Cycle

An ideal DevOps cycle would start from:

Obviously, this approach places a great emphasis on automation of build, deployment, and testing. Use of Continuous Integration (CI) tools, automation testing tools become a norm in a DevOps cycle.

Why DevOps?

Although there are subtle differences between Agile and DevOps Testing, those working with Agile will find DevOps a little more familiar to work with (and eventually adopt). While Agile principles are applied successfully on the development & QA iterations, it is a different story altogether (and often a bone of contention) on the operations side. DevOps proposes to rectify this gap.

Now, instead of Continuous Integration, DevOps involves “Continuous Development”, where the code was written and committed to Version Control, will be built, deployed, tested and installed on the Production environment that is ready to be consumed by the end-user.

This process helps everyone in the entire chain since environments and processes are standardized. Every action in the chain is automated. It also gives freedom to all the stakeholders to concentrate their efforts on designing and coding a high-quality deliverable rather than worrying about the various building, operations, and QA processes.

It brings down the time-to-live drastically to about 3-4 hours, from the time code is written and committed, to deployment on production for end-user consumption.

In a nutshell, DevOps is an extension of Agile or I like to call it as“Agile on Steroids”.

Changed Role of QA in DevOps

Traditionally, QA would get a build which is deployed on their designated environment and QA would then commence their functional & regression testing. The build would ideally sit with the QA for a couple of days before the QA sign-off on the build. All these steps change in DevOps.

QA changes for DevOps Testing:

As already mentioned, DevOps requires a high level of coordination between various functions of the deliverable chain. This also means that the boundaries between various roles of contributors in the chain become porous.

DevOps encourages everyone to contribute to the chain. So, amongst other things, a dev can configure deployments. Deployment engineers can add test cases to the QA repository. QA Engineers can configure their automation test cases into the DevOps chain.

Collectively, everyone in the chain is responsible for the quality and timeliness of the deliverables.

DevOps and Test Automation

To achieve such speed and agility, it is important to automate all the testing processes and configure them to run automatically when the deployment is completed in the QA environment. Specialized automation testing tools and continuous integration tools are used to achieve this integration. This also necessitates the building of a mature automation testing framework through which one can quickly script new test cases.

DevOps Testing Strategy: Tips for DevOps Success

Application Monitoring

QA should also be able to detect problems early and report them proactively. To achieve this, they need to set up monitoring on the Production environment to be able to expose bugs before they cause a failure.

Setting up specialized counters like response times, memory & CPU utilization, etc. can provide a lot of insight into the end-user experience.

For example, if the average response time for login is gradually increasing over the various builds, QA should proactively report this issue for optimizing the login code, else future builds might cause end-user frustration due to high response times.

QA can also use a small subset of existing high priority test cases to be executed periodically on production, to actively monitor the environment. Bugs like, “This bug appears sometimes” or “Cannot Reproduce” can be caught through this strategy which, at the end, makes the application more stable and also gets more satisfied end-users.

Again, these monitors need to be configured to run automatically with rich reporting (like logs & screenshots of failures, etc.).


Waterfall gave way to V-Model which in turn was replaced by Agile as the preferred choice for software development.

DevOps is the future. It’s a continuous improvement cycle that software development models undergo from time-to-time. You need to embrace, understand and inculcate it.

You need to master the various automation and continuous integration tools so that your automation efforts add value to the chain and are lean enough to quickly adapt to changes. You may be working on projects that may involve an alpha, beta and UAT environments before being deployed in the production environment.

The concept essentially remains the same. Automation and more automation is the core of a successful DevOps cycle. But, as a QA you should also be able to draw a line as to how much automation is too much automation.

About the Author: Aniket Deshpande is working as a QA Manager at AFour Technologies, Pune and has been working in the software testing field for the past 9+ years in various domains and platforms. He is passionate about DevOps and is working as a consultant to guide organizations in adopting DevOps testing strategies. If you are interested in knowing more, or you are looking to implement DevOps and associated Test Approach in your organization, feel free to contact the author.

What do you think about DevOps Testing? Do you think by getting developers and operations folks to work together can benefit the project?

Let us know your comments/suggestions about this article.