Automation in Agile is very critical.
Think about the many features that are added and delivered in every Sprint. There has to be a way to make sure that the newly added feature is not impacting the existing functionality.
Because of the low Sprint duration, it is practically impossible to execute the entire suit every time the product is incremented at the Sprint end. Having an automated test suit would definitely play a bigger role here.
However, introducing and maturing into automation would definitely take some time. Doing an initial investment in planning and designing the automation activity would definitely pay off in the long run.
In this 3rd part of Agile Testing advanced series, I am trying to cite a few pointers to consider based on my experience, as you bring automation to your project.
What You Will Learn:
Whenever we are planning to introduce automation into our projects, most of us immediately vote for either the “Smoke Tests suit” or the “regression test suit” to be the best candidate for automation. Of course, they are, but when we think of the automation test pyramid, we can conclude that it’s just the top layer of the pyramid that we are talking about.
Apart from above layer we still have the service layer and the unit layer that are more important.
So what tests, other than Smoke tests and Regression tests, can be good candidates for automation?
In traditional environments, we have predefined builds that can be weekly, fortnightly or sometimes even monthly. One of the reasons is that these deployments take time. The problem with this approach is that we have to wait for the predefined dates to get the bugs fixed or to get the new features implemented, so there is a delay.
The second reason was – by the time testers finish up with the testing and come up with bugs and defects, the programmers have moved on to different pieces of implementation and have less interest in resolving the bugs of the older application. This approach also delays the time for making the feature available in production.
Building and deployments are the entities that are repetitive and sometimes boring. It can also take hours to deploy a build, which delays the testing and eventually the feedback. Being a repetitive task, deployments becomes a good candidate for automation.
Also read => The Release and Deployment Management Process
A few of the advantages of having automated build deployment are:
I have already talked about the importance of automating the unit layer by using the TDD approach in my last tutorial.
This forms the lowest layer of the pyramid, hence the foundation and any foundation needs to be rock solid. The development team should collaborate and work together to accommodate most of the test into this layer.
Web services are the medium in which two applications exchange the data or information in terms of request and response, without bothering with the underlying architecture or the technology. In more simple terms – giving a request and validating the response is what we normally do in the web services testing.
Testing the web services entails writing programs to call those web service methods and validating the value/s that it returns. We can even test the services for various permutations and combinations. Have all the test data in the excel sheet and your program can read the data and call the testable service by passing the test data as a parameter and validate the results.
This particular testing is part of the middle layer of the pyramid. Most of the functional testing can be pushed into this layer. Resolving defects that arise in this layer becomes easy to fix and they are not postponed until the UI is available.
Automating the testing behind the GUI is comparatively easier than automating the actual GUI. Another advantage is that irrespective of the UI changes, functionality remains intact. Even if some of the UI element is changed, the functionality of the feature does not change. This technique mainly focuses on the business logic and rules.
The test cases are mostly written in a tabular format or in a spreadsheet and fixtures/code snippets are written that accept the input from these tables and returns the results. The results are generated immediately and provide a great platform for the non-technical stakeholders to run these tests and get the expected results. One of the tools used for achieving this technique is Fitnesse.
This non-functional testing technique basically involves the Load, Performance and Stress testing. There are various tools readily available in the market that can be utilized to automate these tests.
Many of our testings require us to compare data files, including text files, CSV or excel files
These comparisons can be repetitive, therefore automated.
Searching for a specific entity from a large bunch of files can also be tedious and God helps us if that is a repetitive task. One example is searching through log files. If this is also a tedious and repetitive task than we should think about automating it.
There are no bullet points or a step by step guide that says where to start automation. Kicking off automation for the team requires you to brainstorm and apply deep thoughts on what aspects you are looking to automate, or what is the ultimate goal of the automation?
You can start by:
If you have no automation in the tour project/team, then you can probably go for a multi-layered approach where the unit tests can be targeted first to automate. This would give you the highest ROI.
Simultaneously, testers can start working on smoke test suit and then the regression. Once the team has gained the skills and feels comfortable, gradually move towards automating the other repetitive tasks.
Do not directly jump into buying a new tool without evaluating your needs. As I said earlier, a simple program or a macro can solve your purpose of automating some of the repetitive tasks. So, before deciding to buy a tool, do a POC and evaluate whether that tool would be effective to use.
Please go through these documents where I have provided more details on how to select correct test cases for automation and some insights on Estimating automation efforts in the following articles manual to automation testing process challenges and test estimation of selenium automation project.
Once the scope of automation and tool is finalized, next is to design the framework.
Remember, in Agile, the framework is evolved. Do NOT target designing the entire framework first and then implement. Design and implement for the MVP (Minimum Viable Product) and then enhance the existing framework to include more features. You also need to apply good coding and development practice if you want your automation suite to be robust.
Some very simple tips:
Automating in the Agile world does come with its own challenges:
Amongst all these challenges, the most critical challenge is to upgrade the skills of testers.
Doing and maintaining the automation for a team is almost like a programming (development) activity which the programmers (developers) do. Not only just the implementation but also integrating the automated suit to CI is important and requires that testers learn and adopt new skills and learn new tools and technologies.
Let me conclude with the famous Agile test quadrants:
Quadrant 1 is the Unit and the components test that can be automated with the TDD approach.
Quadrant 2 talks about the functionality testing, where we can apply the BDD approach.
Quadrant 3 is the only quadrant which has a scope of manual testing.
Quadrant 4 basically talks about the testing that can be achieved by some tools. This takes care of the Load tests, Stress tests, Volume tests and Security tests.
There is a lot of scope of automation apart from the Smoke tests and Regression tests. Therefore, we have to break free from the concept of confining automation only to these types of testing, which in turn means that the skill set of a tester in Agile demands more than just finding bugs and defects.
Testers need to be more collaborative and sharpen their programming/automation skills. If more and more tests are automated, it would give the testers more time to engage themselves in more sophisticated and challenging tasks.
About the author: This article is by STH team member Shilpa. She is working in the software testing field for the past 10+ years in domains like Internet advertising, Investment Banking, and Telecom.
Please share your comments and thoughts below.