How to Implement Proof of Concept (POC) in Automation Testing?
Every organization has different testing processes and procedures. Manual Testing is important and irreplaceable- however, automation is picking speed.
Introducing automation testing to an organization is a challenge and following points will determine if it is required at all:
#1. Duration of the project: Short-term or long-term – long-term projects are good candidates for automation
#2. How much regression is done in each testing cycle? – projects that have repetitive and lengthy regression tests as automation reduces the overall testing time and ensures complete coverage.
#3. Stability of the application: Application that is not susceptible to frequent changes should be considered for automation. The product which is not stable, where GUI/Functionality keeps changing, elements or its XPath on page keep changing should not be automated until stable.
#4. Is the project data secure and does testing it requires some complicated procedures? – In this case, it is best to go for manual testing.
#5. Does the organization have a budget for automation? – Automation will add to additional expenditure for the organization like automation tool cost, resource cost, a time required for framework development and writing/maintaining automation test scripts.
With automation, missing tests or taking some test results for granted will never happen. It ensures 100% coverage of given module each and every time the same is tested. Automation will also help perform the same test multiple times on multiple browsers and platforms.
The following figure will help understand the process of automation testing
From the technical testing point of view, the QA team needs to understand the following aspects about their automation tool:
What You Will Learn:
#1. An application under test is a web application or a desktop application.
#2. Choosing an open source tool Vs paid one.
#3. Tool should fulfill the application’s testing requirement
#4. Using the tool – the team’s expertise and comfort level in terms of using and learning the tools
#5. Does it support reporting – If No what other options of reporting are available (open source or paid). If yes then how good it is in terms of conveying correct data from presentations as well as content point of view.
In addition, tool evaluation includes:
While selecting an automation tool it is very important to consider if it is supported on the applications GUI implementation.
It is important to assess the tool on the above points to understand if the tool really meets the project’s testing requirements.
Implementing an automation testing POC is a crucial and most often used method of introducing a tool to an organization. Once it is decided that automation is to be done and a tool has been chosen, it is time to create a prototype as a POC and present it to the management to showcase the real-time usage and benefits.
To do so:
1) Decide the test cases that we will use in the POC.
2) It helps to pick the areas the clients will be most interested in.
3) Plan to show manual vs automation in a way that proves that there is no degradation in the quality by choosing automation.
4) Include a test case that fails and results in finding a defect- this helps reinforce that the tool indeed can find defects
5) Use assertions and validation points wherever necessary.
6) Show clearly areas that can and cannot be automated. Usually, the following aspects cannot be automated:
7) Highlight if the tool satisfies the following requirements?
Once we present our proof of concept to the management and we get a go-ahead from them, the next step is implementing a pilot project using that tool.
There is no one perfect POC template. It generally includes:
Here are a couple of Automation POC templates for reference:
We should define our pilot project by:
Step #1. Choosing test cases for pilot
Step #2. Automation framework development
A test automation framework is the set of concepts, process, procedures, practices, and environment. It is nothing but an integrated system that consists of rules to automate any given product. This system includes set of functional libraries, APIs, test data, object repository and various other modules. The framework and approach of scripting used for test automation have effect on its costs.
Following scripting techniques can be used:
Using any of the above technique, a testing framework can be designed that will help in achieve a specific format to drive the test, simplify test execution and reporting.
Determine templates, naming conventions for objects, test cases, test suites, data repository etc.
Step #3. Script development and execution
Step #4. Reporting: Does the tool have in-built reporting capabilities? Are the inbuilt reports capable of conveying all the required information precisely? Are we going to need another tool for reporting purpose like crystal reports, reportNG, etc.?
Step #5. Maintaining automation scripts
As much as proof of concept and implementing a pilot is important so is presenting it in the correct manner. Following points will help to present it in a positive way.
Be prepared to answer questions related to how much time will it take to automate a single simple or critical functionality. Also, if a minor change happens on application front how much script changes will be required as how much time will be taken to modify.
We hope this guide is useful for you to start writing an automation testing POC document. Let us know if you have any questions.