Overview of Acceptance Test Report (Part-III):
In our previous tutorial on “Acceptance Testing Documentation with Real-Time Scenarios” we discussed Acceptance Test plan.
In this tutorial, we will take an in-depth look at reporting of Acceptance Test Status, Acceptance Test Summary, and Sign-Off.
Some generic templates are included in this tutorial to enhance your understanding in a better way. We will also hover over the concept of Acceptance Testing in Agile and Acceptance Test Driven Development.
In short, this tutorial will explain you about Acceptance test Status Report and Summary report along with some generic templates for your clear understanding and also brushes up the concept of Acceptance testing in Agile and test-driven development in an easily understandable way.
What You Will Learn:
Acceptance Test Report should always summarize the acceptance tests that are performed along with their results. It should be addressed to all the identified stakeholders who are a part of Acceptance Testing Phase. Once the execution of Acceptance Tests have started, the progress should be reported on a day-to-day basis.
Date: Acceptance Test Status Report Date
Today’s Acceptance Tests Execution details:
Acceptance Tests Execution Details till date:
This report has to be reviewed on a daily basis to ensure that the execution is on track and there is no deviation from the planned schedules.
This is the report which summarizes the status of the entire Acceptance Testing phase. This involves details like testing activities conducted, references to criteria met, requirement specifications, business rules, execution results, planned schedules, deviations, etc.
<This is to summarize about acceptance testing activities that took place i.e., Acceptance tests design, Acceptance Test Execution, Defects, Test environment. It also includes details like Release of the product, reference to Acceptance Test Plan, Entry Criteria passed, Exit criteria to be met, requirements/business requirement specifications, and business rules.>
<This helps to improve the Acceptance Test planning in the future releases.>
<Any reason for tests not executed should be addressed immediately. This could be due to dependencies, environment issues, etc. Also, the Test plan should be reviewed for addressing the cases for these kinds of issues).>
<Evaluation should be done for each component under test, by analyzing the success rate based on test execution and defects logged. Also, Entry criteria should be evaluated for actually met. Similarly, exit criteria should be confirmed based on the Acceptance test results of the execution. Deviations should be addressed with correct reasons and possible action items to correct it in the future releases.>
<Based on the entire report, recommendation for the product either to be released to the market or reject is provided. Any improvements required, suggestions from acceptance testers, defects severity, and Pass rate impacts the recommendation for the product to be launched in the market.>
<Actual effort spent for each of the activities in the phase should be explained in detail.>
Once the Product has passed Acceptance Testing, it will be recommended to Go Live. Before it is launched to Production, it has to be Signed-Off formally.
Product Name, Release Version, Build Number
<Mention Product Name and its Release Version. Mention the latest build number that underwent Acceptance Testing.>
<Attach the latest Acceptance Test Report.>
<Mention the date when the latest Acceptance Test Report was reviewed.>
<Mention by whom the latest Acceptance Test Report was reviewed.>
<Mention all the review comments on Acceptance Test Report.>
<Mention the date when the Acceptance testing is Signed-off.>
<Mention who Signed-off the Acceptance Testing for the product.>
<Provide the statement for Go decision for the Product.>
In general, any of the above reports should be reviewed by major stakeholders for its template and has to be agreed upon what has to go as the information within.
All the details that get filled into the report should be cross-checked before sharing it with the Stakeholders. Any discrepancies in the report will highly impact the Business decision and could result in Product Failure in the Market.
Hence, reporting should always be handled by specialists or senior members of the team.
In Agile, Acceptance Criteria of each User story is targeted for Acceptance Tests, i.e., Acceptance tests are derived from the Acceptance criteria of a user story. Each Acceptance Criteria can have one or more Acceptance Tests to cover the scenario.
Acceptance Tests are usually designed by a QA who is the Subject Matter Expertise in the area. Acceptance Testing in Agile starts much early when compared to the other approaches, usually within the sprints itself.
It is performed very frequently as each sprint will have new User stories coming up and also enhancements/continuation of the previous stories.
Acceptance Testing is performed at two different stages in Agile:
Each user story here has to undergo Acceptance Test and should be passed for consideration. Any failures in Acceptance test should be considered as a high priority and fixed immediately, this, in turn, will have the acceptance test to execute it.
Story points are given to each user story based on the success of Acceptance test results for each of the acceptance criteria. Acceptance Testing also defines the completion at User Story level, stating that the Acceptance criteria for the story are met.
Usually, Product managers, Subject Matter Expertise (could be Customer AND/OR Beta testers) perform Acceptance Testing in an Agile environment. Sometimes, QA also involves in this activity along with their regular regression tasks.
There are several benefits of Acceptance Testing in Agile.
Though there are several benefits, there are certain demerits too.
This is one of the Agile development practice where the entire team collaboratively discusses each of the Acceptance criteria of User Story and builds strong Acceptance Tests around them.
This is because different perspectives from each of the team member will give a new way of thinking for each of the Acceptance criteria and will arrive with a good number of acceptance tests covering more scenarios. Sometimes, ATDD is also called Story Test Driven Development (STDD).
Actually, ATDD happens before the development starts. So, the developers, in this approach, will know what actually is expected and how to achieve it. The entire team will share a common understanding of the feature and what is being built.
This describes how the product is being built and in turn, will give a fair idea of how the product will actually function before it is handed over for testing. Hence it is termed as “Acceptance Test Driven Development”.
Acceptance Testing in any of its approach has the common goal of building Customer confidence and Satisfaction on the Product that is developed before it goes Live. This is gained only when there are no/fewer low severity defects in the product that do not hamper any of the functionalities.
In a nutshell:
Hope you would have gained immense knowledge from these Acceptance Testing Tutorials. Feel free to share your thoughts and put forward your questions in the comments section below.