Sample Template for Acceptance Test Report with Examples

Overview of Acceptance Test Report (Part-III):

Previous Tutorial | NEXT Tutorial

In our previous tutorial on “Acceptance Testing Documentation with Real-Time Scenarios,” we discussed the Acceptance Test plan.

In this tutorial, we will take an in-depth look at reporting of Acceptance Test Status, Acceptance Test Summary, and Sign-Off.

Some generic templates are included in this tutorial to enhance your understanding in a better way. We will also hover over the concept of Acceptance Testing in Agile and Acceptance Test-Driven Development.

In short, this tutorial will explain to you about the Acceptance test Status Report and Summary report along with some generic templates for your clear understanding and also brushes up the concept of Acceptance testing in Agile and test-driven development in an easily understandable way.

Acceptance Test Status Report

Acceptance Test Report should always summarize the acceptance tests that are performed along with their results. It should be addressed to all the identified stakeholders who are a part of the Acceptance Testing Phase. Once the execution of Acceptance Tests has started, the progress should be reported on a day-to-day basis.

Generic Template for Acceptance Test Status Report:

Date: Acceptance Test Status Report Date

Today’s Acceptance Tests Execution details:

• Number of Tests Passed
• Number of Tests Failed
• Number of Tests In-Progress

Acceptance Tests Execution Details till date:

• Total number of Tests
• Number of Tests Passed
• Number of Tests Failed
• Number of Tests In-Progress
• Number of Tests Pending

Defects Details:

• Number of Defects Logged
• Each Defect should have the below details:
  • ID, Summary, Component, Severity
• The total number of defects logged till now (in Acceptance Testing Phase).

This report has to be reviewed on a daily basis to ensure that the execution is on track and there is no deviation from the planned schedules.

Acceptance Test Summary Report

This is the report which summarizes the status of the entire Acceptance Testing phase. This involves details like testing activities conducted, references to criteria met, requirement specifications, business rules, execution results, planned schedules, deviations, etc.

Generic Template for Acceptance Test Summary Report:

Summary

<This is to summarize acceptance testing activities that took place i.e., Acceptance tests design, Acceptance Test Execution, Defects, Test environment. It also includes details like Release of the product, reference to Acceptance Test Plan, Entry Criteria passed, Exit criteria to be met, requirements/business requirement specifications, and business rules.>

Variances

<This helps to improve the Acceptance Test planning in future releases.>

Results

<Any reason for tests not being executed should be addressed immediately. This could be due to dependencies, environmental issues, etc. Also, the Test plan should be reviewed for addressing the causes of these kinds of issues).>

Evaluation

<Evaluation should be done for each component under test, by analyzing the success rate based on test execution and defects logged. Also, Entry criteria should be evaluated for actually met. Similarly, exit criteria should be confirmed based on the Acceptance test results of the execution. Deviations should be addressed with correct reasons and possible action items to correct them in future releases.>

Recommendation

<Based on the entire report, the recommendation for the product either to be released to the market or rejected is provided. Any improvements required, suggestions from acceptance testers, defects severity, and Pass rate impacts the recommendation for the product to be launched in the market.>

Efforts

<Actual effort spent for each of the activities in the phase should be explained in detail.>

Sign-off Report

Once the Product has passed Acceptance Testing, it will be recommended to Go Live. Before it is launched into Production, it has to be Signed-Off formally.

Generic Template for Sign-Off Report:

Product Name, Release Version, Build Number

<Mention Product Name and its Release Version. Mention the latest build number that underwent Acceptance Testing.>

Latest Report

<Attach the latest Acceptance Test Report.>

Reviewed On

<Mention the date when the latest Acceptance Test Report was reviewed.>

Reviewed By

<Mention by whom the latest Acceptance Test Report was reviewed.>

Review Comments

<Mention all the review comments on Acceptance Test Report.>

Sign-Off Date

<Mention the date when the Acceptance testing is Signed-off.>

Sign-Off By

<Mention who Signed-off the Acceptance Testing for the product.>

Sign-Off Comments

<Provide the statement for the Go decision for the Product.>

In general, any of the above reports should be reviewed by major stakeholders for its template and has to be agreed upon on what has to go as the information within.

All the details that get filled into the report should be cross-checked before sharing it with the Stakeholders. Any discrepancies in the report will highly impact the Business decision and could result in Product Failure in the Market.

Hence, reporting should always be handled by specialists or senior members of the team.

Acceptance Testing in Agile

In Agile, the Acceptance Criteria of each User story are targeted for Acceptance Tests, i.e., Acceptance tests are derived from the Acceptance criteria of a user story. Each Acceptance Criterion can have one or more Acceptance Tests to cover the scenario.

Acceptance Tests are usually designed by a QA who is the Subject Matter Expertise in the area. Acceptance Testing in Agile starts much early when compared to the other approaches, usually within the sprints themselves.

It is performed very frequently as each sprint will have new User stories coming up and also enhancements/continuations of the previous stories.

Acceptance Testing is performed at two different stages in Agile:

• When the feature is created and in its initial stage – basic.
• When the feature is integrated and stabilized with the other features of the product.

Each user story here has to undergo an Acceptance Test and should be passed for consideration. Any failures in the Acceptance test should be considered a high priority and fixed immediately, this, in turn, will have the acceptance test to execute it.

Story points are given to each user story based on the success of the Acceptance test results for each of the acceptance criteria. Acceptance Testing also defines the completion at the User Story level, stating that the Acceptance Criteria for the story are met.

Who does Acceptance Testing in Agile?

Usually, Product managers, Subject Matter Expertise (could be Customer AND/OR Beta testers) perform Acceptance Testing in an Agile environment. Sometimes, QA is also involved in this activity along with their regular regression tasks.

Benefits of Acceptance Testing in Agile

There are several benefits of Acceptance Testing in Agile.

Benefits are:

• Closer collaboration between the Product Manager and the team.
• Builds confidence at the User Story level.
• It will help to derive more scenarios to cover each Acceptance criteria.
• Increased probability to improvise the solutions to the product through Acceptance criteria in User Stories.

Drawbacks

Though there are several benefits, there are certain demerits too.

Drawbacks include:

• Not all the stories can be considered for Acceptance testing. Only functional stories to be covered – Story-wise coverage may come down.
• Not all the Acceptance Criteria can be considered for Acceptance testing. Only functional criteria are to be covered – Acceptance Criteria wise coverage within the User story may come down.
• Since stakeholders from different backgrounds are involved and as story-wise acceptance testing is performed directly, it is quite difficult for everyone to be on the same page (basically in understanding the level at individual User stories).
• Since the release duration is less when compared to other approaches, it is quite difficult to accommodate Acceptance Testing within Sprints.

Acceptance Test-Driven Development (ATDD)

This is one of the Agile development practices where the entire team collaboratively discusses each of the Acceptance criteria of User Story and builds strong Acceptance Tests around them.

This is because different perspectives from each of the team members will give a new way of thinking for each of the Acceptance criteria and will arrive at a good number of acceptance tests covering more scenarios. Sometimes, ATDD is also called Story Test Driven Development (STDD).

Actually, ATDD happens before the development starts. So, the developers, in this approach, will know what actually is expected and how to achieve it. The entire team will share a common understanding of the feature and what is being built.

This describes how the product is being built and in turn, will give a fair idea of how the product will actually function before it is handed over for testing. Hence it is termed “Acceptance Test Driven Development”.

Conclusion

Acceptance Testing in any of its approaches has the common goal of building Customer confidence and Satisfaction in the Product that is developed before it goes Live. This is gained only when there are no/fewer low severity defects in the product that do not hamper any of the functionalities.

In a nutshell:

• Acceptance Tests are Passed.
• Defects are at Acceptable levels.
• Flow-wise/Scenario-wise coverage achieved.
• The product and its solutions are accepted.
• The customer is confident enough in the product.
• All the Product documents are updated to match the latest functions.
• Result of team’s effort.
• Good to go forward with the Production Launch.

Previous Tutorial | NEXT Tutorial

Hope you would have gained immense knowledge from these Acceptance Testing Tutorials.  Feel free to share your thoughts and put forward your questions in the comments section below.