Documentation of Acceptance Testing (Part- II):
This tutorial is the continuation of our previous tutorial where we discussed what is Acceptance testing, when it has to be done, who does it, its importance, types, process, impact on different teams, etc.
Documents play a very important role in Acceptance Testing and any issues with regards to the document have a huge negative impact. When a proper check is not exercised, then it may even lead to the Failure of the Product.
In this tutorial, we will learn more about the different documentation involved in Acceptance Testing i.e., Acceptance Test Plan, Test Plan Review Checklist, Acceptance Test Template, examples based on real-time scenarios, how to identify and write acceptance tests, etc in detail.
What You Will Learn:
Like any other test plan, Acceptance test plan also includes some components like Scope, Approach, Test Environment, Resources, Responsibilities, Acceptance Tests references, Entry criteria, Exit criteria, Tools, etc.
The only thing that differentiates Acceptance test plan from a regular test plan is its factors that result in Business Decision. Acceptance Test Plan is one of the vital documentation that provides guidance on how to perform acceptance testing for a particular project.
Acceptance test plan has to be reviewed and approved before Acceptance Test Execution. All the subsequent changes again have to undergo review & approval process and has to be in-track.
Acceptance Test Plan review is usually done by Managers/Business Analysts/Customers.
Key points to be considered while designing Acceptance Test Plan:
Here we will take a look at a common template for Acceptance Test Plan which can be further tweaked as per the project requirements.
<Mention Project Name with its release version if required. Say Project_MainRelease1.0>
<Mention the objective of the project. Explain the purpose of the project and also mention the category of the end-users to whom the project is intended to. For Example: If the project is to serve service for Bank employees/Sales Person/Educational Institute etc., then it has to be clearly mentioned. Also, it should mention the reason for which that particular requirement has been chosen.>
Revision History/Change Log
<This should be in a tabular form with the below information:
The very first row in this table should be the document created details. Then follows the details of the changes made.>
Table of Contents
<Mention each and every component along with its page number.>
<Provide references to relevant documents for testing: Requirement Specification Document, Use Cases, etc.>
<Mention the scope of testing (Example: All the business scenarios).>
<Clearly explain how the acceptance testing phase has reached in the project starting from the development. Provide details about the actions performed in this phase and how it was done. Also, the different types of testing performed in this phase can be detailed like Positive testing, Negative testing (not always), etc.>
<Mention all the test items for this phase: Features/modules/sub-modules etc. (Example: Client Application, Server Application, etc.)>
Features to be Tested
<Mention all the features that need to be tested: Basic functionalities (Example: Search by different keywords, navigations between modules and sub-modules, etc.), Security features (Example: Registration to website, login to website, transactions, etc.)>
Features not to be Tested
<Explicitly mention all the features that are not to be tested like Resilience to server, database crashes, network delays, etc. >
<Mention the approach to test in this phase. Provide details on how the testing is performed (Example: Once all the acceptance tests are executed, exploratory testing will be performed for a very short duration to assess the ease and use of the product. Each and every bug encountered will be logged in the bug management tool and will be discussed in bug triage meetings for RCA’s…)>
Test Environment Details
<Mention testing environment details: Staging/UAT/Pre-Prod/ etc., Also mention Test Bed set up information (Refer Acceptance Testing – Part 1).>
<Mention all the entry criteria for Acceptance Testing to begin. Refer Acceptance Testing – Part1 for more details.>
Tests – If there are no separate Acceptance tests written
<List the series of tests to be conducted with each step described in detail.
Each test must include:
Acceptance Tests – If there are separate Acceptance tests written
<Provide the link to Acceptance Tests document.>
<Mention all the exit criteria for Acceptance Testing to end. Refer Acceptance Testing – Part1 for more details.>
<Mention the names of all the members who will be a part of the Acceptance Testing phase in a proper order. Also, mention their role in the phase.>
Roles and Responsibilities
<Mention the responsibilities that each role has to perform.>
<Mention what tools have to be used, maybe for bug logging, test management to update results, etc., Provide proper references to each of the tools which will guide the resources on how to use them.>
Business Decision Factors
<Mention all the factors which lead to business decision either to Go or No-Go.>
<Clearly describe how the Acceptance testing Sign-off takes place and who is responsible for signing off.>
Point of Contact
<Mention the point of contact for the Accepting Testing Phase with the Name, Corporate Email Address, Role and Contact Number.>
Acceptance test plan is considered as the Master Test Plan for the Phase.
Once the plan is ready, it has to be reviewed for completeness, non-ambiguity, clearness, quality, etc. No doubt that, the entire content in the Acceptance test plan has to be reviewed thoroughly for proper information, but, it has to be reviewed against few other points as well let’s say checklist points.
Here, let’s categorize the contents and see the checklist points against them.
|Title||Is the title matching the project title as referred everywhere
Is the title following Project’s naming conventions
|Revision History, Table of Contents||Is every version modifications tracked properly for the plan
Has every version change undergone proper review and is mentioned
Is the versioning convention correct
Does the table of contents match the actual contents in the plan
Is the page number for each content correct
Is the page number updated if the modifications made in the plan changed page number of the contents
|References||Are the references existing and valid
Do they match with scope
Are they complete and considered for tests identification
|Test Items, Features to be tested, Features not to be tested||Are they numbered
Does each feature / module / sub-module comes under scope
Can the schedule planned cover all the identified test items within
|Entry Criteria, Exit Criteria||Are they numbered
Is each and every criteria mentioned in detail
|Test Environment Details||Is it having all the required configurations mentioned
Is the version for each configuration specific or latest to be considered
Do the VMs, environment exists (if not, mention possible date for its availability)
Is the credentials sharing method for particular environment access mentioned
|Acceptance Tests||Are the tests numbered
Are the Preconditions numbered
Are the test steps clear to understand
Are the test steps complete
Is the expected result complete
Is there any open question in the tests (if any, follow-up and complete it)
Is the reference to Acceptance Tests (if written separately) valid and existing
Is the traceability correct
Is there any business requirement missed out to cover for test
|Resources, Roles, and Responsibilities||Are the responsibilities for each role numbered
Can the responsibilities be achieved
Is the identified resource capable of handling mentioned responsibilities
|Tools||Are all the tools mentioned
Are all the tools numbered
Are all the tools versioned
Does any of the tool need license or the existing license valid during the phase
Is the guidance to the tool usage correct and sufficient
|Business Decision Factors||Has all the factors mentioned
Are all the factors numbered
|Sign-Off Procedure||Is the procedure valid
Is the procedure acceptable
Is the procedure clear to understand
|Point of Contact||Is the resource identified as point of contact available in the organization during the phase
Is the resource identified capable of handling the phase
Any test plan satisfying the above checklist document will serve as a strong document for internal audits as well.
Acceptance tests were earlier known as Functional tests. In order to make the name more suitable for Acceptance testing phase and to serve the purpose, it was renamed as Acceptance Tests. Sometimes it is also termed as Customer Tests.
Acceptance tests are always derived from user stories, acceptance criteria, and use cases. These are black-box system tests and represent only those business tests that have to be verified. These should be intended mainly for product behavior, usage, and flows.
The designed acceptance tests can also be taken into consideration for system testing phase in the regression cycles to gain confidence on the product before handing it over to acceptance testing phase.
Key points to be remembered before writing acceptance tests:
General and Simple Template to write Acceptance Tests:
This template can again be tweaked as per the project needs and with more information to include.
Now, let’s take some common scenarios and see how Acceptance test scenarios can be written on them.
This is the scenario where the users are allowed to Create, View, Update, and De-activate their account. In general, it’s a CRUD operation (Create, Read, Update, and Delete). So directly we will get 4 major scenarios to test.
Along with this, in real-time user account handling, we have many areas when it comes to viewing and updating.
Proceeding with writing acceptance tests:
Test 1: Registration/Sign Up/Create Account, verify if a User is able to:
Test 2: To Access and View Account information, verify if a User is able to:
Test 3: To Update Account Information, verify if a User is able to:
Test 4: If De-activation of account is allowed, then, verify if a User is able to:
Test 5: If verifications are required for an Email address or Phone numbers, then, verify if a User is able to:
Purchasing of the product usually has the general flow.
Some general scenarios what end-users look at are listed here:
Pre-condition: User should be logged in to the application.
Test 1: Product Details, verify if a User is able to:
Test 2: Add to Cart, verify if a User is:
Test 3: In the Cart Page, verify if a User is able to:
Test 4: In the Account Details Page, verify if a User is able to:
Test 5: On the Payments Page, verify if a User is able to:
Reviewing Acceptance tests is an important task as it needs to be correct and to-the-point with respect to the business requirements. As these may be conducted by Customers themselves and/or end-users, it is very much necessary to be complete, non-ambiguous, correct, and detailed enough for anyone to understand and execute.
Reviewing Acceptance tests has to be done by Business Analysts, Customers and any review comments should be incorporated into high priority.
At the individual test level, the review should be done against the below:
On the whole, entire Acceptance tests suite review should cover:
In a nutshell, as mentioned earlier, documents play a very drastic role in Acceptance testing.
Hence, any acceptance test that is written should be well-structured and in-flow with its usage, so that it keeps the acceptance testers to be interested in what they are testing and how they are doing it. This, in turn, would automatically bring in success.
Stay tuned and watch out the upcoming Acceptance Testing tutorial to know more about Acceptance Testing Reports along with some generic templates. Also, let us know if you have any queries.