Acceptance Testing Documentation with Real-Time Scenarios

Documentation of Acceptance Testing (Part- II):

Previous Tutorial | NEXT Tutorial

This tutorial is the continuation of our previous tutorial where we discussed what is Acceptance testing, when it has to be done, who does it, its importance, types, process, impact on different teams, etc.

Documents play a very important role in Acceptance Testing and any issues with regards to the document have a huge negative impact. When a proper check is not exercised, then it may even lead to the Failure of the Product.

=> Click Here For Complete Test Plan Tutorial Series

In this tutorial, we will learn more about the different documentation involved in Acceptance Testing i.e., Acceptance Test Plan, Test Plan Review Checklist, Acceptance Test Template, examples based on real-time scenarios, and how to identify and write acceptance tests, etc in detail.

Acceptance Test Plan

Like any other test plan, the Acceptance test plan also includes some components like Scope, Approach, Test Environment, Resources, Responsibilities, Acceptance Tests references, Entry criteria, Exit criteria, Tools, etc.

The only thing that differentiates an Acceptance test plan from a regular test plan is the factors that result in business decisions. Acceptance Test Plan is one vital piece of documentation that provides guidance on how to perform acceptance testing for a particular project.

The acceptance test plan has to be reviewed and approved before Acceptance Test Execution. All the subsequent changes again have to undergo a review & approval process and have to be on track.

Acceptance Test Plan review is usually done by Managers/Business Analysts/Customers.

Key points to be considered while designing Acceptance Test Plan:

• It should be Detailed and Specific. Must include only what has required for testing and what information is necessary for the team to carry out testing.
• It should be Clear and Concise. No ambiguity. If at all there is something that may lead to confusion, then elaborate on it, but keep it short and effective.
• Each and every component in the document should be written keeping only the Business Requirements in mind.
• Reliable and adaptable – It should be updatable as required in future releases.
• Consistent – It should not have more changes in the future.
• Follow the template provided by the Organization or Customer.

Acceptance Test Plan Template

Here we will take a look at a common template for Acceptance Test Plan which can be further tweaked as per the project requirements.

Title

<Mention Project Name with its release version if required. Say Project_MainRelease1.0>

Objective

<Mention the objective of the project. Explain the purpose of the project and also mention the category of the end-users for whom the project is intended. For Example: If the project is to serve service for Bank employees/Sales Persons/Educational institutes etc., then it has to be clearly mentioned. Also, it should mention the reason for which that particular requirement has been chosen.>

Revision History/Change Log

<This should be in a tabular form with the below information:

• Date: The date on which the document was modified.
• Modified By: Who has changed the contents of the document.
• Purpose: Why was the document modified.
• Version: Current Version of the document after modifications (goes as 1.0, 1.1, 1.2, 1.3,… for a particular release. Next release will start from 2, 2.1, 2.2, 2.3,…, The list goes on).
• Approved By: Who has approved the changes made (implicitly means that the document has been reviewed and approved).

The very first row in this table should be the document-created details. Then follows the details of the changes made.>

Table of Contents

<Mention each and every component along with its page number.>

References

<Provide references to relevant documents for testing: Requirement Specification Document, Use Cases, etc.>

Scope

<Mention the scope of testing (Example: All the business scenarios).>

Introduction

<Clearly explain how the acceptance testing phase has reached in the project starting from the development. Provide details about the actions performed in this phase and how it was done. Also, the different types of testing performed in this phase can be detailed like Positive testing, Negative testing (not always), etc.>

Test Items

<Mention all the test items for this phase: Features/modules/sub-modules etc. (Example: Client Application, Server Application, etc.)>

Features to be Tested

<Mention all the features that need to be tested: Basic functionalities (Example: Search by different keywords, navigations between modules and sub-modules, etc.), Security features (Example: Registration to the website, login to the website, transactions, etc.)>

Features not to be Tested

<Explicitly mention all the features that are not to be tested like Resilience to the server, database crashes, network delays, etc. >

Approach

<Mention the approach to test in this phase. Provide details on how the testing is performed (Example: Once all the acceptance tests are executed, exploratory testing will be performed for a very short duration to assess the ease and use of the product. Each and every bug encountered will be logged in the bug management tool and will be discussed in bug triage meetings for RCA’s…)>

Test Environment Details

<Mention testing environment details: Staging/UAT/Pre-Prod/ etc., Also mention Test Bed set up information (Refer Acceptance Testing – Part 1).>

Entry Criteria

<Mention all the entry criteria for Acceptance Testing to begin. Refer to Acceptance Testing – Part1 for more details.>

Tests – If there are no separate Acceptance tests written

<List the series of tests to be conducted with each step described in detail.

Each test must include:

• Test #.
• A description of what is being tested (Example: Verify whether a user is able to create an account successfully).
• The business requirement to which this test maps (Traceability Matrix) – Very important.
• Pre-conditions:
  • State of the product before starting the testing (User should be registered successfully but not activated the account, User should have accessed the product at least 30 days ago, etc.)
  • Any server conditions – Should the server be down for some time.
• Test Steps: Detailed numbered flow (Example: see below
  • Open the application.
  • Attempt to log in with valid credentials with Remember Me checkbox selected).
• Expected Result: What is the expected behavior of the step>

Acceptance Tests – If there are separate Acceptance tests written

<Provide the link to the Acceptance Tests document.>

Exit Criteria

<Mention all the exit criteria for Acceptance Testing to end. Refer to Acceptance Testing – Part1 for more details.>

Resources

<Mention the names of all the members who will be a part of the Acceptance Testing phase in proper order. Also, mention their role in the phase.>

Roles and Responsibilities

<Mention the responsibilities that each role has to perform.>

Tools

<Mention what tools have to be used, maybe for bug logging, test management to update results, etc., Provide proper references to each of the tools which will guide the resources on how to use them.>

Business Decision Factors

<Mention all the factors which lead to the business decision either to Go or No-Go.>

Sign-Off Procedure

<Clearly describe how the Acceptance testing Sign-off takes place and who is responsible for signing off.>

Point of Contact

<Mention the point of contact for the Accepting Testing Phase with the Name, Corporate Email Address, Role and Contact Number.>

An acceptance test plan is considered the Master Test Plan for the Phase.

Reviewing Acceptance Test Plan

Once the plan is ready, it has to be reviewed for completeness, non-ambiguity, clearness, quality, etc. No doubt that, the entire content in the Acceptance test plan has to be reviewed thoroughly for proper information, but, it has to be reviewed against a few other points as well let’s say checklist points.

Here, let’s categorize the contents and see the checklist points against them.

Any test plan satisfying the above checklist document will serve as a strong document for internal audits as well.

Acceptance Tests

Acceptance tests were earlier known as Functional tests. In order to make the name more suitable for the Acceptance testing phase and to serve the purpose, it was renamed Acceptance Tests. Sometimes it is also termed Customer Tests.

Acceptance tests are always derived from user stories, acceptance criteria, and use cases. These are black-box system tests and represent only those business tests that have to be verified. These should be intended mainly for product behavior, usage, and flows.

The designed acceptance tests can also be taken into consideration for the system testing phase in the regression cycles to gain confidence in the product before handing it over to the acceptance testing phase.

Key points to be remembered before writing acceptance tests:

• Keep all the reference documents in place: Software Requirement Specification, Business Requirements Document, Use Cases, User Stories, Data matrix (in case of logic involved), etc.
• Focus only on business requirements (testable business requirements).
• Clear all the doubts, and queries on the business requirements at the earliest.
• Make sure there are no changes to the requirements for the current release at least.

General and Simple Template to write Acceptance Tests:

This template can again be tweaked as per the project needs and with more information to include.

Now, let’s take some common scenarios and see how Acceptance test scenarios can be written on them.

Case 1: User Account handling

This is the scenario where the users are allowed to Create, View, Update, and De-activate their accounts. In general, it’s a CRUD operation (Create, Read, Update, and Delete). So directly we will get 4 major scenarios to test.

Along with this, in real-time user account handling, we have many areas when it comes to viewing and updating.

Proceeding with writing acceptance tests:

Test 1: Registration/Sign Up/Create Account, verify if a user is able to:

• Create the Account.
• Activate the account.
• Activate the account only once (Here, the activation link has to be tested for 2^nd Even though this is negative testing, it is one of the major verification points to be considered).

Test 2: To Access and View Account information, verify if a user is able to:

• Log in to the account.
• View different sections in the Profile (If the Profile section is categorized, then each and every category should be viewable).
• Verify that the data displayed in the Profile is correct as per the User’s input.

Test 3: To Update Account Information, verify if a user is able to:

• Update Account Information (Profile):
  • Update each and every category of the Profile.
• Verify that the updated information is reflected correctly in the Profile.
• Verify if the User is not able to update information in the Profile (In some applications, First name, Last name, Username, etc. will not be allowed to update. Even though this is negative testing, it is one of the major verification points to be considered).
• Cancel the Update flow (Even though this is negative testing, it is also one of the major verification points to be considered).

Test 4: If the De-activation of an account is allowed, then, verify if a user is able to:

• De-activate the account.
• Cancel de-activation flow (Even though this is negative testing, it is one of the major verification points to be considered).
• Access account after canceling de-activation.

Test 5: If verifications are required for an Email address or Phone number, then, verify if a user is able to:

• Update the Email address to the other valid one.
• Verify” updated Email address.
• Verify if an updated and “verified” Email address is considered further – Send some emails from the application and check for its arrival to the updated Email address. The old one should not receive Emails.
• Add the new Phone number.
• Verify added phone number through Call.
• Verify added phone number through SMS.
• Verify that the added and “verified” phone number is reflected in the account.
• Update the Phone number.
• Verify updated phone number through Call.
• Verify ” updated phone number through SMS.
• Verify if the updated and “verified” phone number is reflected in the account.

Case 2: Purchasing Product

Purchasing of the product usually has a general flow.

Some general scenarios that end-users look at are listed here:

Pre-condition: The user should be logged in to the application.

Test 1: Product Details, verify if a user is able to:

• View the Product details page.
• View all the sub-sections on the Product details page (Description, Feature, Brand information, etc.).
• Select the quantity of the product, Color, Size, etc. as available on the Product details page.
• Navigate to the category, sub-category pages from the Product Details page (if available on the Product details page).
• Navigate to the other Product’s details page (if provided relevant products section).
• View comments and ratings on the product.
• Sort Comments about the Product based on ratings.
• View the overall rating of the Product.
• Add Comment on the Product.
• Update his/her comment on the product.
• Delete his/her comment on the product (if provided).

Test 2: Add to Cart, verify if a User is:

• Able to add the product to Cart:
  • Through the Product details page.
  • Through the Product list page.
• Able to add the required quantity to the cart ( 1 to max limit set).
• Not able to add the product to the Cart if Out-of-Stock.

Test 3: In the Cart Page, verify if a user is able to:

• View the Product in the Cart with Price details for added quantity.
• Update quantity (1 to max limit set).
• Remove the Product from Cart.
• Navigate back to shopping.
• Continue to Checkout.
• View Empty Cart when no product is added,

Test 4: In the Account Details Page, verify if a user is able to:

• Continue with the existing Shipping details.
• Update Shipping address.
• Add a new shipping address.
• Continue with the existing Phone number.
• The updated Phone number for the order.
• Add a new Phone number for the order.
• Navigate back to the Cart page.
• Navigate to the Payment page.

Test 5: On the Payments Page, verify if a user is able to:

• Verify the correctness of the amount to be billed.
• Process the order with all the available options (One option for each separate order).
• Process transaction Successfully. Go to the Order Confirmation page.
• Transaction Failure (Even though this is negative testing, it should be considered a major scenario).
• Apply coupons:
  • Valid Coupons – Success. Here verify the change in the amount to be billed.
  • Invalid coupons – Failure
• Expired coupons – Failure.
• Navigate back to the Account details page.

Reviewing Acceptance Tests

Reviewing Acceptance tests is an important task as it needs to be correct and to the point with respect to the business requirements. As these may be conducted by Customers themselves and/or end-users, it is very much necessary to be complete, non-ambiguous, correct, and detailed enough for anyone to understand and execute.

Reviewing Acceptance tests have to be done by Business Analysts, Customers and any review comments should be incorporated into high priority.

At the individual test level, the review should be done against the below:

• Whether the test covers the business requirement or not.
• Are the pre-conditions clear?
• Are the test steps easy to understand and detailed?
• Is the expected result correct and clear?
• Is it mapped to the Business requirements for traceability?
• Is the test complete enough to cover the particular flow or usage?
• Is the particular test required as part of acceptance testing?
• Is there some verification point that is not needed for acceptance testing?
• Is it purely functional or any GUI is covered within (It should be only Functional).
• Is the special input data necessary? If yes, is it provided for particulars?

On the whole, the entire Acceptance tests suite review should cover:

• Bi-directional traceability: Business requirements to Tests AND Tests to Business requirements.
• Is each and every business requirement covered?
• Is every business requirement covered by one or more tests?
• Are the business rules covered?
• Is the special data case handled?
• How many tests are written to cover each requirement or rule?
• Can the tests be grouped together and classified for flows?
• Are the tests sequenced properly so that the execution is efficient?

Conclusion

In a nutshell, as mentioned earlier, documents play a very drastic role in Acceptance testing.

Hence, any acceptance test that is written should be well-structured and inflow with its usage, so that it keeps the acceptance testers to be interested in what they are testing and how they are doing it. This, in turn, would automatically bring in success.

=> Visit Here For Complete Test Plan Tutorial Series

Previous Tutorial | NEXT Tutorial

Stay tuned and watch out for the upcoming Acceptance Testing tutorial to know more about Acceptance Testing Reports along with some generic templates. Also, let us know if you have any queries.