What is User Acceptance Testing (UAT)? UAT Test Plan Doc

Learn What is User Acceptance Testing (UAT), Along with its Definition, Types, Steps, and Examples:

My rule number one when trying to understand a new concept is that the name is always going to be relevant and mostly a literal meaning (in the technical context).

Finding out what that is will give you an initial understanding of it and help me to get started.

=> Click Here For Complete Test Plan Tutorial Series

What Is User Acceptance Testing

Let us put this concept to the test.

=> Read all tutorials in our Acceptance Testing series.

We know what testing is. Acceptance means approval or agreement. The user of a software product is either the consumer of the software or the person who requested it to be built for him/her (client).

So, following my rule – the definition will be:

User Acceptance Testing (UAT), also known as beta or end-user testing, is defined as testing the software by the user or client to determine whether or not it can be accepted. This is the final testing performed once the functional, system, and regression testing are completed.

The main purpose of this testing is to validate the software against the business requirements. The end-users who are familiar with the business requirements carried this validation out.

UAT, alpha, and beta testing are different types of acceptance testing.

As the user acceptance test is the last testing that is carried out before the software goes live, obviously this is the last chance for the customer to test the software and measure if it is fit for the purpose.

When Is It Performed

This is typically the last step before the product goes live or before the delivery of the product is accepted. This is performed after the product itself is thoroughly tested (i.e. after system testing).

Who Performs UAT

Users or clients – This could be either someone who is buying a product (with commercial software) or someone who has had software custom-built through a software service provider or the end-user if the software is made available to them ahead of time and when their feedback is sought out.

The team can include beta testers or the customer should internally choose UAT members from each organizational group to ensure testing of all user roles.

Need For User Acceptance Testing

Developers and functional testers are technical people who validate the software against the functional specifications. They interpret the requirements according to their knowledge and develop/test the software (here is the importance of domain knowledge).

This software is complete according to the functional specifications but there are some business requirements and processes that are known only to the end-users and are either communicated or misinterpreted.

This testing is crucial for validating whether or not all the business requirements are fulfilled before releasing the software for market use. Using live data and real use cases makes this testing an important part of the release cycle.

Many businesses that suffered big losses due to post-release issues know the importance of a successful User Acceptance Test. The cost of fixing the defects after release is many times greater than fixing it before.

Is UAT really necessary?

After performing loads of system, integration, and regression testing, one would wonder about the necessity of this testing. Speaking, this is the most important phase of the project as this is the time at which the users who are going to use the system would validate the system for its fit to purpose.

UAT is a test phase that largely depends on the perspective of the end-users and the domain knowledge of a department that represents the end-users.

It would be helpful to the business teams if they were involved in the project quite early so that they can provide their views and contributions that would help effective usage of the system in the real world.

User Acceptance Testing Process

The easiest way to understand this process is to think of this as an autonomous testing project – which means it will have the plan, design, and execution phases.

The following are the pre-requisites before the planning phase begins:

#1) Gather the key Acceptance Criteria

In simple terms, Acceptance criteria are a list of things that are going to be evaluated before accepting the product.

These could be of 2 types:

1. Application Functionality or Business Related: Ideally, all the key business functionality should get validated, but due to various reasons, including time, it is not practical to do it all. Thus, meeting with the client or users involved in the testing can provide insights into the extent and focus of the testing.
2. Contractual: We will not go into this and the involvement of the QA team in all this is almost nothing. The initial contract that gets drawn up even before the SDLC begins is reviewed and an agreement is reached upon whether all the aspects of the contract have been delivered or not.

We are going to focus only on the application functionality.

#2) Define the scope of QA involvement.

The QA team’s role is one of the following:

1. No Involvement: This is very rare.
2. Assist in this testing: Most common. Here, our involvement could train the UAT users on how to use the application and be on standby during this testing to make sure that we can help the users in case of any difficulty. Sometimes, besides being on standby and assisting, we might share their responses and record the results or log bugs, etc., while the users perform the actual testing.
3. Perform UAT and present Results: If this is the case, the users will point to the areas of the AUT that they want to evaluate and the evaluation itself is performed by the QA team. Once completed, the QA team presents the results to the clients/users, who then decide if the results meet their expectations and if they are satisfied enough to accept the AUT. The decision is never that of the QA team.

Depending on the case on hand, we decide which approach is best.

The primary objectives and expectations:

Usually, UAT is undertaken by a Subject Matter Expert (SME) and /or a business user, who might be the owner or the customer of a system under test. Similar to the System testing phase, the UAT phase also encompasses religious phases before it is brought to closure.

Key activities of each UAT phase are defined below:

UAT Governance

Similar to system testing, the team enforces effective governance for UAT to ensure strong quality gates along with the defined Entry and Exit criteria (provided below **).

** Remember that this is only a form of guidance. The project’s needs and requirements may dictate modifications.

UAT Test Planning

The process is almost the same as with the regular test plan in the system phase.

The most common approach followed in most of the projects is to plan for both system and UAT testing phases together. For more information on the UAT test plan along with a sample, please check out the attached test plan document’s UAT sections.

User Acceptance Test Plan

(This is the same that you would find on our site for the QA training series as well).

Click on the below image and scroll down to find the test plan document sample in various formats. In that template, check the UAT section.

The dates, environment, actors(who), communication protocols, roles and responsibilities, templates, results, and their analysis process, entry-exit criteria–all of this and anything else that is relevant will be found in the UAT test plan.

Regardless of the QA team’s level of participation, we need to plan this phase thoroughly and consider all aspects.

=> Here is a User Acceptance Test Plan Sample Document

User Acceptance Testing Design

The gathered acceptance criteria from the users are used in this step. Samples could look as shown below.

(These are excerpts from CSTE CBOK. This is one of the best references available about this testing.)

User Acceptance Testing Template:

Based on the criteria, we (the QA team) give the users a list of UAT test cases. These test cases are not different from our regular system test cases. They are just a subset as we test all the applications as opposed, just to the key functional areas.

In addition to these, the data, templates to record test results, administrative procedures, defect logging mechanisms, etc., have to be in place before we move to the next phase.

Test Execution

Usually, when possible, this testing happens in a conference or a war room sort of setup where the users, PM, and QA team representatives all sit together for a day or two and work through all the acceptance test cases.

Or in the case of the QA team performing the tests, we run the test cases on the AUT.

Once all the tests are run and the results are in hand, the Acceptance Decision is made. This is also called the Go/No-Go decision. If the users are satisfied it’s a Go, or else it’s a No-go.

Reaching the acceptance decision is typically the end of this phase.

Tools & Methodologies

Typically, the type of software tools that are used during this testing phase is similar to the tools used while performing functional testing.

Tools:

As this phase involves validating the complete end-to-end flows of the application, it might be difficult to have one tool to automate this validation completely. However, to some extent, we will leverage the automated scripts developed during system testing.

Similar to system testing, users would also use test management and defect management tools like QC, JIRA, etc. These tools can be configured to cumulate data for the User Acceptance phase.

Methodologies:

Though conventional methodologies such as specific business users performing UAT of the product are still relevant, in a truly global world like today, User Acceptance Testing sometimes has to involve varied customers across countries based on the product.

For Example, customers would use an e-commerce website across the globe. In scenarios like this, crowd-testing would be the best viable option.

Crowd testing is a methodology where people from all over the world can participate and validate the usage of the product and give suggestions and recommendations.

Crowd-testing platforms are built and are being used by many organizations now. The platform hosts a website or product that requires crowd-testing, and customers can nominate themselves to perform the validation. The feedbacks provided are then analyzed and prioritized.

Crowd Testing methodology is proving to be more effective as the pulse of the customer across the globe can be easily understood.

UAT In Agile Environment

The agile environment is more dynamic. In an agile world, business users will actively participate throughout the project sprints and the project team will enhance it based on their feedback loops.

At the beginning of the project, business users would be the key stakeholders in providing requirements and updating the product backlog. During the end of each sprint, business users would participate in the sprint demo and would be available to provide any feedback.

The team would plan a UAT phase before completing the sprint, where the business users would validate their work.

The feedbacks which are received during sprint demo and sprint UAT, are collated and added back to the product backlog, which is constantly reviewed and prioritized. Thus, in an agile world, the business users are closer to the project and they evaluate the same for its use more frequently, unlike the traditional waterfall projects.

UAT Team – Roles & Responsibilities

A Typical UAT organization would have the following roles and responsibilities. The project manager would support the UAT team, and development and testing teams based on their needs.

7 Challenges Of UAT And Mitigation Plan

It doesn’t matter if you are a part of a billion-dollar release or a startup team, overcome all these challenges to deliver successful software for the end-user.

#1) Environment setup and deployment process

Carrying out this test in the same environment used by the functional test team will certainly end up overlooking the real-world use cases. Also, crucial testing activities like performance testing can’t be carried out in an environment with incomplete test data.

A separate production-like environment should be set up for this test.

Once the UAT environment is separated from the test environment, you must control the release cycle effectively. An uncontrolled release cycle may lead to different software versions on the test and the UAT environment. If you do not test the software on the latest version, it wastes valuable acceptance test time.

Meanwhile, the time required for issue tracking on incorrect software versions is high.

#2) Test Planning

This testing should be planned with a clear acceptance test plan in the requirement analysis and design phase.

In strategy planning, the set of real-world use cases should be identified for execution. It is very important to define the test objectives for this testing as a complete test execution is not possible for large applications in this testing phase. Testing should be carried out by prioritizing critical business objectives first.

This testing is carried out at the end of the testing cycle. It is the most critical period for the software release. Delay in any of the previous stages of development and testing will eat up the UAT time.

Improper test planning, in the worst cases, leads to an overlap between the system testing and UAT. The team deploys the software to this environment even if functional testing is not completed due to less time and pressure to meet deadlines. The core goals of this testing can’t be achieved in such situations.

The team should receive and review the UAT test plan well in advance of starting the test. This will assist them with test planning, test case writing, test script creation, and UAT environment setup.

#3) Handling new business requirements as incidents/defects

Ambiguities in requirements get caught in the UAT phase. UAT testers find issues arising due to ambiguous requirements (by looking at the complete UI which wasn’t available during the requirement gathering phase) and log it as a defect.

The customer expects these to be fixed in the current release without considering the time for the change requests. If a timely decision is not taken by the project management on these last-minute changes, then this could lead to the release failure.

#4) Unskilled testers or testers without business knowledge

When there is no permanent team, the company selects UAT staff from various internal departments.

Even if the staff is well familiar with the business needs, or if they lack training for the new requirements being developed, they cannot effectively perform UAT. Also, a non-technical business team might face many technical difficulties in executing the test cases.

Meanwhile, assigning testers at the end of the UAT cycle does not add any value to the project. Little time to train the UAT staff can significantly increase the chances of UAT success.

#5) Improper Communication Channel

Communication between remote development, testing, and the UAT team is more difficult. Email communication is often very difficult when you have an offshore tech team. A small ambiguity in incident reports can delay its fix for a day.

Proper planning and effective communication are critical to effective team collaboration. Project teams should use a web-based tool to log defects and questions. This will help to distribute the workload evenly and avoid reporting duplicate issues.

#6) Asking the Functional test team to perform this testing

There is no worse situation than asking the functional test team to perform UAT.

Customers offload their responsibility to the test team due to a lack of resources. The whole purpose of this testing gets compromised in such cases. Once the software goes live, the end-users will quickly spot the issues that are not considered real-world scenarios by the functional testers.

A solution to this is to assign this testing to dedicated and skilled testers having business knowledge.

#7) The Blame Game

Sometimes business users just try to find reasons to reject the software. They may demonstrate their superiority by blaming the development and testing team to gain respect from the business team. It’s a rare occurrence, but internal politics can impact teams.

It’s very difficult to deal with such situations. However, building a positive relationship with the business team would help to avoid the blame game.

I hope these guidelines will certainly help you execute a successful user acceptance plan by overcoming various challenges. Proper planning, communication, execution, and a motivated team are the keys to successful user acceptance testing.

System Testing Vs User Acceptance Testing

The involvement of the testing team starts quite early in the project, right from the requirement analysis phase.

All through the project life cycle, some kind of validation is performed for the project, i.e. Static testing, Unit testing, System testing, integration testing, end-to-end testing, or regression testing. This leaves us to understand better about the testing performed in the UAT phase and how different it is from the other testing performed earlier.

Though we see the differences between SIT and UAT, it is important that we leverage synergies but still maintain the independence between both phases, which would enable faster time to market.

Conclusion

UAT is not about the pages, fields, or buttons. The underlying assumption even before this test begins is that all that basic stuff is tested and is working fine. It would be a disaster if the users discovered such a basic bug – the QA team would be devastated.

This testing is about the entity that is the primary element in the business.

Let me give you an example: If the AUT is a ticketing system, the UAT is not going to be about, searching for the menu that opens a page, etc. It is about the tickets and their reservation, the states that it can take, its journey through the system, etc.

Another example is if the site is a car dealership site, then the focus is on the “car and its sales” and not the site. Hence, the core business is verified and validated, and who is better to do it than the business owners? That’s why this testing makes the most sense when the customer is involved to a major extent.

UAT is also testing at its core, so there is a good chance of identifying some bugs at this phase too. It sometimes happens. Aside from the fact that it is a major escalation on the QA team, the UAT bugs usually mean a meeting to sit and discuss how to handle them as following this testing, there is usually no time to fix and retest.

The decision would be either to:

• Push the go-live date, fix the issue first, and then move on.
• Leave the bug as it is.
• Consider it as a part of the change request for future releases.

UAT is classified as Alpha and Beta testing, but that classification is not that important in the context of typical software development projects in a service-based industry.

• Alpha testing is when UAT is carried out in the software builder’s environment and is more significant in the context of commercial off-the-shelf software.
• Beta testing is when the UAT is carried out in the production environment or the client’s environment. This is more common for customer-facing applications. The users here are the actual customers like you and me in this context.

Most of the time in a regular software development project, UAT is carried out in the QA environment if there is no staging or UAT environment.

In short, the best way to find out if your product is acceptable and fit for purpose is to actually put it in front of the users.

Organizations are getting into the Agile way of delivering, business users are getting more involved and the projects are being enhanced and delivered via feedback loops. All being done, the User Acceptance phase is considered the gate for getting into implementation and production.

What was your UAT experience? Were you on standby or did you test for your users? Did the users find any issues? If yes, how did you deal with them?

=> Also read ALL tutorials in this series here

=> Visit Here For Complete Test Plan Tutorial Series