We will discuss, What is User Acceptance Testing (UAT) – its definition, types, steps, detailed process etc. in this tutorial.
My rule number one when trying to understand a new concept is that: the name is always going to be relevant and mostly a literal meaning (in the technical context). Finding out what that is, will give me an initial understanding of it and gets me started.
Let us put this concept to test.
What You Will Learn:
What is User Acceptance Testing (UAT)?
We know what testing is, acceptance means approval or agreement. The user in the context of a software product is either the consumer of the software or the person who requested it to be built for him/her (client).
So, following my rule – UAT means testing a software by the user/client to determine whether it can be accepted or not – the definition. Give this method a try with other technical words too. Trust me, it works every single time.
This is final testing performed when functional, system and regression testing is completed. The main purpose of this testing is to validate the software against the business requirements. This validation is carried out by end users who are familiar with the business requirements. UAT, alpha and beta testing are different types of acceptance testing.
As user acceptance testing is the last testing carried out before the software goes live, obviously this is the last chance for the customer to test the software and measure if it’s fit for the purpose.
When is it performed?
This is typically the last step before the product goes live or before the delivery of the product is accepted. This is performed after the product itself is thoroughly tested (i.e after system testing).
Who performs UAT?
Users or client – This could be either someone who is buying a product (in the case of commercial software) or someone who has had a software custom built through a software service provider or the end user if the software is made available to them ahead of time and when their feedback is sought.
The team can be comprised of beta testers or customer should select UAT members internally from every group of the organization so that each and every user role can be tested.
Need for UAT
Developers and functional testers are technical people who validate the software against the functional specifications. They interpret the requirements according to their knowledge and develop/test the software (here is the importance of domain knowledge).
This software is complete according to the functional specifications but there are some business requirements and processes which are known to end users only are either missed to communicate or misinterpreted.
This testing plays an important role in validating if all the business requirements are fulfilled or not before releasing software for the market use. Use of live data and real use cases make this testing an important part of the release cycle.
Many businesses who suffered big losses due to post-release issues know the importance of successful User Acceptance Test. The cost of fixing defects after release is many times greater than fixing it before.
User Acceptance Testing Process
The easiest way to understand this process is to think of UAT as an autonomous testing project – which means, it will have the plan, design and the execution phases.
The following are the pre-requisites before the planning phase begins:
#1. Gather the key acceptance criteria
Acceptance criteria- to simply put, this is a list of things that are going to get evaluated before accepting the product. These could be of 2 types:
A) Application functionality or business related
Ideally, all key business functionality should get validated but due to various reasons, including time, it is not practical to do it all. Therefore, a meeting or two with the client or the users who are going to be involved in UAT can give us an idea on how much testing is going to be involved and what aspects are going to be tested.
B) Contractual – we are not going to go into this and the involvement of the QA team in all this is almost nothing. The initial contract that gets drawn up even before the SDLC begins is reviewed and an agreement is reached upon whether all the aspects of the contract have been delivered or not.
We are going to focus only on application functionality.
#2. Define the scope of QA involvement. QA team’s role is one of the following:
A) No involvement – This is very rare.
B) Assist in UAT – most common. In this case, our involvement could be training the UAT users in how to use the application and be on standby during the UAT to make sure we can help the users in case of any difficulty. Or in some cases, in addition to being on standby and assisting, we might share their responses and record the results or log bugs etc. while the users perform the actual testing.
C) Perform UAT and present results – If this is the case, the users will point the areas of the AUT that they want to evaluate and the evaluation itself is performed by the QA team. Once done, the results are presented to the clients/users and they will make a decision on whether the results that they have in hand are sufficient and in accordance with their expectations in order to accept the AUT. The decision is never that of the QA team.
Depending on the case on hand, we decide which approach is best.
UAT Test planning
The process is almost the same as with the regular test plan for the system phase. The most common approach followed in most of the projects is to plan for both system and UAT testing phases together. For more information on UAT test plan and a sample, check out the attached test plan document’s UAT sections.
User Acceptance Test Plan
(This is the same that you would find on our site for the QA training series as well).
Click on below image and scroll down to find the test plan document sample in various formats. In that template check the UAT section.
The dates, environment, actors(who), communication protocols, roles and responsibilities, templates, results and their analysis process, entry-exit criteria – all of this and anything else relevant will be found in the UAT test plan.
Whether the QA team is participating, partially participating or not participating at all in this test, it is our job to plan this phase and make sure everything is taken into consideration.
The gathered acceptance criteria from the users are used in this step. Samples could look like the following:
(These are excerpts from CSTE CBOK. This is one of the best references available about this testing.)
User Acceptance Testing Template:
Based on the criteria, we (QA team) give them the users a list of UAT test cases. These test cases are not different from out regular system test cases. They are just a subset since we test all of the application as opposed to just the key functional areas.
In addition to these, the data, templates to record test results, administrative procedures, defect logging mechanism has to be in place before we move to the next phase.
Usually, when possible, this testing happens in a conference or war room sort of a set up where the users, PM, QA team representatives all sit together for a day or two and work our work through all the acceptance test cases.
Or in case of QA team performing the tests, we run the test cases on the AUT.
Once all the tests are run and the results are in hand, the Acceptance Decision is made. This is also called the Go/No-Go decision more colloquially. If the users are satisfied it’s a Go, or it’s a No-go.
The reaching of the acceptance decision is typically the end of UAT phase.
7 Challenges of UAT and Mitigation Plan
It doesn’t matter if you are part of a billion-dollar release or a startup team, you should overcome these challenges for delivering successful software for the end user.
#1) Environment setup and deployment process
Carrying out this test in the same environment used by functional test team will certainly end up overlooking real-world use cases. Also, crucial testing activities like performance testing can’t be carried out on test environment with incomplete test data. Separate production like environment should be set up for this test.
Once the UAT environment is separated from test environment you need to control the release cycle effectively. Uncontrolled release cycle may lead to different software versions on test and UAT environment. Valuable acceptance test time is wasted by not testing software on the latest version. Not to mention the time required for issue tracking on incorrect software version.
#2) Test planning
This testing should be planned with clear acceptance test plan in requirement analysis and design phase. In strategy planning, the set of real-world use cases should be identified for execution. It is very important to define test objectives for this testing as complete test execution for large application is not possible in this testing phase. Testing should be carried out by prioritizing critical business objectives first.
This testing is carried out at the end of the testing cycle. Obviously, it is the most critical period for the software release. Delay in any of the previous stages of development and testing eat up UAT time.
Improper test planning, in worst cases, leads to overlap between system testing and UAT. Due to less time and pressure to meet deadlines, software is deployed to this environment even though functional testing is not completed. The core goals of this testing can’t be achieved in such situations.
The UAT test plan should be prepared and communicated to team well before beginning this test. This will help them for test planning, writing test cases and test scripts and creating UAT environment.
#3) Handling new business requirements as incidents/defects
Ambiguities in requirements get caught in UAT phase. UAT testers find issues arising due to ambiguous requirements (by looking at the complete UI which wasn’t available during requirement gathering phase) and log it as a defect.
The customer expects these to be fixed in current release without considering the time for change requests. If a timely decision is not taken by project management on these last-minute changes this could lead to the release failure.
#4) Unskilled testers or testers without business knowledge
If there is no permanent team, company select UAT staff from various internal departments. Even if this staff is well familiar with business needs, if they are not trained for the new requirements being developed they can’t perform effective UAT. Also, a non-technical business team might face many technical difficulties executing the test cases.
Also assigning testers at the end of the UAT cycle do not add any value to the project. Little time to train UAT staff can significantly increase the chances of UAT success.
#5) Improper Communication channel
Communication between remote development, testing, and UAT team is more difficult. Email communication is often very difficult when you have an offshore tech team. A small ambiguity in incident reports can delay its fix for a day.
Proper planning and effective communication are critical to effective team collaboration. Project teams should use the web-based tool to log defects and questions. This will help to distribute the workload evenly and avoid reporting duplicate issues.
#6) Asking functional test team to perform this testing
There is no worse situation than asking functional test team to perform UAT. Customers offload their responsibility to test team due to lack of resources. The whole purpose of this testing gets compromised in such cases. Once the software goes live, end users will quickly spot the issues which are not considered as real-world scenarios by functional testers. Solution to this is to assign this testing to dedicated and skilled testers having business knowledge.
#7) The blame game
Sometimes business users just try to find reasons to reject the software. It might be their selfdom to show how superior they are or blame the development and testing team to get respect in the business team. This is very rare but happens in teams with internal politics.
It’s very difficult to deal with such situations. Building the positive relationship with the business team would definitely help to avoid the blame game.
I hope these guidelines will certainly help you execute a successful user acceptance plan by overcoming various challenges. Proper planning, communication, execution and motivated team are the keys to successful user acceptance testing.
#1. UAT is not about the pages, fields or buttons. The underlying assumption before even the this test begins is that all that basic stuff is tested and is working fine. God forbid, the users find a bug as basic as that – it is very bad news for the QA team. :(
#2. This testing is about the entity that is the primary element in the business.
Let me give you an example: If the AUT is a ticketing system, the UAT is not going to be about, searching, the menu that opens a page etc. It is about the tickets and their reservation, the states that it can take, its journey through the system.
Another example, if the site is a car dealership site, the focus is on the “car and its sales” not the site really. So the core business is what is verified and validated and who better to do it than the business owners. That’s why this testing makes the most sense when the customer is involved to a major extent.
#3. UAT is also a form of testing at its core which means there is a good chance of identifying some bugs at this phase too. It sometimes happens. Aside from the fact that it is a major escalation on the QA team, the UAT bugs usually mean a meeting to sit and discuss how to handle them because following UAT there is usually no time to fix and retest.
The decision would be either to:
- Push the go-live date, fix the issue first and then move on
- Leave the bug as is
- Consider it as a part of change request for future releases
#4. UAT is classified as Alpha and Beta testing, but that classification is not so important in the context of typical software development projects in a service-based industry.
- Alpha testing is when UAT is carried out in the software builder’s environment and is more significant in the context of commercial off the shelf software.
- Beta testing is when the UAT is carried out in the production environment or the client’s environment. This is more common for customer-facing applications. The users here are the actual customers like you and me in this context.
#5. Most of the times in a regular software development project, UAT is carried out in the QA environment if there is no staging or UAT environment.
About the Author: This article is written by STH team member Swati S. She is having 9+ years of experience in software testing.
What was your UAT experience? Were you on standby or did you test for your users? Did the users find any issues? If yes, how did you deal with them?
Share your experiences with us and let us know your comments and questions below.