Usability Testing Tutorial: A Complete Getting Started Guide

This comprehensive guide to Usability Testing explains what is Usability Testing, why we need it,  its methods, examples, process, and tools:

We test a software product, website, or service to measure the quality of the deliverables. This test consists of both functional and non-functional testing. Usability testing is one of the many non-functional testing types.

Usability Testing is a realm of the IT world that tries to incorporate how “people” would like their applications to be– which is never easy. It is like trying to read minds. What sounds like common sense, is a profound science when it comes to testing usability.

In this article, we will learn and understand the necessity of usability testing, understand its various categories. We will also see the different tools and methods. Learn about different phases of Usability testing and ways to analyze the results.

Usability Testing

Why Is Usability Testing Necessary

Why Usability Testing

[image source

We create websites or products or services for users or customers to provide solutions for their requirements. By performing usability testing, we can understand whether the created solution satisfies the customer or user requirement or not.

It may happen that the service or product created can be working as expected in functionality, but is not user friendly. Hence, it cannot attract the users or customers of the product and provide a better user experience. This will eventually impact the business of the company, as users would find other products or services much easier to use.

What Is Usability Testing

It is a non-functional type of software testing. It is broadly divided into understandability, learnability, operability, attractiveness, and compliance. Usability testing is to determine the extent to which we understand the software product, easy to learn, easy to operate, and attractive to the users under specified conditions and requirements.

This type of testing is usually executed by real-life users and not the development team. The development team is the one who has created the product, and hence they fail to find fewer defects that are related to user experience.

Usability testing is mainly divided into three categories.  These are:

  • Explorative
  • Assessment
  • Comparative

Let us understand these categories in detail:

#1) Explorative

We usually consider this category in the early stages of the software testing process. The earlier usability testing is executed in the testing process, the minimum is the risk in the product. The design of the product and concepts related to the product or service is usually considered for usability testing in this stage.

#2) Assessment

This category describes the assessment of the end to end execution of the test and also analyze the effectiveness of the product and user satisfaction.

#3) Comparative

In this category, two or more products of similar technology are compared at different attributes as the design of the product, advantages, disadvantages of the product or services, and which helps in selecting the product that provides a better user experience.

As the name suggests in the comparative category of usability testing, we compare two or more similar kinds of products and understand the features, working, pros and cons of the product. This helps to decide the areas of improvement in the product build to attract the customers and rule the market.

Usability Testing Methods

Following are the usability testing methods that are most commonly used:

#1) Hallway Testing

This method is one of the most effective and cost-efficient methods available when compared with others. In this method, few random people are given the website or product to test rather than trained professionals.

As random people test the service without prior knowledge of the product, they test it more effectively and provide more accurate results and honest feedback for improvement, if any.

#2) Remote Usability Testing

As the name suggests, remote usability testing takes place by people who are located at remote locations, i.e. who are located in several states or sometimes in several countries for testing purposes. This type of testing is done remotely and issues can be reported if found.

Feedback can be recorded and can be submitted by random people and not a technology expert. Sometimes remote testing is executed using video conferencing. This type of usability testing adds less cost value as compared to other types of testing.

#3) Expert Review

The expert in the field is asked to test the product or service and provide feedback and then submit the results. This type of testing is usually expensive, as the company needs to hire a technology expert. An expert review can also be done remotely to submit the results.

This type of usability testing is executed quickly and takes less time. The expert quickly finds the loopholes and detects the flaws in the product or service. Such testing is usually expensive, hence clients avoid this option.

This testing is reliable because of the experience possessed by the expert. Expert review is worth the expense when the product has a critical feature, and the company needs to find the user’s reaction before launching the product.

#4) Paper Prototype Testing

Paper prototype testing is one of the most traditional approaches for usability testing. This method involves a dry run of the test execution, hand sketch, drawing of the models or prototype.

paper prototype

[image source

Discussing the flow and drawing them on paper and considering all the possible inputs and scenarios and conditions is the objective of this type of testing. This is one of the primary types of testing which is mostly observed across all the projects to eliminate primary issues. By performing paper prototype testing, one can have more clarity on the execution process.

The paper prototype testing is usually done within the project team. Hence it is considered in the earlier stages of the testing process. This is relatively a cheaper method of usability testing but is not the most effective way of testing as it is more time-consuming at times and there is a higher possibility that even after testing we may lose a few issues.

#5) Automated Usability Testing

As the name suggests, this testing method is performed by writing automation scripts. We write scripts for automation and when the scripts are triggered, test cases are executed. After the execution of the test, results are recorded and submitted.

For this type of testing method, a company needs to hire a resource who is well acquainted with writing scripts and building an automation framework. This is one of the most used testing methods as this is not as expensive as the expert review type of testing and not as primary as the paper prototype testing method.

This is one of the effective types of usability testing as there is less human intervention because of automated scripts and less likely to lose any issues.

Usability Test Tools

There are many tools that help this process along. All these tools can be roughly categorized as follows:

Category #1) Create tasks/tests and give them to users (finding the users and giving them tasks is a manual activity, outside of the tool). While they are performing these tasks, the facilitator could watch their screen and interact with them. This could be in the lines of how you would “Skype”.

Category #2)  Tool provides users or you can pick your own users. You can submit your page/design and the tasks to be performed. The tool, in turn, will provide you with the videos of the user interaction plus the user’s comments. You can make your own analysis.

Category #3) Tools that use eye-tracking and heatmap methods to determine which part of the page the user has spent the most time on. Some of the tools in this category also record the user’s clicks, scrolls, mouse moves, etc.

Category #4) Tools that provide you with feedback based on the website, page or wireframe that you submit as input. Some tools of this type also provide surveys that help in giving conclusive evidence regarding usability issues.

Category #5) Tools that recruit users for your usability test.

The above is a very broad classification. There are many other tools. And also, the division into a certain category is not always so clean. Sometimes the tools employ multiple methods at the same time.

Here is a list of tools category wise and the link to find more info about them. 

usability Testing Tools

Using different tools for usability testing can help the user complete the task in a more effective and efficient way. These tools help in the identification of a better user experience. Not all the projects use tools for usability testing as they are expensive and clients do not want to invest in tools for this purpose.

CrazyEgg is one of the most popularly used usability testing tools. It is a click-based user experience tool which makes it easy to use.

Here is the list of most popularly used tools:

  1. UXtweak
  2. CrazyEgg
  3. Userzoom
  4. Loop11
  5. Optimizely
  6. TryMyUI
  7. UserTesting.com
  8. Qualaroo
  9. Usabilla
  10. UserFeel
  11. Userlytics
  12. Validately.com

Further reading => List of Best Usability Testing Tools

Advantages and Disadvantages

Advantages of Usability Testing:

  • By executing a usability test, we can improve on the flaws of the product, if found any, and rectify them before the launch of the product. Executing tests will thereby help in boosting product quality.
  • Many times internal discussions are headed towards debates that can be resolved after performing usability testing, which helps in understanding the user experience better.
  • Sometimes it is observed that the development team fails to detect a few minor defects which can be found by executing usability testing.
  • As the feedback is directly from the user side, it becomes beneficial for the business to improve the product according to the target audience.

Disadvantages of Usability Testing:

  • Financial constraint is one of the major disadvantages as one needs to recruit and manage the resources for usability testing.

Different Participants or Elements

The participants of usability testing depend on the complexity and budget of the project. If the project is highly complex and contains crucial and critical features, then more participants are considered. Whereas a small team of participants is considered for testing the mid-level complexity project.

In most cases, there is a researcher and a few participants for usability testing. The number of participants ranges between three to twenty-five, while any team of participants with five to ten members is mostly observed.

core elements of usability testing1

The researcher is also known as a facilitator or moderator, where the researcher is the one who observes the participants and moderates the task performed by them and also records the feedback.

Mainly there are three elements, which are as follows:

Researcher, Participant, and Task to be performed.

flow of info

The participant is the one who executes the task whereas the facilitator guides the participants performing the task. The facilitator and the participants have brainstorming sessions among themselves so that participants can test the service more effectively and in the right direction.

The facilitator also checks if the results submitted are accurate. The researcher passes the instructions to the participants via task sheets or verbally or on video conferencing if testing is performed remotely. The facilitator can ask cross-questions to the participants just to check that the task performed by participants is on track.

If the researcher found any deviation in the task performed, then the researcher can control the execution flow.

Different Phases Of Usability Testing

Let us understand how to start usability testing? OR what are the different phases of usability testing?

Same as for other testing techniques, usability testing also should be started as early as possible. Different stages of includes planning, recruiting, execution, data analysis, and reporting.

steps of usability-testing

[image source

#1) Planning

This stage is the first stage in software testing. The planning stage is when the objective of the project is defined. The documentation of the project is done in the planning stage. This is one of the most fundamental and critical stages in software testing.

In this stage, we plan the road map to proceed with the testing process. We plan for questions like what to test? how to test? who would be testing which features? which different scenarios should be considered for testing?. In the planning stage, we gather the business requirement and prepare the plan.

#2) Recruiting

In this phase, we recruit the facilitator and the participants according to the budget and complexity of the project.

#3) Execution

In this stage, the participants perform usability testing and perform the assigned task.

#4) Data Analysis

During the data analysis stage, the feedback received from usability testing is analyzed. Results are categorized and patterns are identified. The conclusion is generated from the feedback submitted to improve the product and thereby improve the business.

#5) Reporting

The feedback and findings received are shared with the customers and the development teams across the project. Discussion on the areas of improvement and its implementation is considered in this stage.

Process of Usability Testing

[image source

Factors To Consider While UX Testing

First and foremost, the earlier you test the better it is to prevent risk, wastage of time, and budget of the project.

  1. Areas of improvement can be handled more easily if testing has started at an early stage.
  2. Select a team of users who would be executing the task.
  3. We need to understand the target audience to know more about the user experience and the product. This will help us understand who will be using the product, what are the needs of the user of these categories. Purpose of using the product. Is the product able to satisfy the users?
  4. Possible ways of using the product and additional features which the user might look for in the future.
  5. Discuss the improvement areas with the development team and work with them accordingly.
  6. We also need to consider the performance and the rate of response while testing the product.

Usability Testing Examples

For analyzing testing results, we should always think through users’ points of view. We need to keep in mind the features that will impact and improve the user experience. To understand this better, let us discuss a few examples below.

Example 1: Usability Testing Of A Travel Website.

Consider the below scenario for testing a travel website.

#1) Booking a flight

Usability testing of a travel website

While considering scenarios for booking a flight, the user should be able to select one-way and return flight options. One option which can improve the user experience is if the user could select the multiple cities option.

#2) Booking of hotels

Booking a flight

While considering that the user is traveling to a city, he/she may also look for accommodation. If a user could find both flight booking and hotel accommodation in one place, can add up to user satisfaction. As the user does not have to browse ten different websites. Everything is taken care of in one place and hence more users will be attracted towards the website.

#3) Renting a car/two-wheeler

Renting a car

While the user is planning to travel to a city for vacation or for small business, they may require a vehicle for traveling locally. If the website has the option of renting the vehicle, it may attract the users to the website positively.

#4) Tracking the flight

This is one of the important features which customers look for before arriving for the flight.

This screen should show the flight status of both from arrival and departure as per the source city mentioned. Also, for a better user experience, the page should contain search options for a quick search based on Airline, Time, or Destination.

#5) Things to do/best places to visit

This could be an extra feature that will interest the users of the website. Through this feature, the user gets to know about the place better and things to explore around. This can add the possibility that the user would rent a vehicle for local transport. If the website has the feature of renting a vehicle and places to visit around, this will add an advantage.

#6) Managing past booking

This feature is of great help for both the user and the business owner as the user can easily repeat the recently traveled option if the user has similar requirements. Because of this feature, the business owner can understand the type of user and the type of bookings made, which can help in providing better deals and offers to the user to improve the business.

This feature is of great help in case if the user needs to submit a receipt of the bookings made for reimbursement, which just got misplaced. Because of this feature, the person can get a duplicate receipt from the website.

As this feature helped the person, in future the person could consider this website and would be the mouth of a good word to others.

#7) Customer Service

Besides all the features we have on the website, if the user comes across any difficulty or needs any help, customer service is the feature that can be looked for. This is a very critical feature as it directly relates to customer satisfaction, which is our objective while usability testing.

Getting connected to customer service via call, email, or chat. Posting frequently asked questions will also help customers.

Let’s discuss another application

Example 2: Getting Transportation Online Using Uber

#1) Booking a ride

This is the primary functionality of the application that even if 100’s users are trying to book the ride at the same time, the user should be able to book the ride.

Booking a ride with uber

#2) Types of ride

The user should be able to view and book all the types of rides available starting from micro, mini, prime, Sweden, carpool, etc. The number of available seats in the ride is also displayed with the type of ride.

type of ride

#3) Estimated time of arrival

While booking it should display the estimated time of arrival so that the user can decide between the different ride options available.

#4) Estimated cost while booking the ride

While booking it should display the estimated cost so that the user can book from the different ride options available.

#5) Ride later option

If the user wants to pre-book the ride, so the person should be able to book a ride of personal desire.

Ride later option

#6) Share ride option

If the user has booked a ride for his friend and wants to share the ride details with his friend. The person should be able to share the ride details.

#7) Adding different payments

The user should be able to pay in all the modes. Like a credit card, debit card option, UPI, wallet, or cash payment. The application should ask for confirmation before saving the credit card, debit card credentials.

Adding different payments

#8) Customer support

Users should be able to connect to customer service via call, email, or chat. Posting frequently asked questions will also help customers. It is also helpful in case if the person by mistake forgot any belongings, or if the driver has charged him incorrectly, also if the driver tried to misbehave or if the user wants to file a complaint or seek any kind of help.

This feature proved to be one of the most important features for users seeking help.

uber help

#9) Cancel the booked ride

If the user booked a ride just by mistake and wishes to cancel, then the user should be able to cancel the booked ride. 

#10) Call the driver

It may happen that the driver is not able to locate the rider at the pickup location, or the user is at the pickup location and not able to find the driver. This feature reduces confusion and saves time if the driver is not aware of the location, he can contact you and reach you easily.

What Happens After Usability Testing

While considering the above example, the features mentioned are the ones received as feedback from the user while using the travel website and online vehicle booking application. There could be few more required features depending upon the customer requirements and usage of the website.

After completion of the usability testing, the first thing to do is to bring together all the feedback and conclusions and then to discuss what all we can be accommodated as improvement of the product/service. How much time will be required to incorporate the changes? If the project team is not clear about the changes to be implemented, then they can discuss again and clear the ambiguities.

The impact of the enhancements depends on the target audience who is going to use the application, the purpose of the application, the features that need to be added or improved on the website. After all of the above points are discussed, it is a management call while considering the features of the product. This also depends on the business impact of the feature.

Frequently Asked Questions

Q #1) What is usability testing in software testing?

Answer: It is a non-functional type of software testing which is used to determine how easily the users can understand and use the product.

Q #2) How is Usability Testing done?

Answer: Mostly a dedicated team performs usability testing. The team consists of participants who perform the task assigned to them and report the observation. Whereas the facilitator is the one who moderates the team.

Q #3) What is the purpose of Usability Testing?

Answer: The product developed should satisfy the user requirement. By performing usability tests, we can find about user experience and improve the product where there is a scope of improvement.

Q #4) What are the three main principles of Usability Testing?

Answer: These are:

  • It deals with product development.
  • It involves studying real users as they use the product and understand their user experience.
  • It involves improvement in the product according to user experience and setting goals and a timeline to achieve that.

Q #5) What are Usability features?

Answer: There are five characteristics of Usability. These are as follows:

  • Effective
  • Efficient
  • Engaging
  • Error Tolerant
  • Easy to learn

More About Usability Testing

usabilitytesting

[image source]

A usability test makes sure that the interface of an AUT is built in a way that fits the user’s expectations with respect to meeting requirements (effectiveness) easily (efficiently) in a simplistic satisfying manner.

The primary focus is on:

  • Ease of use
  • Ease of Learning or familiarizing with the system
  • The satisfaction of the user with the entire experience

Usability has many dimensions to it. It is all about the user’s ‘experience’ during their interaction with an application and their ‘feeling’ towards it. A structured Usability Test translates this experience/feeling into a Validation Process.

Why It Is Performed

The Web and mobile applications rule the business world in recent times. These apps being efficient, effective, easy, simple, appealing, engaging, etc. is very critical for them to be embraced by the customers. The Usability Test is all about determining if a site is what the user would want to use and come back to or not.

This not only applies to software systems. Any machine/interface that has human interaction has got to satisfy these rules. How do you ask? Democracy would suffer if the voting machines were not usable.  I wouldn’t vote if I had to click more than one button to choose my candidate, would you? Exactly!

For a more software-specific example, check out this 300 million dollar article by Jared Spool that will clearly explain how the placement of a button has caused the business to be impacted.

When Is Usability Testing Conducted

As testers, we know that the earlier a defect is found in the SDLC the cheaper it is to fix it. The same concept holds true for testing Usability also.

Usability Test results affect the design of the product. So, ideally, the usability test should start at the design level. But that is not all; software undergoes many changes/interpretations/implementations throughout the SDLC process. To make sure that we do not make usability related mistakes at any of these steps – this testing should be conducted often and continuously for maximum results.

Who Performs This Testing

It can be done as an internal process, when the designers, developers, and anyone else can sit down and analyze their system and get the results. Based on these results, the design and/or code can be modified to be in accordance with the changes they all agree on.

A more advanced approach is to hire real-time users and give them particular tasks. A facilitator/s can devise these tasks and get the results from the users.

The users can then provide information, on whether:

  • The task was successful or not
  • The task could be performed easily
  • Was the experience interesting, engaging or annoying – their feeling towards the software

How To Conduct Usability Testing – Process

Testing is a validation of software against its requirements.  A usability test is not different – The only requirement, in this case, is to validate if the software is as per a mental map of how a user would want the software to be like, what makes it comfortable for them to use, what kind of holistic experience is the user going to take away from the interaction, etc.

These are just a few of the ways in which this testing is carried out.

#1) During the design phase, you could just take draw your website/application design on a piece of paper and evaluate whether it is going to work or not.

#2) An exploratory method would be to build the site and perform some random tests (by the development/design/QA- any or all internal teams) to determine usability factors.

#3) Hire a set of real-time users to work on the site and report results.

#4) Use a tool that would provide statistics based on the input wireframes and designs submitted.

#5) Hire a third-party usability team that specializes in this field.

#6) Submit your site design and wireframes to an external evaluator and get results from them.

The structured Usability Testing process contains the following steps:

Step #1) Identifying the users to perform the Usability Test – it helps to choose the set of users that is close to how the real-time users are going to be. Care has to be taken not to pick experts or complete newbies. The experts are going to simply run through the entire process and the novices need lots of background training to even get started- neither situation is optimum.

Step #2) Designing the tasks that the users are going to perform on the application –A list of situations that the users are going to use the application form is to be made prior to starting the test. This can include something like: ‘Search for an X-box and buy it” or ‘submit a customer care question” etc. on an eCommerce site. The tasks should closely represent the real transactions the users would use the site for.

Step #3) Facilitating the testing – The usability team will have the users perform the tasks on the site and are going to gather information regarding the test progress and results. It really makes a huge difference when the design team participates while the users are working on the application. It gives them a better picture of how the app was used and where it did not deliver what the user wanted etc, firsthand.

Step #4) Analyze results – At the end of the test, we might end up with the time it took to perform tasks, whether the task was successful or not, etc, so basically raw data. The results have to be presented to all the stakeholders and analyzed for the identification of potential problem areas.

From what we have discussed so far, these are my impressions:

#1) Testing Usability is not an out and out QA task. A role that traditional QA teams could play in this context is that of a facilitator who would create tasks, conduct the test and interpret the result. The ‘tester’ in this case is actually the user.

#2) The end results or outcome of usability testing are not defects. It is simple information on how the user felt, what they liked, what they did not – basically raw data. Usability tester’s job does not end with reporting the results (in case of traditional QA jobs, we report bugs and do not get involved in fixing them), it includes providing suggestions on how to make the software more conducive to user interaction.

#3) By understanding these concepts, we can actually make a difference as functional testers. To add more value to our role as QAs we can apply these usability concepts and provide insightful suggestions on how to improve the usability of an application.

Conclusion

In the above tutorial, we have learned in detail about Usability testing in the software testing life cycle. The tutorial also covers topics like why is usability testing necessary, its advantages and disadvantages, different participants or elements of usability testing, etc.

We have also covered how to analyze testing results, different Usability testing tools in detail with examples.

Usability testing is necessary as it gives the chance for improvement before the launch of the product. This improvement can thereby boost the business positively. Hence usability testing is extremely important and should be performed to provide better deliverable products to the customers.

Also read: A complete guide to testing web applications.

Author: This article is written by STH team member Swati S.

I hope you find this article helpful to understand usability testing. Please add your comments and queries in the comments below. 

Recommended Reading

24 thoughts on “Usability Testing Tutorial: A Complete Getting Started Guide”

  1. Hi Swati, Its very nice article.Can you also post an article on Accessibility Testing. As Accessibility Testing is a subset of Usability (as per the standard definitions) it would be helpful if you highlight on Accessibility Testing aspect in detail.

    Reply
  2. Hello Swati,

    This article really helped us to improve our knowledge. There are lot of terms which confuse you about usability but few of the points mentioned above, pinpoints few important issues user can face while doing this testing.

    Reply
  3. @Deepika: GUI testing is more functional, because we want to make sure the functional elements in the interface are working as they are supposed to. Usability test is not able whether its working or not, its more about how it feels to work with a certain element. Having said that, it is not very easy to differentiate between both- there is only a thin line of seperation

    Reply
  4. @Shiv: I am glad you found the article useful. This type of testing is definitely very interesting. I was hooked to the topic while I was researching for this project.

    Reply
  5. HI Swati & Vijay, hope you guys doing good.

    The articles which you guys posting it’s very very useful, day by day am learning new things in testing because of you guys Thank you soo much Swati & Vijay.

    If possible can you post the article on Mobile Testing.

    Reply
  6. Thanks a ton for providing the list of tools in the article. These links are very helpful. I always enjoy reading articles on STH..:)

    Reply
  7. Dear Swatiji
    you have given very well structured & explained information about the UT.
    my dought about U.T. are fully cleared.
    Thanks.
    Please provide the article on security with study cases.

    Reply
  8. Hi Swati,
    I am just going to write usability test cases for gateway i.e my DUT. can you please help me how to write these cases
    , basically what will be test suite.please explain in deatil.thnks

    Reply
  9. Hey, I work for the company which sells test equipment. We have planned to introduce types of equipment which will be used to test IoT devices. Can anyone help with the keyword research? I want to know the common keywords end consumer will use to find this type of equipment.
    For example: ” iot testing equipment”

    Reply
  10. Hi Swati and Vijay,

    Great article guys, it has covered almost all the necessary topics which are helpful to jump start the Usability Testing.

    Can you share something which focuses on making Product Owners and Stake holders understand the importance of UI / UX in the process of Software Development.

    Also, how to implement the UX on the software products which are there in market since a decade and heavily used by customers. So that the redesigning should not affect or create delays to the current software in the market

    Reply

Leave a Comment