Mobile App UI Testing Guide: Learn how to perform iOS and Android UI Testing
With the flourishing market for mobiles, testing of mobile applications has become exciting day by day.
Just by running functional tests on a mobile application, you cannot sign off the app. There are few other testing types like field testing, network testing, UI testing, battery life testing, etc., that need to be done.
UI testing is one of the important tests in mobile application testing and it should not be taken lightly.
Graphical User Interface creates a lot of difference in how interesting & interactive a user finds your application. The importance of a decent and attractive GUI can be felt more significantly in a smart device environment where screen size is far smaller as compared to a laptop or desktop screen.
What You Will Learn:
- Mobile App UI Testing Importance
- How to Decide on how much UI Testing is Required?
- Guidelines: What to Test in Mobile App UI Testing
- How to Test UI Variations across different OS Versions?
- Real Devices or Emulators: What to Choose for UI Testing?
- Manual or Automation UI Testing?
- Mobile App UI Testing Tools
- Checklist for Testing Mobile App UI
- 5 Myths On Automated Mobile UI Testing
- The Myth And Reality
- Recommended Reading
Mobile App UI Testing Importance
As a user will you feel like using an app that lacks user interaction and makes it difficult to understand how to use it?
When the users use a mobile application for the first time, it’s not just the performance that steals the attention but also the appealing UI. A UI friendly application sells more when compared to an app which is best developed but with a nasty UI.
If an application has a perfect and a splendid UI on one device but on the other device it is completely twisted just because it has a different size or a different OS, then it will leave a very bad impression. The commercial success of the application will be badly affected.
Will you promote an app where the button is too small to click blocking the whole set of functionality?
Aren’t these unpleasant experiences for the users? Due to the above-mentioned cases, it becomes very important to test the UI of an application. The two major verification that needs to be done for mobile applications is user-friendliness and appearance across different models and OS versions.
Following is an Example of how the UI should be perfect across different screen sizes:
How to Decide on how much UI Testing is Required?
The following chart denotes the various verticals in which mobile apps can be categorized:
From the above chart, you can make out that Gaming apps take the majority of the market share of about 24.43%, and then followed by business and education apps.
- Apps developed as gaming apps need thorough testing at every aspect as the UI is the biggest contributor to gain success no matter if it’s a Native or Hybrid app
- A business app may not completely rely on the UI for its success because in most cases the target audiences are trained to use the app. Hence, such apps may have a simple UI.
- Apps developed for educational purposes need thorough UI testing.
- Commercial apps like shopping, travel, etc also need complete UI testing across devices and various OS versions.
In short, depending on the purpose of the app, the depth of UI testing can be decided but UI testing should always be done on at least 3 different OS versions.
Guidelines: What to Test in Mobile App UI Testing
While testing UI on a mobile application, various characteristics need to be verified.
Following are some of the characteristics that should be tested for every app:
#1) Screen Resolution
Following are some of the common screen resolutions that are considered while creating testbeds:
- 640 × 480
- 800 × 600
- 1024 × 768
- 1280 × 800
- 1366 × 768
- 1400 × 900
- 1680 × 1050
All of these resolutions are a must for testing when you have a multi-column layout in your app.
Hence verification needs to be done starting from the smallest to the biggest resolution. Apart from this, if your app has a long list of cards with information then those also need to be tested on a different resolution for their information wrapping.
#2) Screen Size
There are too many variations in screen sizes and available resolutions. In smart devices especially, controls sizes are not static, they have relation to the available screen size.
While testing, make sure that controls size looks esthetically good and control is completely visible on the screen without any scrolling. Test the GUI on different devices with different screen sizes and resolutions.
Emulators are good for this purpose but nothing matches the real device. So make sure that you test on at least two or three real devices. Also, don’t forget to test on landscape and portrait orientations if the device supports it.
You must test the application under commonly used resolutions to ensure its usability.
Few things that you must understand here are:
The difference between screen size and resolution: The screen size is the length of the screen in inches measured diagonally or from one corner to another corner of the screen. The screen resolution is the width and height, Example 640w × 480h that represents the number of pixels going across the screen multiplied by several pixels going down.
#3) Different UI Elements
The UI elements like buttons, headings, icons, images, selection fields, text fields, checkboxes, etc., are some of the different elements that need to be verified for their appearance and size on the screen.
Specifically for text fields, if the soft keyboard shows up on tap in the text field should be tested and verified.
Most importantly thorough testing of button sizes is required because I remember in our app while testing on Galaxy S phone, we found a blocker where a button had blocked the entire app just because the button appeared too tiny to click on.
The position of the UI elements should also be verified against the requirement i.e. if all are to be center-aligned or left-aligned etc.
#4) Style: Color and Theme Scheme of the Device
The app UI and color scheme should be consistent with different colors and theme schemes of the phone. The color and theme of a Samsung phone are very different from that of Nokia or the MI phone.
Hence you need to verify if the app is looking consistent across such phones.
Your application has a specific design. And the style of the controls should match with that design. You might have seen many applications where some controls e.g. panels have round edges and other controls e.g. text boxes have sharp edges.
Although these type of issues doesn’t affect the usability or functionality of the app, still a consistent look of the application helps to build a friendly relation between the application and the user.
One of the most important things in style is the font of the different pages. The font should be tested well to avoid any inconsistency in the look and feel of the application.
Most of the time, we focus on the text that is visible in normal situations and ignore the text that appears in specific situations. Success and Failure messages are an example of such type of text.
Another factor which important in style is a relation between the font color and the situation in which text is displayed.
For example, Red color is used for Error messages, Green for success, Yellow for warnings, and Blue for hyperlinks.
#5) Multi-touch or Single touch
If your app is supporting the multi-touch feature like pinch to zoom or pinch to shrink etc., then you need to thoroughly test this feature and create a lot of test cases for this for all the applicable screens.
#6) Long or Short Press
A long press on an icon shows the context menu while a short touch performs the very first action of the menu. If this feature is provided in your app, then you need to verify this functionality and all the functionalities around it.
Location and position are the two words that are used alternatively and, interestingly, they are further used to convey two different concepts that are explained below:
1) Sometimes it is the area on the screen where a control appears.
For example, Header is located on top of the page, Labels are Left Aligned, and Textboxes are Right Aligned, etc. Here, ‘top’, ‘left Aligned’, and ‘Right Aligned’ are relative positions of the controls.
2) Sometimes it is the order of control among the other controls.
For example, while getting personal info, First Name is followed by the last name. Or, the format of controls to ask for a US address should be in order– ZIP, City, State.
For both these situations, we are talking about the location of the controls.
While testing for location and position of controls, make sure that everything is logically placed on the screen and shows a good aesthetic sense.
There are situations where one or more controls appear on more than one screen. In this situation, you must make sure that they appear in the same location and the same relative order on all the pages.
How to Test UI Variations across different OS Versions?
The UI varies with the OS version and with the launch of a new version, enhancements are made in the UI.
Let us observe the UI of the 3 latest OS that is currently available and understand how these variations affect a mobile application.
Looking at the above list of new UI or functionality features, as a QA you need to design test cases around this.
- Create test cases for the effect of the new design on your app.
- Not necessarily for all the screens but create test cases for accessing the new shortcuts on your app.
- If your app deals with emojis, create test cases to verify the new emojis. Apps that allow the users to write reviews or to chat are the ones that use emojis frequently.
- When your app is published and installed for the first time, it may need to ask for permission hence the need for UI testing of the new permission screen needs to be done. And create test cases for the same.
- If your app is using Google Now then you need to create test cases to test the updated Google Now feature’s UI.
- Thorough testing of your app needs to be done for the daydream reality mode and hence create test cases accordingly.
- Create test cases to verify the menu options for your app.
- If your app deals with emojis and GIFs, then create test cases to verify the new emojis and the option to send GIFs.
Real Devices or Emulators: What to Choose for UI Testing?
When you have to test a mobile application you may think about what the testbed should be?
Whether to test on a real device or emulator or both? There’s no firm answer to this because the choice depends on what you want to test.
For testing the functionality, performance, network response, field test, etc., you should always prefer a real device. But for things like UI you should choose emulators along with some real devices.
The pros of using emulators for UI testing are:
1) It is not practically possible to collect the devices of all resolutions and that would also cost an enormous amount of money. But emulators cost nothing.
2) With an emulator, you can create all screen resolution and OS combinations.
3) If you have only one set of real devices but the QA team is of more than 1 person, then all QAs can’t test for the same testbed in parallel. With an emulator, every QA can create the same combination on their machine and test in parallel.
4) Testing on an emulator is less time consuming and is faster when compared to a real device.
5) Common bugs related to the UI like alignment etc can be easily caught on emulators.
1) Gestures can’t be tested on emulators. Only one gesture can be emulated at a time.
2) Physical inputs of GPS, dropping or weak network, etc also can’t be tested.
3) There is no way that you can create an emulator for Sony, LG, Nexus, etc phones.
4) It is not possible to create a real environment with a low battery or low memory etc., on the emulator.
Hence the decision should be made depending on your app and the testing requirement.
Manual or Automation UI Testing?
No product whether it a desktop app or web app or mobile app can be released without testing. As a QA we struggle to find and report every defect but still, they are reported by customers.
Do you know why?
Because lengthy tests that are often avoided or missed thus leaving undetected bugs. Also 100% coverage, in-depth execution is not possible with manual testing.
UI testing is pretty simple and straightforward and you just have to look at how it is appearing to your eye. Now if this is done manually it is very time-consuming. Also, most of the times we need to create huge data for UI testing like a scroll will appear only if the rows of cards cross a particular count.
Creating big data is very time-consuming. Having an automated suite can solve both problems.
On the contrary, if the functionalities or UI of the app are still in a changing phase then it doesn’t make sense to invest in automation. Similarly, if the functionalities of the app are vital, then it is better to test manually.
Hence depending on the following pointers, you should decide whether to test manually or to automate:
- The nature of your app.
- The stability of your app.
- The resources available like manpower to study the tools and compare them.
- How much time is invested study and ramp-up of an automation tool that is required?
- Is the client ready to invest time in the ramp-up and study?
Mobile App UI Testing Tools
Following is a list of 5 tools that can be used for UI testing of a mobile application for Android and/or iOS.
(For functionality testing tools you can refer to the list of automation tools on our automation tools for testing Android applications page).
Selendroid is one of the best and most recommended tools for mobile application automation for UI testing.
It can be used for both Native and Hybrid apps. It can be used only for Android apps and the client API tests are written using Selendroid 2. It can also be used with more than one device and is fully compatible with JSON.
This is a cloud-based tool and can be used for a variety of devices, different screen resolutions and OS versions of both Android and iOS. Parallel device testing is a big advantage of this tool and is a good tool for UI testing. It helps the developers to improve the time-to-market.
It is a paid tool and can be used for Android, iOS, Windows, Symbian, etc.
It is a cross-platform tool and hence the advantage is that the same test can be run on all platforms. It can be used for all mobile applications and the tests can be run in parallel on more than one device.
#4) UI Automation
This is the official UI testing tool for Apple and is the best tool for automating iOS applications. Although it is hard to learn, it offers a great advantage with libraries, performance, UI testing, etc.
It can be used for both Android and iOS testing for native or hybrid apps. It is a cross-platform tool and it is best used for automating gestures, screenshots, assertions, etc. It can be used on real touchscreen devices. It also has support for cucumber.
When developers are unit testing the app, they can also do UI testing using Android Studio but it can only be used for Android applications.
Recommended Reading=> Automate user interface tests
Checklist for Testing Mobile App UI
Below is the checklist for testers to ensure that the GUI is tested completely well on smart devices:
|✅||Test overall color scheme and theme of the app on the device.|
|✅||Check the style and color of icons.|
|✅||Test the look and feel of the web content across a variety of devices and network conditions.|
|✅||Test for multi-column layout – check if columns are aligned correctly and viewable even on a lower resolution.|
|✅||Test if progress indicators are visible when pages are loading.|
|✅||Check the Menus and how they are invoked.|
|✅||Check the Items contained in the Menu.|
|✅||Screen Orientation is tested in both portrait and landscape mode.|
|✅||Check the use of virtual keyboard while changing the screen mode.|
|✅||Check the pinch-to-zoom effect through touch screens and trackballs – details should not be distorted on the zooming.|
|✅||Test the sliding effect – should work in a single stroke; next screen must into the screen resolution without distortion|
|✅||Test the buttons sensitivity – should be clickable with any kind of touch (a large fingertip or stylus).|
|✅||Virtual keyboard opens up automatically when the user wants to enter text in any text field.|
|✅||Test if the application is integrated well with the mobile hard keys- start, home, menu, back buttons.|
|✅||Check if the page navigation and scrolling are working fine through the trackball.|
|✅||Test the overall responsiveness of the application on the device.|
5 Myths On Automated Mobile UI Testing
Automated Mobile UI Testing is considered a lot of crucial when the question of application success arises. But there are some myths related to automated testing.
Such myths may not be true because they may be superficial. Delving deep into the process of automated testing makes it disappear. Let us dig deeper into them.
Myth 1: Speed
This myth is very common. Most people related to IT industries have a myth that performing “automation testing” takes more time when compared with “manual testing”. This fact is true to some extent in a few scenarios.
The reason is that manual testing produces fast results when compared with Automated Mobile UI Testing. But this is the case only in preliminary and initial stages.
With repeated kind of testing, you require either addition of a lot more features of testing or diminishing testing qualities. Whereas with Automated Testing, you always run similar testing levels every time thereby results in saving time in the long run.
Myth 2: Coverage
In the present-day scenario, new Android devices are released in markets regularly. And the number of apps of such operating systems (OS) are increasing. Then there are operating systems such as iOS which have even more apps made for daily usage.
Manual testing for so many apps becomes very difficult. But in cases of automated testing maintenance of cloud servers shall suffice. With the help of automated testing, total and complete test coverage of the apps are possible.
Myth 3: Cost
It is a fact that automated testing of the apps costs more compared to the costs of manual testing. However, this is true only if tests are being done for the app’s bare essentials. As the app’s environment and the software gets complicated, manual testing gets more expensive.
This is because more sophisticated tools are needed for optimum test results. Along with those sophisticated testing tools, there is a need for highly trained staff who could manage such tools. This shall require training them.
So manual testing becomes costlier when compared with an automated one.
Myth 4: Consistency
In the case of manual testing, there is always room for different perceptions that vary from one tester to another. This also depends on tests considered, environments, and apps along with Operating System(OS).
When you apply manual testing to the software, there are holes through which few bugs could pass through. Hence manual testing is good only for detecting basic bugs. Automated testing runs on the scripts with no room for varying perception making it foolproof.
Myth 5: Reluctance
It is not true that automated testing has replaced human beings rather it is for manual tester’s betterment. Automated tests provide automated results repeatedly, multiple times with maximum accuracy. So query arises, why there is a need for humans?
Automated testing needs scriptwriting and the whole testing procedure planning. This task requires human effort. The procedure of automated testing helps to save time and money such that you utilize such resources for the betterment of procedures of manual testing. The development of better tools will in turn help in the advancement of already existing procedures of automated testing.
Above mentioned are few of the most popularly existing myths prevailing in the industry of automated testing. This needs to be eradicated for the betterment of Automated Mobile UI Testing.
The Myth And Reality
The fact is that even most sophisticated developmental companies are using manual testing for mobiles, or not conducting complete testings at all. As per Xamarin 2014 surveys, 13.2%of developers of mobiles perform testing being automated UI’s. As per studies of Forrester Research, only 53% of developers perform cursory tests on single devices.
Five most common factors as to why teams of mobiles have not automated qualities of mobile apps and five reasons as to why these do not just make real sense are as follows:
a) Speed is the first myth.
A person cannot take time for automation. In the year 2014, the vendors had introduced 7000 new Android device types. Then there were 10000 APIs that have been specific for mobiles. The application of mobiles ship faster and modify quickly. With Quality Assurance (QA) in constant crunching modes, there is no time for creating test scripts, in turn, keeping them in synchronization with regularly changing features.
The practical scenario of the first myth:
One is presently wasting precious time. It is very true. Testing manually is faster than testing automation-wise. But this is for the very first testing run. On subsequent runs, whatever marginal benefits manual testing brings forth towards erosions. This is almost immediately. Together with all repeated test runs or feature additions, app developers should either scale back testing scope or additional testing resources.
Along with finite budgeting, this ultimately leads to the vicious cycles of those qualities that are diminishing. In response to the data engagements and negative reviews of users from devices that are untested, teams desire an expansion of the device’s coverage. These further increases stress on QA departments already as capacities.
It is that the business struggles for maintaining, researching, and procuring devices while doing test executions. Even best-funded UI manual programs of testing fall shortened towards completed coverage.
In the US, teams of mobile require testing on 188 devices for covering 100 percent of marketing shares. As per 2014’s Xamarin research, the majority of development teams tests frequently on 25 or fewer devices.
More than one-quarter of these developer communities target five or fewer devices. In real-world situations of testing, automation pays off almost instantaneously and immediately. On the very first testing run, it is witnessed that consumers accelerate testing timelines by 4 times. It is over entire manual testing when running forth against fifty or devices more than that.
Runs that are in subsequence have been a lot faster. Yet shortening is there for a nearly full testing week to just usages of few hours.
b) Coverage is the second myth.
Fragmentation is the cause of no possibility of getting broadened device’s coverage. Together with greater than 19000 devices of unique Androids and permutation of dozens for forming operating systems and factors for iOS, a lot of teams believe that it is not possible for covering majorities of devices in provided markets.
So there is a default testing on the handful of these devices like good enough.
The reality of the second myth:
One could have completed the device’s coverage. In case people maintain devices in-house in handfuls then they are doing a lot. Device procurement is difficult.
Maintaining their money, costs, and time, in turn, making their testers availabilities where and when its need is felt creates logistical logjams. It had been stated by Gartner that mobile developers should find ways for achieving high automation rates for keeping up with the platform’s pace and spread of change. This had been in hosting. Different functions used internal management.
The path to such automation is through cloud services of third parties. The third-party cloud services help in automation of app loading processes, test script execution, results reporting, and re-setting device backs for standards of factories securely. Subsets of tests of apps run in parallel also thereby speeding results.
While testing forth on wide ranges of real devices, Test clouds allow one and all on teams for knowing precisely as to how app functions, in turn, eliminating typical guesswork of mobile developments.
Example: Product Managers set fewer system requirements together with confidences that are justified in the performances of devices. Developers receive visual objective confirmations of fixing bugs before the commitment of newer builds. This is regardless of where and when they operate.
c) Cost is the third myth.
Individuals can afford only manual testing. Automation testing needs the creation of test scripts, learning curves for QA staff, and infrastructure. Lots of teams have been struggling already for hitting deadlines. They are over budget already. So automation testing seems to be at a long-distance away.
The practical scenario for the third myth:
Manual testing saves money only in case people sacrifice coverages. Testing manually seems less costly only in most environments that are bare-bones.
In case of testing includes quick “gut checking” of basic functionalities of fewer devices then manual testing seems like bargains. But any resemblances with test coverage and device that is comprehensive shall make testing manually much more costly than automation testing. This might be quick even.
Manual testing only scales by the addition of more people and masses. Costs are having no true linearity. Scaling up staff for meeting demands brings forth tremendous overheads in forms of coordination and training. Division of test cases thereby reduces efficiencies of all testers by conducting the removal of perspectives.
Additionally, testers who have enough sophistication for digging beyond user behaviors thereby exploring and anticipating reasons as to why applications may fail, might neither be plentiful nor cheap. Automation testing always requires somewhat more overheads at the time of initial set up.
But as stated above, it may produce gains and profits dramatically in test speeds. It also produces corresponding staff reduction within a couple of days. Cloud-based test environments have reduced costs further. This is by the elimination of under-utilized and expensive on-premise testing infrastructure.
d) Consistency is to be the fourth myth.
Good enough needs to be conducted, executed, and operated. For different testing teams, ready deployments are subjective decisions built on perceptions of a lot of varied manual testers. They have a knowledge that meaning is as to that bugs fall through cracks.
Overlapping test coverages must catch the most common and critical issues before releases. The rest of the bugs wait for maintenance releases.
The real scenario of the fourth myth:
Qualities are not qualitative. The readiness of productions must not be factors and matters of opinions. In a pure manual testing environment, perceptions vary from one test to another test and one tester to another tester. This leads to erratic results of tests and in-consistent documentations.
Decisions become complicated when considerations of readiness of products are present. It leads to failures of compliances, masses disenchantment, and lost revenues. Additionally, there is the creation of pockets of tribal uncaptured understandings that are lost when people and employees go out of the door.
Automation, in turn, create quantifiable metrics. This serves as truth’s objective sources for informing decisions regarding the justification of the business decision, the product’s readiness, and the progressing of chart teams.
e) Reluctance is the fifth myth.
Manual testing has been replaced by automation testing. Lots of different developers have entrance into test automation because they expect to replace testers who are manual with machines.
In case the automation of tests repeat similar tests 1000 of times with 100 percent accuracies, then queries arise as to that why there are needs of humans for testing purposes. Automation of test scripts can be done by machines too.
The real-time picture of the fifth myth:
Manual testers become better with automation testing. Machines and humans possess good scenarios at lots of various things and factors. Testers, who do manual testing always can test more creatively.
Automation testing frees them from doing this. Whilst humans look forward to newer ways of breaking apps, automation ensures compliances across the wide range of devices. This is from Unit Tests to full Regression Tests. 2 approaches need not work forth in isolation.
To perform tests being exploratory manual whilst back-ended systems have not been under loads from automated testing. These are excellent ways for discovering errors cropping up in environments of productions. Automation Testing does not replace testers who are humans. This lets them conduct rewarding and interesting work.
Better consistency, coverage, cost, and speed add up to those qualities that are improved. Saving money and time means one can do more testing and not less. This is the case when one reaches milestones being critical. This allows testing to keep pace together with teams of agile developments instead of standing in ways.
So organizations release code much more frequently. This reduces impacts and amounts of defects are given builds. This means that developers work with clean codes. The fixing of bugs has been lesser complicated dramatically. This frees testers through sole acting as gatekeepers focus on creativity. Exploratory testing thereby improves the qualities of products.
Automation Testing of Mobile UIs offers advantages concerning times and qualities. Tools that are automated making it easy for testers for assessing user interfaces of apps through widened ranges of the mobile devices together with making modifications for easy boosting of user experiences.
A Bad GUI is an unpleasant experience for the user. The GUI testing is highly recommended and important especially when it comes to smart devices because here the screen size is comparatively small and a lot of variations of the devices are available in the market.
Your application may look and behave differently on different devices. So, it is important to test the app on at least some standard sizes and variations of devices.
All the mobile applications need UI testing but the deepness of testing required is defined by the category or purpose of the application. You should do a complete analysis of the application’s UI features against the phone’s model or OS versions before you finalize your testbed.
Based on this analysis, you should create your test cases for testing. Use automation wherever possible to save time.
Keep an open eye while testing the UI because it is simple yet it has a big effect on the selling of your app.
Take a look at our upcoming tutorial for detailed information on Mobile Responsive Test.