Introduction to Mobile Application Testing:
Gone are the days when the telephone used to be an appliance that sat in a corner and had to ring to get our attention or a computer was a machine only few people used – they are now an extension of our being- a window to the world and virtual servants that do as they are told. Computers were a rage and changed how we humans thought, behaved, learnt and existed.
Mobile technology and smart devices are the trend now and will change the future of the world as we know it. We all can vouch for it, can’t we? Now, it will be amateurish if I list what we use these mobile devices for. You all know it – May be better than we do. J
Let’s get straight to what this tutorial is going to be about.
This tutorial will be both an introduction and your guide to Mobile Testing. So, read through!
Types of Mobile Testing
There are broadly 2 kinds of testing that take place on mobile devices:
#1. Hardware testing:
The device including the internal processors, internal hardware, screen sizes, resolution, space or memory, camera, radio, Bluetooth, WIFI etc. This is sometimes referred to as, simple “Mobile Testing”.
#2. Software or Application testing:
The applications that work on mobile devices and their functionality is tested. It is called the “Mobile Application Testing” to differentiate it from the earlier method. Even in the mobile applications, there are few basic differences that are important to understand:
a) Native apps: A native application is created for use on a platform like mobile and tablets.
b) Mobile web apps are server-side apps to access website/s on mobile using different browsers like chrome, Firefox by connecting to a mobile network or wireless network like WIFI.
c) Hybrid apps are combinations of native app and web app. They run on devices or offline and are written using web technologies like HTML5 and CSS.
There are few basic differences that set these apart:
- Native apps have single platform affinity while mobile web apps have cross platform affinity.
- Native apps are written in platforms like SDKs while Mobile web apps are written with web technologies like html, css, asp.net, java, php.
- For a native app, installation is required but for mobile web apps, no installation is required.
- Native app can be updated from play store or app store while mobile web apps are centralized updates.
- Many native app don’t require Internet connection but for mobile web apps it’s a must.
- Native app works faster when compared to mobile web apps.
- Native apps are installed from app stores like Google play store or app store where mobile web are websites and are only accessible through Internet.
The rest of the article is going to be about Mobile Application Testing.
Significance of Mobile Application Testing
Testing applications on mobile devices is more challenging than testing web apps on desktop due to
- Different range of mobile devices with different screen sizes and hardware configurations like hard keypad, virtual keypad (touch screen) and trackball etc.
- Wide varieties of mobile devices like HTC, Samsung, Apple and Nokia.
- Different mobile operating systems like Android, Symbian, Windows, Blackberry and IOS.
- Different versions of operation system like iOS 5.x, iOS 6.x, BB5.x, BB6.x etc.
- Different mobile network operators like GSM and CDMA.
- Frequent updates – (like android- 4.2, 4.3, 4.4, iOS-5.x, 6.x) – with each update a new testing cycle is recommended to make sure no application functionality is impacted.
As with any application, Mobile application testing is also very important, as clientele is usually in millions for a certain product – and a product with bugs is never appreciated. It often results in monetary losses, legal issue and irreparable brand image damage.
Basic Difference Between Mobile and Desktop Application Testing:
Few obvious aspects that sets mobile app testing apart from the desktop testing
- On desktop, the application is tested on a central processing unit. On a mobile device, the application is tested on handsets like Samsung, Nokia, Apple and HTC.
- Mobile device screen size is smaller than desktop.
- Mobile devices have less memory than desktop.
- Mobiles use network connections like 2G, 3G, 4G or WIFI where desktop use broadband or dial up connections.
- The automation tool used for desktop application testing might not work on mobile applications.
Types of Mobile App Testing:
To address all the above technical aspects, the following types of testing are performed on Mobile applications.
- Usability testing– To make sure that the mobile app is easy to use and provides a satisfactory user experience to the customers
- Compatibility testing– Testing of the application in different mobiles devices, browsers, screen sizes and OS versions according to the requirements.
- Interface testing– Testing of menu options, buttons, bookmarks, history, settings, and navigation flow of the application.
- Services testing– Testing the services of the application online and offline.
- Low level resource testing: Testing of memory usage, auto deletion of temporary files, local database growing issues known as low level resource testing.
- Performance testing– Testing the performance of the application by changing the connection from 2G, 3G to WIFI, sharing the documents, battery consumption, etc.
- Operational testing– Testing of backups and recovery plan if battery goes down, or data loss while upgrading the application from store.
- Installation tests– Validation of the application by installing /uninstalling it on the devices.
- Security Testing– Testing an application to validate if the information system protects data or not.
Mobile Application Testing Strategy
The Test strategy should make sure that all the quality and performance guidelines are met. A few pointers in this area:
1) Selection of the devices – Analyze the market and choose the devices that are widely used. (This decision mostly relies on the clients. The client or the app builders consider the popularity factor of a certain devices as well as the marketing needs for the application to decide what handsets to use for testing.)
2) Emulators – The use of these is extremely useful in the initial stages of development, as they allow quick and efficient checking of the app. Emulator is a system that runs software from one environment to another environment without changing the software itself. It duplicates the features and work on real system.
Types of Mobile Emulators
- Device Emulator- provided by device manufacturers
- Browser Emulator- simulates mobile browser environments.
- Operating systems Emulator- Apple provides emulators for iPhones, Microsoft for Windows phones and Google Android phones
List of few free and easy to use mobile device emulators
i. iPhone Tester – All you need to do with this is – enter the URL in search box and you can see the real time preview of how it appears on an iPhone.
ii. Mobile Phone Emulator – Used to test handsets like iPhone, blackberry, HTC, Samsung etc.
iii. MobiReady – With this, not only can we test the web app, we can also check the code.
iv. Responsivepx – It checks the responses of the web pages, appearances and functionality of the websites.
v. Screenfly – It is a customizable tool and used to test websites under different categories.
3) After a satisfactory level of development is complete for the mobile app, you could move to test on the physical devices for a more real life scenarios based testing.
4) Consider cloud computing based testing: Cloud computing is basically running devices on multiple systems or networks via Internet where applications can be tested, updated and managed. For testing purposes, it creates the web based mobile environment on a simulator to access the mobile app.
- Backup and recovery- Cloud computing automatically takes back up of your data from remote location making recovery and restoring of data easy. And also, the storage capacity is unlimited.
- Clouds can be accessed from different devices and anywhere.
- Cloud computing is cost efficient, easy to use, maintain and update.
- Fast and quick deployment.
- Web based interface.
- Can run the same script on several devices in parallel.
- Less control– Since the application runs on remote or third party environment, user has limited control and access over the functions.
- Internet connectivity issues– the setup is on Internet. Network issues affect the availability and functioning
- Security and privacy Issues– Cloud computing is an Internet computing and nothing on Internet is completing secure, so chances of data hacking are more.
- If the application contains new functionality, test it manually.
- If the application requires testing once or twice, do it manually.
- Automate the scripts for regression test cases. If regression tests are repeated, automated testing is perfect for that.
- Automate the scripts for complex scenarios which are time consuming if executed manually.
Two kinds of automation tools are available to test mobile apps:
Object based mobile testing tools– automation by mapping elements on the device screen into objects. This approach is independent of screen size and mainly used for Android devices.
- Eg:- ranorex, jamo solution
Image based mobile testing tools– create automation scripts based on screen coordinates of elements.
- Eg:- Sikuli, Egg Plant, RoutineBot
6) Network configuration is also necessary part of mobile testing. It’s important to validate the application on different networks like 2G, 3G, 4G or WIFI.
Test Cases for Testing a Mobile App
In addition to functionality based test cases, Mobile application testing requires special test cases which should cover following scenarios.
- Battery usage– It’s important to keep a track of battery consumption while running application on the mobile devices.
- Speed of the application- the response time on different devices, with different memory parameters, with different network types etc.
- Data requirements – For installation as well as to verify if the user with limited data plan will able to download it.
- Memory requirement– again, to download, install and run
- Functionality of the application– make sure application is not crashing due to network failure or anything else.
Download Some Sample Test Cases for Testing Mobile Applications:
Typical activities and proceedings in Testing Mobile Application
The scope of the testing depends on the amount of requirements to be checked or the extent of changes made to the app. If the changes are few, a round of sanity testing will do. In case of major and/or complex changes, a full regression is recommended.
An example application testing project: ILL (International Learn Lab) is an application designed to help admin, publisher to create websites in collaboration. Using a web browser, instructors choose from a set of features to create a class that meets their requirements.
Mobile Testing process:
Step #1. Identify the types of testing: As ILL application is applicable for browsers, so it’s mandatory to test this application on all supported browsers using different mobile devices. We need to do usability, functional and compatibility testing on different browsers with the combinations of manual and automation test cases.
Step #2. Manual and Automated testing: The methodology followed for this project is Agile with the iteration of two weeks. Every two weeks dev. team releases a new build to testing team and testing team will run their test cases on QA environment. Automation team creates scripts for set of basic functionality and runs the scripts that help determine if the new build is stable enough to test. The Manual testing team will test the new functionality.
JIRA is used for writing of acceptance criteria; maintaining of test cases and logging /re-verification of defects. Once the iteration gets over, iteration planning meeting held where dev. Team, product owner, business analyst, and QA team discuss what went well and what needs to improve.
Step #3. Beta Testing: Once the regression testing is completed by the QA team, the build moves into UAT. User Acceptance Testing is done by the client. They re-verify all the bugs to make sure every bug was fixed and the application is working as expected on every approved browser.
Step #4. Performance test: Performance testing team tests the performance of the web app using JMeter scripts and with different the loads on the application.
Step #5. Browser testing: The web app gets tested across multiple browsers- both using different simulation tools as well as physically using real mobile devices.
Step #6. Launch plan: After every 4th week the testing moves into staging, where a final round of end to end testing on these devices is performed to make sure the product is ready for production. And then, it goes Live!
Designing the right test strategy, choosing the right mobile simulators, devices and mobile testing tools can make sure that we have 100% test coverage and help us include security, usability, performance, functionality and compatibility based tests into our test suites.
Well, this has been our effort to fulfill multiple requests from our readers on a mobile application testing guide.
About Author: This is a guest post by Nancy Ratnakar. She is a Senior QA Engineer, in a MNC with more than 5 years of experience in QA and mobile testing. With extensive experience in STLC, she is an expert handling projects from initial requirement stage to the final release.
Please let us know how we did in the comments. Also share your experiences if you are working or have worked on this kind of mobile testing. Your questions and suggestions are most welcome!