Mobile Application Performance Testing Guide

Guide to Performance Testing of Mobile Applications:

In this Explicit range of Performance Testing tutorials, we explored the concept of Cloud Performance Testing in our last tutorial.

Here, you will learn what is a Mobile application, types of mobile applications, performance testing mobile application approach, how to set up a mobile application performance testing environment, how to troubleshoot performance issues with mobile applications, the challenges involved and the tools available etc in a detailed manner.

=> Click Here For Complete Performance Testing Tutorials Series

Overview

In today’s tech world, Mobile applications are widely used than regular desktop applications. The number of users who use smartphones for accessing most of their applications than using a desktop or a laptop is increased drastically.

Also, the end user will be expecting a good response from the application when he is using it. Hence, the developer who builds the mobile applications needs to keep this in mind while developing the application.

Now Performance Testing is critical either it is a regular web application or an application on a mobile device. If an application goes live without being performance tested and if the end user faces performance issues like a slow response, etc., then he will try to find another better application which will perform in a better way.

Performance testing is expensive and it takes more time as well. Hence many clients avoid going for performance testing, but this will help a lot with the issues in production. Thus, any application has to be tested thoroughly before it is made available to the end user.

What is a Mobile Application?

Mobile applications are a set of predefined functions that are designed to work on mobile devices such as Smart Phone, Tablets and other feature phones. Not all mobile applications are the same, there are three different types of mobile applications and these are based on the technology that has been used to develop those applications.

3 types of Mobile Application include:

• Native Mobile Applications
• Web-Based Mobile Applications
• Hybrid Applications

#1) Native Mobile Applications:

Native Mobile Applications are those applications which run on your own Smartphones or Tablets, they run on the operating system of the mobile devices. They provide high performance and are very reliable too. These type of applications also access the other applications on your smartphone device such as photos, address book etc.

Native applications are available in the mobile application store. For Android, we have play store and for iPhone we have iTunes. So these applications are downloaded from their respective stores and installed on mobile devices.

Examples of Native applications are a calculator, calendar, Facebook application etc.

#2) Web-Based Mobile Applications:

Web-Based Mobile Applications run directly from the browser, they are not installed on the mobile device.

Examples of Web-Based Mobile Applications are online games, social networking sites etc.

#3) Hybrid Applications:

Hybrid Applications are similar to Native applications and they run inside the native container, they are a mix of both the applications and they use the device browser and not the actual browser to render the application web pages.

Examples of Hybrid application are Instagram, Yelp etc.

The Architecture of Mobile Applications

In the above figure, smartphones and tablets come under the client-side. They include the devices that we usually use like the Smart Phone, Tablets, their network like 3G, 4G, etc. An operating system of the devices like Android, ios, Windows, etc.

Mobile Cell Network, Wireless, Internet, and Firewall comes under Network. For the Cell network, we see signal strength, Network Congestion, dropouts, etc. For a wireless network, we see signal strength, speed, etc.

Web, App, and DB come on the server-side where a majority of the work is done.

Mobile Application Performance Testing Approach

Mobile application performance testing is different from web application performance testing. The laptops or desktops that are used to access the web applications do not suffer much from poor network conditions including packet loss or latency.

Whereas in Mobile applications network conditions, packet loss, latency, the type of device that we are using like a smartphone or smart device and bandwidth all matters and we shouldn’t ignore all these factors while considering performance testing of mobile applications.

Before you develop your performance strategy you should understand what type of application you are running on your mobile device.

There are three types of applications:

• Browser Based applications
• Native applications
• Hybrid applications

#1) Browser-Based Applications:

These type of applications can be directly accessed through a mobile browser and they don’t need any additional software to be downloaded or installed on the device.

They are lightweight and are built on the same stack in all types of devices such as Android, iPhone etc. Also, the responsive web design movement makes it, even more, easier to create a multi browser-based application as you can render the application to different screen sizes.

These type of applications have drawbacks like they can’t be accessed without an internet connection, they don’t have all the features built into the device and they also tend to appear slower to the users.

So. while performance testing this type of applications it’s important to replicate the user load from a mobile browser and it is also important to test the web page rendering on target devices.

#2) Native Applications:

These type of applications are built on a particular platform using the software which is installed directly on the device.

Native applications are faster and more responsive as they are locally built on the device. As the code on each device is different i.e for iPhone, Android etc, while performance testing we need to consider testing on each platform.

#3) Hybrid Applications:

Hybrid applications are the combination of both Browser and Native applications. They are composed of a Native shell which provides a fast integrated experience.

While performance testing targets the load generated by the users of such hybrid application on the server side, as well as gauges the on-device application performance from an end-user perspective.

So how do we test these mobile applications?

In order to test such mobile applications, we need to build an environment with emulators and real devices. Mobile emulators are software’s which needs to be installed on our laptops or desktops that will imitate a mobile application or a mobile platform.

Hence, using these emulators we can generate a significant amount of load for our load testing.

The other method is using real devices, here we have the device physically and we can see how the application is performing on each device. Using real devices, we don’t have the option to generate the load which is our main goal but to do that we need to have multiple real devices.

However, if you want to test the behavior of the application on a specific device you can use the real device, but to generate load we need to use emulators.

The next point to note while testing a mobile application is to know the end user’s access location. It is very important to know if the users are accessing from the same city, same country or if are they spread across the world etc. If we plan our test or environment without considering this then our tests would be wrong.

Application Performance on a Mobile Application like a Smart Phone or Smart Device is usually measured in terms of Device Performance, Server/API Performance, and Network Performance.

We need to check the following under each performance.

Device Performance:

• Application startup time
• Battery Time while using the application
• Memory consumption
• Hardware and Software variation
• Usage with other Applications
• An application running in the background

Server/API Performance:

• Data to and from the server.
• API calls generated
• Server downtime

Network Performance:

• Packet Loss
• Network Speed

Thus, while writing the approach for Mobile Application performance testing you need to consider all the above points and match your project scope and requirements.

Mobile Application Performance Strategy

Performance Test Approach differs for each type of mobile application. The approach varies from one application to the other as each application is developed with different components and the processing mechanism is different for each.

Browser Based Mobile Applications act differently than the regular web browser application. This behavior is due to the bandwidth from the data plan that the end user is using and it also depends on the configuration on the mobile devices.

The cost of developing this type of browser-based application will be less as the development work would be to see its compatibility with the browsers that are available on mobile devices.

Hence for performance testing such as mobile browser-based applications, we need to replicate the user load on the browser. We can do this by using emulators or any other additional components. Apart from the user load on the mobile browsers we also need to test the web page rendering on the devices.

The development cost of Native Mobile Applications is more. The applications are installed directly on the device which makes the user experience better, but the application is platform dependent.

So if we want to test performance in these type of applications then we need to test all the platforms of the mobile devices like Android, ios, windows etc and also on all types of devices like smartphone, tablets etc.

Hybrid Mobile Applications are a combination of both Browser Based Mobile Applications and Native Mobile Applications, hence here we need to test the user load on the server and also we need to do full a stretch performance testing on all the mobile devices and platforms.

For any application on the mobile device, there are two main factors which affect the performance of the end user:

• Request on the mobile device
• The page rendering at the mobile device

#1) Request on the Mobile Device:

Under this, the request is delivered to the server, and the server response is generated based on the request received. Once the response is generated at the server, this response is sent back to the mobile device which has sent the initial request.

Here the parameters which need to be looked into are the server load, the different network types like 3G, 4G etc., a load on each network and any other components.

We can test these by using emulators i.e., we can generate the scripts and use emulators to mimic the user load on the server.

#2) Page Rendering at the Mobile Device:

Page rendering at the mobile device is nothing but the response that is received from the server for the request we sent from our mobile application.

Here the parameters which need to be looked into are the operating system of the device, the configuration setting of the mobile device, etc. We can test these by actually logging into the mobile device and monitoring them.

Now, let us take a detailed look at each step mentioned in the above figure:

a) Create Real-Time User Scenarios:

Similar to the regular web browser applications, we need to identify the scenarios or the conditions which are expected on the server in real time. There will be many scenarios to be tested.

Let us assume a condition where there is a web application which is available on both desktops as well as mobile device, also it is native in Android and ios devices. Here we need to develop a test script based on the application access type. Hence, we need to test it separately on each device.

Also, we need to consider the workload pattern for that scenario i.e., we need to understand how the load is distributed to each device. For Example, 500 users will be accessing the application through web, around 100 will be using a native application on their ios device and another 200 will be accessing the application from their native device on Android application.

We should also consider the geographic locations of the load on the server. For Example, 30% of the users are accessing from the USA, 50% from India, and rest 20% from other locations. It is mandatory to mimic different user location as it will have a very huge impact on the servers.

Testing all 100% from a single location will yield wrong results. So it is advised to test the load on the servers by giving a different location. The cloud solution can be used for mimicking different geographic locations.

b) Create Real-Time Network Conditions:

Here, we need to create different network conditions and also the quality of the network like 3G network, 2G network, Wi-Fi with the best speed, average speed, and low speed. We need to do network emulation for creating network conditions.

We should also test the performance by giving 50% bandwidth conditions, we can achieve this by doing network emulation. We can use any tool like Wind Tunnel, Shunra, etc.

Also, test the network with respect to different geographies like 3G network in Bangalore. 4G in Hyderabad etc. Here also we need to do network emulation by using appropriate tools.

c) Create Real-Time Device Conditions:

This is nothing but the type of application on the mobile device like whether it is a Thin client application or a Thick client application, what is the operating system on the mobile device, the device type like iPhone, Nokia etc., and any other application which will be running parallel during the use of this particular application etc., also has to be tested as part of the scenarios.

d) Test the performance of each component:

After creating all the above scenarios, we need to performance test the application including all the software and hardware. Now collect all the necessary metrics during the load test.

Some of the server side metrics which need to be collected are CPU Usage, CPU Load, Processing Time, Total Byte available, Packets sent, packets received etc.

Some of the Network metrics which need to be collected are Packets and bytes sent over the network, packets, and byte received over the network, Packets dropped and total average delay.

Device metrics to be collected are CPU, Memory Utilization, Response time etc and also measure all the transaction response times and throughput.

Mobile Application Performance Testing Environment

Before we set up the performance testing environment for a mobile application, we need to understand what are the mobile applications that will be tested, what all operating systems or devices will be tested and the different geographic locations as well.

Hence, first, we need to create an environment which consists of Emulators and real devices. So what is an Emulator? An emulator is a software which can be installed on our laptops or desktop machines, and these act as mobile devices. We can generate huge load using these Emulators, hence they are best to use for load testing any mobile applications.

On the other hand, Real Devices are physical mobile devices. So we need to verify the application behavior on each device and on each network. But we will not be able to test the bandwidth, packet loss etc., when multiple devices are connected to the same network and are accessing the same application.

The next thing to consider is the geographic location of the application devices, this is a mandatory and very important aspect to be considered or else our test results will not be as what happens in the real world scenario.

So, once you analyze all the above select the respective load testing tool to conduct the performance tests.

Mobile Application Performance Testing Challenges

The challenges that we face while Performance Testing Mobile Applications include:

• When we need to test our application on different mobile devices, we need to organize all the devices like IOS, Android, Windows etc.
• Network simulation is another challenge we will face i.e we need to simulate networks like 3G, 2G, 4G and wifi networks.
• The memory available in the mobile devices to handle the test runs.
• The battery life of the Mobile devices.

Troubleshooting Performance Issues with Mobile Applications

After the test run is complete, its time to gather the performance bottleneck identified during the test run. Once the issues are consolidated, we need to identify the root cause of the issue and fix them.

Let’s have a look at some performance issues with mobile applications:

• A delay in the response from the mobile application: This delay is usually caused by the memory or by caching. In such cases, it is advised to clear the cache and rerun the test.
• Application some time gets restarted, hanged or gets froze: In such cases check the software as if any updates need to be done, check the application code for any optimization, handling the caching are some areas to be looked into.

Mobile Application Performance Testing Tools

The tools may vary based on the type of the device and platform.

Some of the most commonly used tools are given below.

For Android devices:

Robotium: Using this tool, we can record and replay the several steps for conducting the test. Click here for a Robotium tutorial.

Monkey Runner: Using this tool we can run our tests on the devices by connecting them to a desktop. Using the Interface of the tool we can access the smartphone.

For iOS devices:

Automator: Using this tool we can drag and drop the workflows and automate them.

Conclusion

I’m sure you would have learned more about Mobile Applications and how to go ahead and perform Load testing on the mobile applications in a perfect way through this informative tutorial.

Our upcoming tutorial will brief you on How to Perform Manual Performance Testing in simple terms.

=> Visit Here For Complete Performance Testing Tutorials Series

PREV Tutorial | NEXT Tutorial