Mobile Testing is really challenging domain. Let's see why.
While searching for educational apps/games for my toddler, I found a game where a kid can do different saloon operations on vehicles like washing, cleaning, painting, tattooing (vehicles too can have tattoos!) and the way it was presented, I loved it.
I immediately downloaded the paid version and gave it to my toddler with a hope that I will be allowed to work for next 15-20 minutes while he will be busy decorating vehicles.
He excitedly accepted and started exploring (because kids never believe in scripted testing :) ). Within 5 minutes, he came back complaining – this is not working, please start it again…..
I saw a pop-up saying “Unfortunately the “XYZ” has stopped working”. I pressed OK and started it again and gave it to my son. Within two minutes again same complain and same warning. It was becoming interesting. Carrying a tester’s mind always, I started the investigation.
I asked my toddler what he did. (How did you find the issue?)
Nothing, I was just playing – he said.
I rephrased my question – how did you play the game, can you show me? (Can you reproduce it?)
He started playing and I watched. It was working fine and I grew impatient. I was about to turn to my work and I heard the same complain. He rejected the game saying he did not like it because it stopped for so many times.
Now the tester in me awoke and I settled myself in a chair and started playing the game. It worked fine for 10 minutes and I did not see any crash pop-up. I checked multiple tapings, data connection on/off, GPS on/off, portrait/landscape mode, power on/off but really did not find anything. Tired and frustrated, I asked my son to play it again as I wanted to see how the game stopped for multiple times for him.
One more playing round started. He selected a car from a range of vehicles to play with. He applied the little soap and then sprinkled water via designer fountain, dried the car and moved to next section where he can decorate it. He painted a car with green colour. But then he thought he did not want the green colour. Rather than changing colour, he went back to the cleaning section and sprinkled water on the car in a hope that car would gain its original colour. At this point, the pop-up blinked on the screen “Unfortunately the XYZ has stopped working”. I almost jumped out of my chair. After following same steps again, I too could see the crash.
What was happening?
After some combinations of trial and error, I found –
When the user was coming back from decorating to cleaning section, the application was crashing.
OMG, what a silly bug, I told to myself. While doing web application testing for more than eight years, I have been executing some test cases by default. One of it is – go forth and back and then again forth and do some activity. Didn’t the mobile game developer take care of this silly thing?
Ok, the developer forgot about it. Didn’t the tester find this silly mistake which made the kid to reject the game?
Why did I describe the case?
In the era where mobile is dominating all other domains and almost every fortnight a new mobile phone is launched, mobile testers are under a huge pressure. Most of the interview questions for mobile testing does not include any question about basic testing or even functional testing.
Mobile Testing, is it really tough?
The perceptions about mobile testing are – The tester should know about how to do memory leakage testing, security testing, interruption testing, device-specific testing, OS-specific testing and should be knowing one or two tools to check performance. Well enough?
Agreed that mobility is really a challenging domain. Reasons of it are –
- The leaps-and-bounds growth of the mobile industry has pushed every industry to launch their app for mobile. Today, if you look for, you will be able to find apps for almost every known keyword.
- Mobile has become so much part of life that we reject or decline a site or application which is not mobile. We want everything at our fingertip. And these huge demands have created tons of stress for every project to be mobile compatible.
- A large number of smartphones in the market along with the addition of the same on monthly basis makes testing coverage a critical factor and pain point for everyone involved.
- With time, more and more OSes are being built up independently, in addition to traditional OSes like iOS, WebOS, Blackberry and Android. And most of the apps developed today are and expected to be cross-platform and therefore thorough testing on different supported OSes is required.
- Mobile testing automation is still in its childhood and even though there are numbers of tools available, mobile testing automation testers are not easily available.
- Mobile testing comes up with other different challenges like networking issues – poor, good network, network changes like Wi-Fi to 3G/4G and vice-a-versa, performance while multiple apps are being used simultaneously, memory leakage issues, battery consumption issues etc.
When mobile comes to picture, the most Important feature is user-friendliness. No matter, how good your application is, display-wise or feature wise, if the user is not able to handle it on that little screen for more than 5 seconds, he will never come back. Mobile users are spoiled with a number of options and therefore users never bear any bug.
You might think at this point – Web testing is also the same. User friendliness is required over there too. Correct. But computer users are patient than mobile users. Even as a tester, you would see that different behaviour in yourself – when you test any website on desktop, you can wait for 2-5 seconds whereas, on mobile, you keep tapping the loading symbol to kill those 2 seconds.
So, mobile testing is not tough because testers have to think about vast coverage and real-time usage (yes, they are surely the differentiating factors from any typical testing) but mobile testing has become tough because while running a race with allowed time to do testing, most of the mobile testers prefer to do interruption, memory leak, performance testing but avoids/gives less importance to basic functional testing (This statement is a general statement and should not be taken as a commenting weapon :) )
If you have worked in different domains, you must know that no domain is easy and every domain has its specific tough factors. Mobile domain too is not different. Yes, it’s different and it has changed the equation of requirements and users’ expectation in many ways. But it’s tough because testers think testing coverage for a number of devices is more important than thorough functional testing on at least one device.
Again, whatever I have mentioned above, are my personal thoughts and they are not written for any specific person/industry.
About the author: This post is written by STH team member Bhumika Mehta. She is a project lead, carrying 10+ years of software testing experience. She is totally into testing and loves to test everything exists.
In our next article, we will discuss more on Android Version.