I recently had this unique experience of coaching a QA (10 years experience) to attend a client software testing interview with a leading Entertainment company in Los Angeles. The site to be tested was a simple customer facing website (sort of like an online TV channel) that had both Web and Mobile components.
The situation was: A consulting company was projecting profiles to this client for an onsite tester + coordinator position but none of them was making it through the testing interview process. So they decided to collect the QA interview questions from the previous attendees and they gave me a questionnaire. They wanted me to give the answers to the next candidate and coach that person to be successful in the testing QA interview.
When I got the list of questions, I was surprised and ‘not-surprised’ at the same time. Surprised- because the questions were really basic and a 10 year experienced QA should have been able to answer them easily. Not so surprised because QA is the field of IT that has most weeds in my opinion – but let’s not get into it.
After being done with the exercise, I thought it would be nice to share this experience with the STH readers. For beginners, this will be a good live exposure. For others, it will be a friendly reminder of how important fundamentals are no matter how experienced we are.
What You Will Learn:
Step 1 – is to get a thorough understanding of the AUT –
a) This could be by reading the requirement documents thoroughly
b) In absence of docs, we could try to understand any point of reference that we have – a previous version of the application or wire-frames or screenshots
Step 2 – After understanding the requirements, we make a list of what are the areas in this application that will have to be tested. In other words, we identify the test requirements. The focus in this step is to identify “What” to test. The outcome of this step is a list of test scenarios.
Step 3 – Once we have the test scenarios, we concentrate next on “How” to test them. This phase involves writing detailed steps about how to test a particular feature, what data to enter (test data) and what is the expected result.
Once these 3 steps are done, we are ready for testing.
Following important fields should be included in a good bug report:
With any application that we test, we are trying to see if a certain set of requirements are met by the application or not. But when it comes to a user-facing site, apart from concentrating on functionality, we also have to look into few the usability features, may be performance and security aspects also to a certain extent.
The first level of testing is – Does the site satisfy its functional requirements. Example: if it is a loan management site, we need to look at – are the new customer able to apply for a loan, are the existing customer able to access their loan info, is the interest % applied to the loan amount correct, etc.
The next level of testing is – how easy is it to use the site, do the options make a logical sense and meet the expectations of the user or not. For example, if the user has to be pass 3-4 screens to submit the basic information they are going to be annoyed, so such issues have to be addressed. Another example, after entering username and password, the user might click on tab- which means the control should go to “Sign in” button, instead if it’s going to cancel, the user is going to be really annoyed and the experience of using the site is going to be compromised. Such issues have to be caught.
Performance testing to the complete extent might not be in scope but simple situations like, how long does the search results take to be displayed and how much time does it take for the system to retrieve a customer info at the peak hour – these are some example of the kind of things we would want to keep an eye on.
Security – for sites where there is a secure login to access the site, the minimum functionality around it has to be tested. For example, if I leave the site idle for more than 10 minutes, is it auto logging out or not. Something as basic as that should be focused on.
IF the detailed standard documentation like BRD and FSD are unavailable, the tester will have to depend on some point of reference.
b) A previous version of the application
c) Wireframes …etc
Another factor that helps immensely, is to talk to the developers or the business analysts (when available) to get a confirmation on our understanding or clarifications in case of doubts.
When none of these situations works, we can just conceptualize the application based on our previous IT application experience and create the basic set of test scripts. When testing phase comes up, we can set up a portion of test cycle time and do some test case management (make the already created scripts perfect) so we have the doc for the next phases.
The key is to make sure that all the testers know about all the modules and that there is no knowledge concentration in one place. Involving everyone in test script peer reviews, defect meetings and KT sessions is going to ensure that everyone is aware of the application to the best extent possible.
Also, by encouraging the concept of teamwork we can have the team members collaborate, help and aid each other for better productivity.
Regular follow up meetings also help the process very much.
The onsite coordinator is a point of contact for the offshore team and to the client for any information regarding the testing engagement.
This job includes:
Yes, even an onsite coordinator has to test.
Every bug has to be noted and analyzed – whether it is encountered at onsite or offshore, whether repeatable or not. A real value-add to a tester’s job is when we involve ourselves in the Root Cause Analysis process for a bug rather than simply reporting it.
Some of the ways we can handle this situation are:
#1. All the onsite and offshore team members should follow a guideline that screenshots had to be taken for every error that we encounter – repeatable or not
#2. If there are logs, system files or anything like that, that might help us find any evidence of the issue- we should try to find it
#3. Despite all these steps, if we still can’t tell why and when the problem occurs- we should report it to the developer all the same – with as much information as we can.
How to test an application having video or audio?
Here are the important points to consider:
– Access levels (restricted or not – password controlled)
– Different kinds of environments
– Browser compatibility
– Screen resolutions
– Internet connection speeds
– The specific options on a video – like play, stop, mute etc.
– Video by size
– Response to the videos – comments (limitations on the comment length and number of comments it can take)
– Video responses to the videos
– Interface with social networking sites – interoperability
– Buffering speed
– Embedding the video
Mobile App Testing Important Test Scenarios:
– Check if the app works well with multiple carriers and multiple devices
– Usability of the features in a mobile screen
– Testing it on different mobile platforms – like Android and iOS
– Installations, uninstalling, launching the app with network and without network, testing functionality
– Network connections –WiFi, 2G, etc.
– Logs at iOS iPhone configuration utility for Android Monitor.bat can be used for debugging
That was it. Now, wasn’t that simple.
As a final note, I reiterate the philosophy at STH – know the basics well, the rest automatically follows.
I conclude, hoping that this effort will be beneficial and meaningful to our readers. Please let us know below in the comments section how we did.
Author: This post is written our STH team member Swati Seela.