Everything about Cross browser testing – what, why, how, who, when, and where.
Many times, I have encountered an issue with a website and on calling the technical support, they simply tell me to try it in another browser? When I do, it works and I end up feeling like a total idiot, even though I earn my living working in the software industry.
I bet this has happened to all of you, hasn’t it?
I always end up thinking-‘why didn’t I think of that?’ But trust me, over time I have realized it’s not my fault; it’s just that the website has not been tested extensively with respect to cross browser compatibility testing and as an end user I have just found a bug.
That being the background story, I bet you all have figured out the topic for today’s discussion- Cross Browser Testing.
As is a general practice at STH, we are going to focus on the basics. We believe that any concept will make a world of sense when we ask the basic question words around like- “What, why, how, who, when, where”.
Let us do just that as we go.
What is Cross-Browser Testing?
#1) Cross browser testing is simply what its name means- that is, to test your website or application in multiple browsers- and making sure that it works consistently and as in intended without any dependencies, or compromise in Quality.
#3) What kinds of applications undergo this? – Customer facing applications are the best choice. You might wonder at this point, “Aren’t all applications customer facing?” Well, yes. They are. However, let us look at an example.
Application 1: An application developed for a company to internally keep track of its inventory
Application 2: This is for the end users to buy products from this company
- It is apparent that the best idea would be to test Application 2 for browser compatibility testing since it is impossible to control what browsers/platforms/versions the end user is going to use.
- On the other hand, if all computers internal to the company use Windows 8 machines with Chrome browser- then there is no need to look or test for anything else with respect to Application 1.
Why is Cross Browser Testing performed?
For that matter, why is any kind of testing done?
- To know what is wrong and be able to fix it.
- To enhance efficiency and user experience and thereby, business.
- To be informed of any possible pitfalls
But specifically, if we think: What is the intent of cross browser testing? – This is twofold.
- The rendition or appearance of the page in different browsers- is it the same, is it different, if one is better than the other, etc.
- The functionality and the working of it. (Of course!)
Who performs this type of testing and who are these results relevant to?
- Are you thinking, “There are a million browsers, versions and platforms out there- which ones to choose?” – This, thankfully, is not a decision that is the tester’s responsibility. The client, business analysis team and the marketing teams have a major role in this decision. Also, companies collect usage/traffic statistics to narrow down what browsers, environment and devices are mostly in use.
- The entire project team should have an invested interest, time, money and infrastructure to support this endeavor.
- The QA team can be involved in this process or it might be the design team who are keen on knowing how the application fares in multiple browsers.
- Whether it is performed by QA or any other team- the results are interpreted by the design and development teams and the relevant changes are made.
How is Browser Compatibility Testing performed?
Now we are talking!
First things first- is it done manually or using a tool?
It can surely be done manually- multiple machines, multiple OSs, Multiple browsers, multiple machines and but clearly, this leads to multiple problems, multiple investments and multiple challenges.
So, lots of tools are available in the market to make this easier.
The tools help us with one or more or all of the following depending on the tool itself and the licensing types:
- They provide a VPN (Virtual Private machine) using which you can connect to remote machines and check the working and rendition of your JAVA, AJAX, HTML, Flash and other pages. Most of these are secure, but since you are submitting your information to a third party, a certain analysis on discretion is advised.
- Screenshots are provided for the pages and links submitted of how they appear in multiple browsers. This is, of course, static.
- Multiple browsers are synchronized with respect to operations performed on one and the results are presented browser wise.
- Show the rendition of a page at multiple screen resolutions
- When a problem is encountered, a video or screenshots are recorded to transport the problem for further analysis.
- Support generally is available for both web and mobile apps
- Private pages that require authentication to be accessed can also be tested
- Local, with in a private network/firewall pages can be tested too
To summarize “how” to cross browser test:
#1. Traffic statistics help determine what browsers to test.
#2. A detailed analysis should be done on the AUT (Application under test) itself to determine what parts of the application or if all of it has to undergo this. It is advisable that all of it be tested on multiple browsers, but again costs and time have to be considered. A good strategy is to perform 100% testing on one browser per platform and for the other just test the most critical/widely used functionality.
#3. Once the decision of “What” to test and “Where (browsers)” is made- infrastructure decisions are to be made- do we acquire tools or perform this manually etc. Again, cost has to be considered. Viability, risks, security concerns, people to be involved, time, acceptance criteria, issue/defect fixing schedules/process – are few things that have to be addressed.
#4. Perform the testing. The regular functional testing test cases can be used when validating the efficiency of the system. For look-and-feel/rendition test cases are not necessary.
The operation I was talking about in the beginning of this article that failed for me was an online bank transfer. I logged into my bank account, chose the amount for transfer as about one lakh and tried to perform the transfer and a servlet error was showing up no matter how many times I tried. So if the transfer operation is chosen for cross browser testing, this is how the test script is going to looks like.
- Login to the online bank account
- Select the account from which the transfer is to be done
- Enter the transfer amount: 100,000
- Select payee and click “Transfer”
- Expected result: The transfer should be successful
- This will simply be run on all the browsers chosen.
Again, please note that this does not look different to a functional test case. Please check this non functional testing article for further information on this.
#5. Report the results back to the design team, if they were not involved in the testing process. Change follows.
When is the best time to do this?
Any testing reaps the best benefits when it is done early on. Therefore, the industry recommendation is to start with it as soon as the page designs are available.
But it also can be performed when the site is fully integrated and functional.
If you have missed the bus on performing cross browser testing during design, development and QA phases, it can still be done while the application is in production. However, this is the costliest of all and risky too.
Where is browser compatibility testing performed?
Usually, the answer to this question would be one of- Dev/QA/Production environments. But for cross browser testing this, is not a definite and irrelevant (if I may say so). It can be done in any one or all of them.
A few points to note, in conclusion:
- Having been a QA teacher for a while now, I can tell what’s coming next and that is –the question, is it functional and non-functional testing? I think it is neither and both.
- It also should not be confused with Cross Platform testing, which is testing your application in multiple target environments like Windows, Linux, Mac etc. Although sometimes the two have to integrate together as some of the older browser versions might be compatible only with the older versions of the platforms.
- It is also a continues process as software environments, browsers and devices are evolving every day and to make sure there are no unpleasant surprises, Cross Browser Testing should be added to the repertoire of regression suites.
As always, this has oriented you towards cross browser testing as much as it did us. This is yet another testament to the concept that QA field or software testing is a multi dimensional field and there is something for everyone to excel in.
Author: This cross browser testing guide is written by STH team member Swati S.
Please post your comments and questions below. We are always thrilled to hear from you!