Have you performed Software Installation Testing? How was the experience? Well, Installation Testing (Implementation Testing) is quite an interesting part of the Software Testing Life Cycle.
Installation Testing is like introducing a guest in your home. The new guest should be properly introduced to all the family members in order to feel comfortable. Installation of new software is also quite like the above example.
If your installation is successful on the new system then a customer will be definitely happy but what if things are completely opposite. If an installation fails then our program will not work on that system not only this but can leave a user’s system badly damaged. A user might require reinstalling the full operating system.
In the above case will you make an impression on a user? Definitely not! Your first impression to make a loyal customer is ruined due to incomplete installation testing.
What do you need to do for a good first impression? Test the installer appropriately with a combination of both manual and automated processes on different machines with a different configuration. Major concerned of installation testing is Time! It requires a lot of time to even execute a single test case.
If you are going to test a big application installer then think about the time required to perform such many test cases on different configurations.
We will see different methods to perform manual installer testing and some basic guideline for automating the installation process.
To start Installation Testing first decide on how many different system configurations you want to test the installation.
Prepare one basic hard disk drive. Format this HDD with the most common or default file system, install the most common operating system (Windows) on this HDD. Install some basic required components on this HDD.
Each time creates images of this base HDD and you can create other configurations on this base drive. Make one set of each configuration like Operating system and file format to be used for further testing.
How can we use Automation in this process? Well, make some systems dedicated to creating basic images (use software’s like Norton Ghost for creating exact images of the operating system quickly) of the base configuration. This will save your tremendous time in each test case.
For example, if the time to install one OS with basic configuration is said 1 hour then for each test case on the fresh OS you will require 1+ hour. But creating an image of OS will hardly require 5 to 10 minutes and you will save approximately 40 to 50 minutes!
You can use one operating system with multiple attempts of installation of the installer. Each time uninstalling the application and preparing the base state for the next test case. Be careful here that your uninstallation program should be tested before and should be working fine.
Installation Testing Tips with some broad Test Cases:
#1) Use flow diagrams to perform installation testing. Flow diagrams simplify our task. See example flow diagram for the basic installation testing test case.
Add some more test cases on this basic flowchart Such as if our application is not the first release then try to add different logical installation paths.
#2) If you have previously installed a compact basic version of application then in next test case Install the full application version on the same path as used for the compact version.
#3) If you are using flow diagram to test different files to be written on disk while installation then uses the same flow diagram in reverse order to test uninstallation of all the installed files on disk.
#4) Use flow diagrams to automate the testing efforts. It will be very easy to convert diagrams into automated scripts.
#5) Test the installer scripts used for checking the required disk space. If the installer is prompting required disk space 1MB, then make sure exactly 1MB is used or whether more disk space utilized during installation. If yes flag this as an error.
#6) Test disk space requirement on different file system format. Like FAT16 will require more space than efficient NTFS or FAT32 file systems.
#7) If possible set a dedicated system for only creating disk images. As said above this will save your testing time.
#8 ) Use a distributed testing environment in order to carry out installation testing. The distributed environment simply save your time and you can effectively manage all the different test cases from a single machine.
The good approach for this is to create a master machine, which will drive different slave machines on the network. You can start installation simultaneously on a different machine from the master system.
#9) Try to automate the routine to test the number of files to be written to the disk. You can maintain this file list to be written on the disk in an excel sheet and can give this list as an input to an automated script that will check each and every path to verify the correct installation.
#10) Use software’s available freely in the market to verify registry changes on successful installation. Verify the registry changes with your expected change list after installation.
#11) Forcefully break the installation process in between. See the behavior of the system and whether the system recovers to its original state without any issues. You can test this “break of installation” on every installation step.
#12) Disk space checking: This is the crucial checking in the installation-testing scenario. You can choose different manual and automated methods to do this checking.
In manual methods, you can check free disk space available on the drive before installation and disk space reported by installer script to check whether the installer is calculating and reporting disk space accurately. Check the disk space after the installation to verify accurate usage of installation disk space.
Run a various combination of disk space availability by using some tools to automatically making disk space full while installation. Check system behavior on low disk space conditions while installation.
#13) As you check installation you can test for uninstallation also. Before each new iteration of installation make sure that all the files written to disk are removed after the uninstallation.
Sometimes uninstallation routine removes files from only last upgraded installation keeping the old version files untouched. Also, check for rebooting option after uninstallation manually and forcefully not to reboot.
I have addressed many areas of Manual as well as Automated Installation Testing procedure.
Still, there are many areas you need to focus on depending on the complexity of your software under installation. These not addressed important tasks includes installation over the network, online installation, patch installation, Database checking on Installation, Shared DLL installation and uninstallation, etc.
Hope this article will be a basic guideline to those having trouble to start with Software Installation Testing both manually or in automation.