How to Improve the Test Release Process for Successful Bug Free Software to Production

In this article, we will learn how to have successful bug free software release to production. Let’s get started. 

Let’s look at the typical process involved in delivering software from the development phase to the testing phase for a successful bug-free software release to production/client.

These processes are either overlooked or skipped by software companies, which results in poor test management and thereby’ a “buggy” software releases to the client, which leads to “unsatisfied customers”.

Although a lot of time and great effort is being given by the testing team for each and every release, when the released software is not having the defined or bench-marked quality or does not meet the expected criteria, it will not only affect the company reputation with the customers but also demotivates and demoralizes the project team and most importantly the testing team as a whole.

How To Have Successful Bug Free Software Release to Production

If you are part of a testing team in this scenario, you might keep thinking “how to improve my testing capabilities and is there any better way of overcoming this situation”.

I have provided some tips and suggestions, based on my experience with various testing teams involved in software applications and enterprise product releases with multiple domains, and platforms and with multiple testing frameworks, on how to improve the test release processes, which will simplify your professional life as a test engineer or a test manager for delivering world-class software.

Test Release Process

The table given below gives an overview of a test release process with three universal phases like Input, Process & Output.

Each testing team should create a unique checklist for software release, as per the domain and platform of the software and the project management methodology (like Agile Scrum etc.,) and as per manual/automated testing framework, to accept the released build before starting the test execution to save time and effort.

This is one of the most important efficiency parameters in the test release phase.

Test Release Process Improvement

1) Review the Release Report for the new functionality, customization/modification of existing functionality, bug fixes from the previous build, which will decide whether to start executing the Smoke Testing or Sanity Testing or a combination of both.

2) Review the Update Testing Documents as per the new functionality and bug fixes, if not updated already. Normally, during the software development life cycle, these documents get updated by the testing team based on the regular weekly project review meetings.

3) Review the Software Build in Configuration Repository has been updated for the build number and version number labelled or commented with the release name as per the standards defined in the project plan. Also, ensure the build is successfully compiled and installed on the testing server.

4) Schedule a Quick Project Review Meeting after Release to discuss the pros and cons of the released build, known bugs, critical functionality, etc., to avoid any miscommunication and to review any important client requirements. Strictly avoid any oral communication between the development and testing teams as it highly impacts the quality of the software release.

5) Ensure the Bug Tracking Tool is Configured Properly for the allocated testing team and development team of the project, software build and release numbers as well as the modules/functionality of the software, which will help to log the bugs efficiently. If not, it should be escalated to the project manager or test manager on a high priority basis.

6) Return the Build to the Development Team without any compromise if the build fails in Smoke or Sanity Testing. Strictly, testing should not continue if the system fails Smoke Testing. This will save a lot of time and effort and improve the quality of the software released in the subsequent releases.

7) Schedule the Project Release on the 1^st Day of the Week which will help the test manager to plan the upcoming test cycle based on the build stability and also to send a quick test report to the project manager which will escalate the quality of the software well in advance. If the development team schedules the project release on Friday, the weekend can be utilized for any slippages as well as any build issues in a manual or automated build framework.

8) Ensure the Testers are Trained Properly in the Domain which will help the testing team to adhere to the testing schedule and gather time for the next round of testing. Also, the testing team should be trained and have the exposure to the required technology like Scripting and SQL if the project demands white boxing.

9) Avoid the Allocation of Testers in Multiple Projects as it greatly affects the quality of test execution in real time. In practice, even experienced testers overlook the features and functionality as well as skip the test cases assuming that some test cases never fail, when overloaded with work or allocated on multiple projects with deadlines.

10) Appreciate the Testing Team to have Passion as testers should not work for the “Day” or comment “Call it a Day”. When software has multiple modules and the functionalist is completely or partly integrated or interrelated, testers should have the passion for writing/executing the test cases with great coverage and traceability matrix, targeting the quality of the final software/product. Because even a cosmetic issue is a “bug” and counted as “1 bug”.

11) Ensure the Software Installation is Easy and Straightforward as it helps the testing team to re-install the software when required instead of waiting for the development manager or an installation manager to do the same job, which will kill the available testing time unnecessarily. For example, even though Windows-based installation is easy, when it involves multiple web servers and wide area networks in a multi-tier testing environment, testers may take hours to install the software. If the software testing covers and installation, uninstallation, patches or updates of software, it is more likely to be discussed in detail the process of executing the test cases with the testing team.

12) Ensure the Automated Tools Are Available with License for an Automated Testing Framework. The execution of test cases in an automated framework is easy compared to a manual testing scenario, provided if the automated tools are properly configured and licensed for multiple users. Especially when the testing plan involves performance and load testing apart from regular test case execution and regression testing, testers should cover executing test cases on multiple environments like multiple servers, multiple browsers, multiple users etc.

13) Ensure the Ghosted Machines are Setup for Testing before starting test execution. Ghosted machines are machines with different testing environments. For example, a web application software may be planned to test in multiple environments like Windows 7 & Access DB or Windows 2008 & SQL Server or Windows 8 & Oracle or Mainframe & DB2 etc., with all the Browsers like Chrome, Firefox, Internet Explorer, Safari etc., Some “System Testing” even requires to completely format the hard disk and install a fresh software or update the existing software with patches and updates etc.

14) Avoid Implementing the New Features/Change Requests by stopping test execution and re-releasing the software to start the testing phase again. This is a very bad practice in many software organizations as per the business requirements to satisfy the external customers or at least to meet the demands of the management steering committee or sometimes the sales/marketing teams. Even though the change requests from the customers are always encouraged in an “Agile” project environment, it should be properly planned and implemented before the release of the software to the testing team.

Managing and Controlling the Test Release Content

Managing and controlling the test release content is most important for any IT software or even for any non-IT software environment which will be depicted in the figure below.

• The project manager and/or the project steering committee, depending on the authority of organization matrix, will be responsible for selecting the content for each and every release.
• Once the bugs and/or the new features and the change request from the customers are identified and approved, it will be implemented by the development team which should be intimated to the project stakeholders before the development/implementation is started.
• Based on the implemented final release, the testing team will update the related documents and prepare for the testing accordingly.
• Testing team will start Smoke / Sanity testing as per the defined requirements in the release report.
• Once the sanity passes, the testing team will start the test execution as per the schedule and allocated tasks viz., Functional Testing, Non-Functional Testing, Security Testing, System Testing, Performance Testing, Load Testing, User Acceptance Testing etc.
• Once the first round of testing cycle is completed, test reports will be sent to all the stakeholders and the development team manager to plan for the next iteration of test execution.
• Depending on the status of the test reports and bug severity and complexity, a complete cycle of a second round of test execution or regression testing will be planned together with the user acceptance testing.
• After completing the planned test execution cycles, the test reports will be sent to all the stakeholders of the project for the passed/failed/missed of the features, functionality and bug fixes.

Sample Release Report Template

Note: Sample MS Word template for Release report is also available for download below.

Find below a “Sample Release Report” which covers the main aspects of the release process, which makes the whole project team’s professional life much happier than ever before.

GPSNavigation_Release_Report_Ver_1.0.7_Release_14.0_Build_105.25.03

#1) Scope

GPS Navigation for XYZ Company Limited is being released for internal testing. The released version is 1.0.7, the release number is 14.0 and build number 105.25.03. The software release includes new features and major bug fixes from the previous release. Smoke testing is passed from the development phase, but a Smoke & Sanity is required before going for Regression Testing.

#2) References

GPSNavigation_URD_1.0.12, GPSNavigation_FFD_2.17, GPSNavigation_BusinessUseCases_1.23.10, GPSNavigation_TestPlan_1.44, GPSNavigation_TestSuites_2.10, GPSNavigation_UnitTesting_23.3

#3) Release Description

This release is a controlled release of GPS Navigation and contains the following features and functionality:

The features marked in * are new in this release.

The following features have not been implemented in this release.

1. Module 1

1.1 Feature 1

1.1.1 Functionality 1

#4) Configuration Management
We are using Visual Source Safe as configuration management tool. The build is available on the following path:

Internal Link:  http://234.23.45.111/internalbuild/gpsnavigation/release1.0.13
External Link:   https:// 234.23.45.111/externalbuild/gpsnavigation/release1.0.13

#5) Installation Instructions and Steps
Give detailed information for the installation of the build to QA/Testing team.

#6) Issues / Bugs Fixed
The status of the bug is updated in the defect tracking system.

#7) Issues / Bugs To Be Fixed

#8) Deliverables

#9) Known Bugs / Issues

#10) Release Check List

=> Download: Click here to Download Sample Release Report Template in MS Word format.

Conclusion

How to Improve the Test Release Process in a Continuous Manner

Tip #1) Set up a release engineering team that will take care of the critical factors of maintaining the software releases and builds and is responsible for the centralized software configuration management systems.

Tip #2) Motivate and appreciate the project teams for following the processes involved in Software Development Life Cycle, Product Development Life Cycle, and Software Testing Life Cycle. We can define the process but until and unless it is being followed by the people involved, there is no use in defining the process.

Tip #3) Estimate the testing effort based on previous experience and history. Writing test cases is different from executing the same. The testers should understand what to test, how to test, and when to test, otherwise, the efforts given to the testing cycle get wasted, even though multiple rounds of the test cycle happened.

Tip #4) Finally, if possible and feasible, automate the testing phase using some globally accepted testing tools. Use of automated build tools and automated testing tools reduces testing efforts by more than 50% improving the quality of software and ensure 100% quality if the automation framework is properly designed.

Tip #5) Last but not least, test release is not just a job, it’s an art of making all the stakeholders’ life in a project easier and much more comfortable.

About the author: Balu A. is an experienced techno-functional IT professional with over two decades of IT software experience and a decade of project and test management experience delivering enterprise applications and mobility solutions across domains using Microsoft, Oracle, Java, and Mobile technologies. He is basically a leader with a passion for promoting people to become leaders with the right attitude loves to work in a process-oriented environment and believes process improves employee efficiency, quality, and productivity.

In the next tutorial, we will learn – How to Improve Test Case Efficiency.

Let us know your thoughts and queries in the comments section below. We would love to hear from you. 

Get a Software Release as per the Process!
Complete Testing On Schedule with Great Productivity and Effort!!
Try to Achieve Bug-Free, Quality Assured Software Delivery!!!

If you like this article, please share it with your friends!