Several documents and reports are being prepared as part of Testing. Some are Test Strategy doc, Test Plan doc, Risk management Plan, Configuration management plan etc. Among these Test Summary Report is one such report which is prepared after the Testing is completed.
I have tried to explain the purpose of ‘Test Summary Report’ and provided a sample Test Summary Report template along with an actual report for download.
What is a Test Summary Report?
As we know, Software Testing is an important phase in SDLC and also it serves as the “Quality Gate” for the application to pass through and certified as “Can Go Live” by the Testing Team.
Test Summary Report is an important deliverable which is prepared at the end of a Testing project, or rather after Testing is completed. The prime objective of this document is to explain various details and activities about the Testing performed for the Project, to the respective stakeholders like Senior Management, Client etc.
As part of Daily status reports, daily testing results will be shared with involved stakeholders every day. But Test Summary Report provides a consolidated report on the Testing performed so far for the project.
Recommended reading => How to Report Test Execution Smartly (Status Report Template download)
Assume that if the Client who sits in a remote location need to understand the results and status about a Testing project which was performed for a period of, say for example – four months, Test Summary Report will solve the purpose.
This is also an artifact required to be prepared as part of CMMI process.
What Test Summary Report contains?
A typical Test Report template will contain the below information, however based on each Company’s format & practice, the contents may vary. I have also provided real examples for better understanding.
At the end of this article you can download a test summary report sample.
12 Steps Guide to writing an effective test summary report:
Step #1: Purpose of the document
<Short description about the objective of preparing the document>
Example: This document explains the various activities performed as part of Testing of ‘ABCD transport system’ application.
Step #2: Application Overview
<Brief description about the application tested>
Example:‘ABCD transport system’ is a web based Bus ticket booking application. Tickets for various buses can be booked using the online facilities. Real time passenger information is received from a ‘Central repository system’, which will be referred before booking is confirmed. There are several modules like Registration, Booking, Payment and Reports which are integrated to fulfill the purpose.
Step #3: Testing Scope
- In Scope
- Out of Scope
- Items not tested
<This section explains about the functions/modules in scope & out of scope for testing; Any items which are not tested due to any constraints/dependencies/restrictions>
Example: A functionality verification which needs connectivity to a third party application cannot be tested, as the connectivity could not be established due to some technical limitations. This section should be clearly documented, else it will be assumed that Testing covered all areas of the application.
a) In Scope
Functional Testing for the following modules are in Scope of Testing
b) Out of Scope
Performance Testing was not done for this application.
c) Items not tested
Verification of connectivity with the third party system ‘Central repository system’ was not tested, as the connectivity could not be established due to some technical limitations. This can be verified during UAT (User Acceptance Testing) where the connectivity is available or can be established.
Step #4: Metrics
<Metrics will help to understand the test execution results, status of test cases & defects etc. Required Metrics can be added as necessary. Example: Defect Summary-Severity wise; Defect Distribution-Function/Module wise; Defect Ageing etc.. Charts/Graphs can be attached for better visual representation>
a) No. of test cases planned vs executed
b) No. of test cases passed/failed
c) No of defects identified and their Status & Severity
d) Defects distribution – module wise
Step #5: Types of testing performed
- Smoke Testing
- System Integration Testing
- and Regression Testing
<Describe the various types of Testing performed for the Project. This will make sure the application is being tested properly through testing types agreed as per Test Strategy.
Note: If several rounds of Testing were done, the details can also be included here.>
a) Smoke Testing
This testing was done whenever a Build is received (deployed into Test environment) for Testing to make sure the major functionality are working fine, Build can be accepted and Testing can start.
b) System Integration Testing
- This is the Testing performed on the Application under test, to verify the entire application works as per the requirements.
- Critical Business scenarios were tested to make sure important functionality in the application works as intended without any errors.
c) Regression Testing
- Regression testing was performed each time a new build is deployed for testing which contains defect fixes and new enhancements, if any.
- Regression Testing is being done on the entire application and not just the new functionality and Defect fixes.
- This testing ensures that existing functionality works fine after defect fix and new enhancements are added to the existing application.
- Test cases for new functionality are added to the existing test cases and executed.
Step #6: Test Environment & Tools
<Provide details on Test Environment in which the Testing is carried out. Server, Database, Application URL etc. If any Tools were used like Quality Center (now HP ALM) for logging defects>
Step #7: Lessons Learned
<This section is used to describe the critical issues faced and their solutions (how they were solved during the Testing). Lessons learnt will help to make proactive decisions during the next Testing engagement, by avoiding these mistakes or finding a suitable workaround>
Step #8: Recommendations
<Any workaround or suggestions can be mentioned here>
- Admin control for defect management tool can be given to Offshore Test manager for providing access to Testing team.
- Each time the onsite Admin need not be contacted for requests whenever they arise, thereby saving time due to the geographical time zone difference.
Step #9: Best Practices
<There will be lot of activities done by the Testing team during the project. Some of them could have saved time, some proved to be a good & efficient way to work, etc. These can be documented as a ‘Value Add’ to show case to the Stakeholders>
- A repetitive task done manually every time was time consuming. This task was automated by creating scripts and run each time, which saved time and resources.
- Smoke test cases were automated and the scripts were run, which ran fast and saved time.
- Automation scripts were prepared to create new customers, where lot of records need to be created for Testing.
- Business critical scenarios are separately tested on the entire application which are vital to certify they works fine.
Step #10: Exit Criteria
<Exit Criteria is defined as a Completion of Testing by fulfilling certain conditions like
(i) All planned test cases are executed;
(iI) All Critical defects are Closed etc.>
a) All test cases should be executed – Yes
b) All defects in Critical, Major, Medium severity should be verified and closed – Yes.
c) Any open defects in Trivial severity – Action plan prepared with expected dates of closure.
No Severity1 defects should be ‘OPEN’; Only 2 Severity2 defects should be ‘OPEN’; Only 4 Severity3 defects should be ‘OPEN’. Note: This may vary from project to project. Plan of Action for the Open defects should be clearly mentioned with details on when & how they will be addressed and closed.>
Step #11: Conclusion/Sign Off
<This section will mention whether the Testing team agrees and gives a Green signal for the application to ‘Go Live’ or not, after the Exit Criteria was met. If the application does not meet the Exit Criteria, then it can be mentioned as – “The application is not suggested to ‘Go Live’. It will be left with the decision of Senior Management and Client and other Stakeholders involved to take the call on whether the application can ‘Go Live’ or not.>
Example: As the Exit criteria was met and satisfied as mentioned in Section 10, this application is suggested to ‘Go Live’ by the Testing team. Appropriate User/Business acceptance testing should be performed before ‘Go Live’.
Step #12: Definitions, Acronyms, and Abbreviations
<This section mentions the meanings of Abbreviated terms used in this document and any other new definitions>
=> Download Sample Test Summary Report:
Click here to download a sample test report template with an example.
Few points to note while preparing the Test Summary Report:
- As part of Test Execution, collect all required information on the Testing performed. This will help to prepare a sound Test summary report.
- Lessons learned can be explained in detail, which will convey the Responsibility which was taken to solve these issues. Also this will be a reference for upcoming projects to avoid these.
- Similarly, mentioning the Best Practices will portray the efforts taken by the team apart from regular testing, which will also be treated as a “Value Addition”.
- Mentioning the Metrics in graphics form (Charts, Graphs) will be a good way to visually represent the status & data.
- Remember, Test summary report shall mention and explain the activities performed as part of the Testing, to the recipients to understand better.
- Few more appropriate sections can be added if required.
Test summary report is an important deliverable and focus should be to prepare an effective document, as this artifact will be shared with various stake holders like senior management, client etc.
After performing an exhaustive testing, publishing the test results, metrics, best practices, lessons learnt, conclusions on ‘Go Live’ etc. are extremely important to produce that as an evidence for the Testing performed and the Testing conclusion.
We have also made available the test report sample for download. It is a perfect example of how to prepare an effective test summary report!
About the author: This is a guest post by Baskar Pillai. He is having around 14 years of experience in Test management and end to end software testing. CSTE certified Testing professional, trainer, worked in IT majors like Cognizant, HCL, Capgemini and currently working as Test Manager for a large MNC.
Please let us know your Comments/questions/thoughts.
Like this article? Please consider sharing it with your friends. After all sharing is caring!