Several documents and reports are being prepared as part of Testing. Some are Test Strategy doc, Test Plan doc, Risk management Plan, Configuration management plan etc. Among these Test Summary Report is one such report which is prepared after the Testing is completed.
I have tried to explain the purpose of ‘Test Summary Report’ and provided a sample Test Summary Report template along with an actual report for download.
What is a Test Summary Report?
As we know, Software Testing is an important phase in SDLC and also it serves as the “Quality Gate” for the application to pass through and certified as “Can Go Live” by the Testing Team.
Test Summary Report is an important deliverable which is prepared at the end of a Testing project, or rather after Testing is completed. The prime objective of this document is to explain various details and activities about the Testing performed for the Project, to the respective stakeholders like Senior Management, Client etc.
As part of Daily status reports, daily testing results will be shared with involved stakeholders every day. But Test Summary Report provides a consolidated report on the Testing performed so far for the project.
Recommended reading => How to Report Test Execution Smartly (Status Report Template download)
Assume that if the Client who sits in a remote location need to understand the results and status about a Testing project which was performed for a period of, say for example – four months, Test Summary Report will solve the purpose.
This is also an artifact required to be prepared as part of CMMI process.
A typical Test Report template will contain the below information, however based on each Company’s format & practice, the contents may vary. I have also provided real examples for better understanding.
At the end of this article you can download a test summary report sample.
12 Steps Guide to writing an effective test summary report:
Step #1: Purpose of the document
<Short description about the objective of preparing the document>
Example: This document explains the various activities performed as part of Testing of ‘ABCD transport system’ application.
Step #2: Application Overview
<Brief description about the application tested>
Example:‘ABCD transport system’ is a web based Bus ticket booking application. Tickets for various buses can be booked using the online facilities. Real time passenger information is received from a ‘Central repository system’, which will be referred before booking is confirmed. There are several modules like Registration, Booking, Payment and Reports which are integrated to fulfill the purpose.
Step #3: Testing Scope
<This section explains about the functions/modules in scope & out of scope for testing; Any items which are not tested due to any constraints/dependencies/restrictions>
Example: A functionality verification which needs connectivity to a third party application cannot be tested, as the connectivity could not be established due to some technical limitations. This section should be clearly documented, else it will be assumed that Testing covered all areas of the application.
a) In Scope
Functional Testing for the following modules are in Scope of Testing
b) Out of Scope
Performance Testing was not done for this application.
c) Items not tested
Verification of connectivity with the third party system ‘Central repository system’ was not tested, as the connectivity could not be established due to some technical limitations. This can be verified during UAT (User Acceptance Testing) where the connectivity is available or can be established.
Step #4: Metrics
<Metrics will help to understand the test execution results, status of test cases & defects etc. Required Metrics can be added as necessary. Example: Defect Summary-Severity wise; Defect Distribution-Function/Module wise; Defect Ageing etc.. Charts/Graphs can be attached for better visual representation>
a) No. of test cases planned vs executed
b) No. of test cases passed/failed
c) No of defects identified and their Status & Severity
d) Defects distribution – module wise
Step #5: Types of testing performed
<Describe the various types of Testing performed for the Project. This will make sure the application is being tested properly through testing types agreed as per Test Strategy.
Note: If several rounds of Testing were done, the details can also be included here.>
a) Smoke Testing
This testing was done whenever a Build is received (deployed into Test environment) for Testing to make sure the major functionality are working fine, Build can be accepted and Testing can start.
b) System Integration Testing
c) Regression Testing
Step #6: Test Environment & Tools
<Provide details on Test Environment in which the Testing is carried out. Server, Database, Application URL etc. If any Tools were used like Quality Center (now HP ALM) for logging defects>
Step #7: Lessons Learned
<This section is used to describe the critical issues faced and their solutions (how they were solved during the Testing). Lessons learnt will help to make proactive decisions during the next Testing engagement, by avoiding these mistakes or finding a suitable workaround>
Step #8: Recommendations
<Any workaround or suggestions can be mentioned here>
Step #9: Best Practices
<There will be lot of activities done by the Testing team during the project. Some of them could have saved time, some proved to be a good & efficient way to work, etc. These can be documented as a ‘Value Add’ to show case to the Stakeholders>
Step #10: Exit Criteria
<Exit Criteria is defined as a Completion of Testing by fulfilling certain conditions like
(i) All planned test cases are executed;
(iI) All Critical defects are Closed etc.>
a) All test cases should be executed – Yes
b) All defects in Critical, Major, Medium severity should be verified and closed – Yes.
c) Any open defects in Trivial severity – Action plan prepared with expected dates of closure.
No Severity1 defects should be ‘OPEN’; Only 2 Severity2 defects should be ‘OPEN’; Only 4 Severity3 defects should be ‘OPEN’. Note: This may vary from project to project. Plan of Action for the Open defects should be clearly mentioned with details on when & how they will be addressed and closed.>
Step #11: Conclusion/Sign Off
<This section will mention whether the Testing team agrees and gives a Green signal for the application to ‘Go Live’ or not, after the Exit Criteria was met. If the application does not meet the Exit Criteria, then it can be mentioned as – “The application is not suggested to ‘Go Live’. It will be left with the decision of Senior Management and Client and other Stakeholders involved to take the call on whether the application can ‘Go Live’ or not.>
Example: As the Exit criteria was met and satisfied as mentioned in Section 10, this application is suggested to ‘Go Live’ by the Testing team. Appropriate User/Business acceptance testing should be performed before ‘Go Live’.
Step #12: Definitions, Acronyms, and Abbreviations
<This section mentions the meanings of Abbreviated terms used in this document and any other new definitions>
=> Download Sample Test Summary Report:
Click here to download a sample test report template with an example.
Test summary report is an important deliverable and focus should be to prepare an effective document, as this artifact will be shared with various stake holders like senior management, client etc.
After performing an exhaustive testing, publishing the test results, metrics, best practices, lessons learnt, conclusions on ‘Go Live’ etc. are extremely important to produce that as an evidence for the Testing performed and the Testing conclusion.
We have also made available the test report sample for download. It is a perfect example of how to prepare an effective test summary report!
About the author: This is a guest post by Baskar Pillai. He is having around 14 years of experience in Test management and end to end software testing. CSTE certified Testing professional, trainer, worked in IT majors like Cognizant, HCL, Capgemini and currently working as Test Manager for a large MNC.
Please let us know your Comments/questions/thoughts.
Like this article? Please consider sharing it with your friends. After all sharing is caring!