In my Software Testing career, I have never heard people talking much about software testing documentation.
The general opinion about testing documentation is that anyone who has free time can do the documentation like a Test case, Test plan, status report, Bug report, project proposal, etc.
I myself won’t stress more about the documentation, but I can say it’s my habit to place all the data in black and white and to update others as well about that.
What You Will Learn:
Importance of Software Testing Documentation
My Experience
Just wanted to share my experience with you:
We have delivered a project (with an unknown issue in it) to one of our clients (angry client). The issue was found on the Client-side and it was a very bad situation for us. As usual, all the blame was on the QA’s.
The issue was something regarding the compatibility of one website. When it came to me, I was having proof that I didn’t get such a requirement document which stated that I have to check the compatibility of the website as well. Thank god I was safe.
That was a lesson for me, I realized the importance of documentation and from that day I started to work on documents and started creating test documents like Test plan, Test cases, sanity testing checklist, bug report and many.
“Ink is better than the best memory” – Chinese proverb
Test Documentation: What’s That?
We all read various articles on testing technologies and methods, but how many of us have seen articles on documentation? No doubt there are few, is it that documents are not important? No, but its’ because we have not yet realized the importance of documents.
But, if we observe then the fact is, projects that have all the documents have a high level of maturity.
Most of the companies do not give even a little importance to the documentation as much as they give to the software development process. If we search on the web then we can find various templates on how to create various types of documents. But how many of them are really used by organizations or individuals?
The fact is that careful documentation can save an organization’s time, effort and money.
While going for any type of certification, why is there a stress given on documentation, it’s just because it shows the importance of client and processes to individual and organization. Unless you are able to produce a document that is comfortable to the user no matter how good your product is, no one is going to accept it.
In my experience, we own one product, which is having a bit of confusing functionality.
When I started working on that I asked for some help documents from my manager and I got the answer “No, we don’t have any documents” Then I made an issue of that because as a QA I knew, no one can understand how to use the product without documents or training.
If the user is not satisfied, then how are we going to make money out of that product?
“Lack of documentation is becoming a problem for acceptance” – Wietse Venema
Even the same thing is applicable to User manuals. Take the example of Microsoft, they launch every product with proper documents, even for Office 2007 we have documents that are very explanatory and easy to understand for any user. That’s one of the reasons for which all their products are successful.
In small companies, we always hear “project rejects in propo sal or kickoff phase” it’s just because the proposal documentation lacks concise and expressive language, and to show the capability of the organization.
It’s not that small companies can’t deliver good quality projects but it’s their inability to express their capability. (I also work with a small organization of 80 employees, and I have heard this many times).
I personally feel Quality is the only department that can make it possible. We are the only department which can argue on this and can provide a successful future for our organizations.
Let’s organize all the discussions in a few points from a quality perspective:
- Clarify quality objective and methods.
- Ensure clarity about tasks and consistency of performance.
- Ensure internal coordination in client work.
- Provide feedback on preventive actions.
- Provide feedback on your planning cycle.
- Create objective evidence of your quality management system’s performance.
10 Tips to Help You Achieve Test Documentation Goals
In general, software testing documentation is “It can be done only by the person who has free time”. We need to change this mindset, and only then we can leverage documentation power on our projects.
It’s not that we don’t know how to do the documentation right. We just don’t think it’s important.
Everyone must have standard templates for all kinds of documentation starting with Test strategy, Test Plan, Test cases, and Test data for the Bug report.
This is just to follow some standards (CMMI, ISO, etc.) but when it comes to the actual implementation, how many of these documents are really used by us? We just need to synchronize our quality process with documentation standards and another process in an organization.
The simplest way to follow all kinds of documentation is to involve a person in the project from the kick-off phase who understands the project dynamics, domain, objective, and technology. Who else is better than QA person for this (of course there are technical writers present to do this – but consider a general scenario of small companies where technical writers are not present).
To achieve this goal of testing and documentation, I feel we need to focus on some points as given below.
Here are the top 10 tips to help you achieve your test documentation goals:
#1) QA should involve in the very first phase of the project so that QA and Documentation work hand in hand.
#2) The process defined by QA should be followed by technical people, this helps to remove most of the defects at a very initial stage.
#3) Just creating and maintaining Software Testing Templates is not enough, force people to use them.
#4) Don’t just create and leave the document, Update as and when required.
#5) Change requirement is an important phase of the project, don’t forget to add them to the list.
#6) Use version control for everything. This will help you to manage and track your documents easily.
#7) Make defect remediation process easier by documenting all defects. Make sure to include a clear description of the defect, reproduce steps, affected area and details about the author while documenting any defect.
#8) Try to document what is required for you to understand your work and what you will need to produce to your stakeholders whenever required.
#9) Use the standard template for documentation, just like any excel sheet template or doc file template and stick to it for all your document needs.
#10) Share all project-related documents at a single location that is accessible to all team members for reference as well to update whenever required.
I am not saying that by applying the steps you will get sudden results. I know this change won’t happen in a day or two, but at least we can start so that these changes start happening slowly.
After all “the documentation needs documentation”. Isn’t it?
There are hundreds of documents used in the software development and testing lifecycle.
Important Software Testing Documents
Enlisted below are a few important software testing documents that we need to use/maintain regularly:
- Test Plan
- Test Design and Test Case Specification
- Test Strategy
- Test Summary Reports
- Weekly Status Report
- User Documents / Manuals
- User Acceptance Report
- Risk Assessment
- Test Log
- Bug Reports
- Test Data
- Test Analysis
Software testers regularly need to refer to the following documents:
1) Software Requirement Specifications
2) Functional documents
Conclusion
Software Testing Documents always play an important role in the Project development/Testing phase. So always keep things documented whenever possible. Don’t rely on verbal communication. Always be on the safe side.
Documentation will not only save you but also help the organization in the long run by saving thousands of dollars on training and most importantly on fixing issues caused due to lack of development and testing documents.
Don’t document, just to avoid finger-pointing at you, the habit of documentation will certainly bring a systematic approach to your testing process, leaving the ad hoc testing behind.
About Author: This article is written by STH team member Tejaswini. She is working as a QA manager in an organization.
What other documents do you maintain during your daily testing activities? Feel free to share your thoughts in the comments section below.
Nice reading your experience. I too have faced same incidences in my career. Always keep written proofs for what you do or don’t do.
Also having documents can help new team members to understand the project.
Its really a nice article & its very true.
am working in a small organization of 20 people, we are only 3 tester here, so can you plz guide me which documents i should ask to development team or client ?
I strongly feel the need for same. Very nice article.
Hello Tejaswini, first let me say Thanks a lot to you for your effort.
Documentation is an important not only to QA people, it is also important to Development team and organization, because it is a proof to client if we get any issues and important for future reference. Every document will have a particular piece of information. I will wait for your part II.
Nice work!
You narrated very well about the importance of documentation. But, about the situtaion you had with your angry client, I don’t think that the proof of requirements document saved you (or QA team) from getting blamed. Almost every process oriented 0rganization maintains designing the requirements document. In your case, instead of searching through all the requirements documents, the best practice could be looking at Traceability Matrix document, through which you could easily identify if there is any such requirement from the customer and if so, have we addressed it or not.
I also feel that, requirements document is not a work product of tester’s effort. However, tester can verify it for the correctness of same.
As I said earlier, you did a great job on posting this article. I also agree that most of them in the team have an impression that the documentation is a free time work. To vanish this myth, tester should be more creative to prepare effective documentation. To derive effective scenarios they should speak to SMEs/BAs/Sr.Developers to understand the customers business process well. In our agile process, we use user stories in preparing test cases.
Once again, thanks for sharing the info and keep up the good work.
Best Regards,
Sidhartha
Nice and informative tejaswini.
Thnx for the nice post and looking ahead for your next.
But what ever you have suggested is an ideal case but in real time the actual documentation(i mn user guides, TRR etc) will be prepared only during I cuts or at the last phase of the projects, so my query here is..Can we(QA) refer to SDS and Functional specs for writing test strategy and in turn test cases documents?
Urgent opening !!
Domain Web Development
Technical skills required LAMP, PHP Lead, Ajax, Drupal, Joomla , Smarty, PEAR and PECL libs, HTML, XHTML
Experience 1.0 to 2.0 Years
Salary As per Industry standards [Negotiable]
No. of Job Positions 06
Role Software engineer
Additional Skills required Good command over written & verbal ENGLISH communication.
send your CV’s on anju_2085@rediffmail.com
Above opening is in Pune
great article. Documetation are important as provide lot of information. For QA, documentation can be a protective cover or proofs to prevent any blaming.
Hi Vyas,
Surely we can use Functional Spec doc to write test cases, but you can cover only functional test cases from that. You need a brief workflow to understand the system.
Hey Sid,
Thanks for your reply. Appreciate your comment on the post. Yes yo are right that only requirement document can’t save us, but i must say, it’s better to have something than nothing.
Hi Tejaswini,
Great work ! Very useful article about Documenting.
Thanks.
hi sid ,
ur points looked very valid, and thnkx for the updates
good one!!
HI Tejaswini,
Nice article. Actually in CMM level companies they have document for every thing. They have defined process to follow as Sidartha mentioned above, but still we should practice it religiously. I am giving you an exmaple of my own why this habit is imp.
When i switched to CMM level 5 from a small organization, I wasn’t much disciplined about documents. I had to work on very confidential defense project and that also was complete and about to release. I worked hard and got many defects. My team was happy with my efforts and they fixed all critical defects. Client approved our project and our software was best. (There were several other organizations in the race).
After finishing all this when i was back in office QA people asked me to start document related process.(Actualy when i had joined i don’t had enough time to maintain all documents)
God i found myself lost in document jungle and worst part was i had forgot half of defects i had found.
Thankfully my team ( development team) was very helpful and QA ppl were little friendly i completed all with their help.
So now onwards i never go without proper documents.
Just noticed the last section of your article – ‘What other documents do you maintain in your daily testing activities?’. Sorry for missing that. Ours is a SCRUM, an agile development approach. We spend very less time on documenation. We interact a lot with the developers and product owners to get the requirement details. According to those requirement details, we first prepare a TAD (Test Approach Document) in which tester describes the feature he/she is testing. Provides a brief description on the functionality before and after adding the feature, what needs to be tested, exit criteria, for what scenarios the automation scripts are covered and this document will be get verified with the developer and product owner. If they feel that everything is covered then we go ahead with testing. Or it will be adjusted with any changes they suggested.
While preparing TAD, tester gains more functional knowledge and also user stories from the product owner helps the tester to understand the business process specific to the customer. Individuals are never blamed in our process because TAD has been reviewed by fellow testers in the team and also with the Developer and Product Owner.
Best Regards,
Sid
Interesting. Just a query, how processes fits in Agile methodology, as i m very new to this concept, just want to know more about this.
Tejaswini
Nice article really stresses the importance of the documention.
I had also worked in a ajile project as a lone QA. I liked the term user story as did mentioned by Sid because it’s more I had came across. In my project I can say there was no SRS. Infact it was the mails from the client (technical client) & discussions over the phone. Basically our(test + dev team) clarifications & queries were answered and I just derived the test cases from from the same. At the signoff the only documented stuffs which supported the project is the test cases, bug report & test execution report. We are not dealing with the end client directly…
My project is released by phase by phase. As I documented the defect leakage, It helps me in the comparison with next phases and just shared the same with all…
Hi,
Liked your article..
I am working on an AGILE project. We maintain the requirements of a story on the twiki. This helps multifold- requirement sign-off, requirement traceability, test automation traceability.
The thumb rule is to question yourself about the benefit of each and very documentation you or your team works on..Ask yourself Why is this required? How will the organisation/ stakeholders benefit from this? What will be it contain? How often will it be updated? Who will update it?…..
For e.g, in AGILE we don’t document test plans as they are outdated as soon as they are documented. The test plans are made daily in the stand-up meeting..
Hi Tejaswini
I am fresher in testing from non IT background. your post is very helpful for people’s like me . it gives the importance of documentation
Thank u……………………………
Dear Tajeswini Da,
Thanks for your useful and helpful topics. I have some questions dada,
1. When and why testing plan is necessary?
2. Who is responsible for test plan (tester who is beginner or what level of tester) ?
3. When test plan start and stop?
4. What types of problem I have fetched if I dont have any test plan?
The answer is helping me to get clear idea about test plan.
Thanks,
Nigar Sultana Kana
Hi
It is true that documentation is necessary as it help the tester to understand the things whenever it is needed and the way it is working and do not have to keep the same in the memory.
I always use to maintain document for each testing phase/ testing module etc. And I used to ask my team also to do so. This document helps other team members to understand easily and in very less time. People call me strict but for maintaining good quality one has to do it.
But I always feel if one has made the mistake the other person should not do the same mistake and let him make another mistake but it should be documented along with the solution this will achieve quality and understanding level.
Why I said ‘Let him make another mistake’ as generally 99% of people learn from mistakes.
We generally used to send out open clarifications through emails. Our client on the other side of the globe, either answered through email replies or over the status call.
Later when testers or BA at client side logs a ticket or makes issue out of any behaviour that is earlier being clarified, we had to search for the clarifications answered, date, person answered, etc…
To avoid all this, now we have a single clarification excell file. Any clarifications that goes between offshore and client, vice versa is documented here. And this is our first point of reference in case of bugs\issues.
The above is one of documents that we maintain and track.
I see, you have missed out tracebility document. This helps us during impact analysis.
Thanks,
Madhu K
Hi Tejaswini,
Really a nice article.
It gives more clear idea about the documentation process which generally do not get follow in small companies like ours.
But after going through this article it gives more clear idea why the documentation is really important.
Keep posting up.
@ Nigar Sultana Kana
Thanks for your appreciation.
You might get all your questions answers in my next article.
Want to know about Test Strategy? how it is different from test plan? please give me details for what test strategy documents contains possible with example?
email – aishwarya.koche@gmail.com
documentation is a vital reference for the future releases as well
Yes, a properly documented software always get success in teh market and also saves the developer as well as the QA Person. 100% agreed
vijay, why dont u block this “AROON”
Hi,
i have worked with an organization where documentation was everything but now in my new org there is nothing like that and i am very much here for everything they say “we should have done this but didn’t do” i am very new to the org and joined as a test lead don’t want to cont.. they way everything is manage here can u please suggest me what should i do
Manu
Hi,
i have worked with an organization where documentation was everything but now in my new org there is nothing like that and for everything about documentation they say “we should have done this but didn’t do” i am very new to the org and joined as a test lead don’t want to do the same as the way everything is managed here can u please suggest me what should i do
Manu
Hi tejashwini,
first let me say Thanks a lot to you for your effort. You narrated very well about the importance of documentation.Few months back only I have started working as a QA engineer. Right now I am working with very small organization here we don’t prepare any documentation for testing purpose. But after reading your article, I am going to start maintaining documentation for each and every small things.
Once again, thanks for sharing your knowledge and your experience.
Hi Tejaswini,
first let me say Thanks a lot to you for your effort. You narrated very well about the importance of documentation.Few months back I have started working as a QA engineer. Right now I am working with very small organization here we don’t prepare any documentation for testing purpose(other than test report and bug report). But after reading your article, I am going to start maintaining documentation for each and every small things.
Once again, thanks for sharing your knowledge and experience.
@ Manu
Thanks for your appreciation
Take your current work and organization as challenge and start from yourself. When you could able to show the importance of processes and documents to your organization then definitely they will start using it.
That’s what i can suggest
@ Advait
Thanks for your comments, It’s rally a nice feeling that after reading my experiences at least once person started implementing it.
Thanks a lot
Thanks for such nice explaination about testing document it will be really very helfull for me.
even you other topic also very usefull.
over all i would like to says that all information about software testing posted by you is accurate.
Thanks lot..
Abrar khan
Hi tejashwini,
I need your help i have one query, Most of the times we wont get enough time to perform testing in detail, in that case we perform Ad-hoc testing.So while performing Ad-hoc testing which kind of documents we should maintain? Which things we should note down for further reference?
@ Tejaswini
Dear Tejaswini,
I am working as a Manual Tester in small organizasion. They are not providing any documents. Simply they use to explain about modules and they assign to test it. Can I know how and what are the documents can we maintain at this situation. Pls tell me the proper procedure. Thanks in advance…
nice article about documentation.what is the first document we produce from the testing department
@Advait
While doing Ad-hoc testing i used to prepare my own document which contains the major functional part or the major areas which client/manager wants us to highlight, like, OS you used, or any major functionality part
I hope it answers you query
@VNK
There are lists of documents you will find on softwaretestinghelp.com 🙂
Hello Tejaswini,
Thank you very much. Now i got rough idea.I will try to maintain such kind of documents while doing Ad-Hoc testing. If i get any doubt while preparing such documents i will get back to you.
I hope you will clear my doubts.
Thanking you.
@ Tejaswini,
Dear Tejaswini,
I need some help from you. I want to know which is the best procedure when our organization does not have base documents. As usually I use to understand about module by exploring or interacting with developer. After that I use to prepare test cases and execute. I use to prepare bug report if I find any. Is it enough ??? or anything else??? Pls let me know.
Hello Tejaswini,
I am working with a product based company since 6 months. I am the only tester and multiple projects are there.
but there is complete absence of documentation, so its difficult to make test cases.and no test process has been defined.
can you pl suggest me how do i test my projects efficently? or give some testing tips. thanks a lot in advance.
Hi,Tejaswini
Thanks for giving information for testers,this is very good work for giving the knowlege about the testing.
Thanking you.
Hello Tejaswini,
Thanks for sharing importance of Documentation.
This is true that it is a responsibility of the QA to create/ update all project related document.
Hi Tejaswini,
A very worth reading article.Thanks for sharing
Hi all,
All your document related articles are very useful to all and especially to me.Thank you all
I’m staying in bangalore.I’ve been worked as a java developer for 2 years then they put me in to performance tuning and testing side.I have learnt some online tools,plugins and IBM Rational performance tester tool which is related to performance testing.I have around 6-8 months in this.This is purely technical testing.Now i’m confused whether i have to continue into performance testing itself or again going to developing side.can anyone tell me which is best? Performance tester carreer is better than developer?
Please reply as soon as possible.I’m planning to switch other company.If performance testing carrer is good then do i need to take any course.
Waiting for your reply,
Nagaraj
“We won’t get anything without we try it”
Hello Tejaswini,
HI Tejaswini i got some important of documentation in ,manual testing .but one thing can explain the for every organization follow the same documentation or not iam still not confirm.so please explain.but your done a great job yaar. after user acceptance testing is there need any tester or not .
@ anybody
I need some help from you. I want to know which is the best procedure when our organization does not have base documents. As usually I use to understand about module by exploring or interacting with developer. After that I use to prepare test cases and execute. I use to prepare bug report if I find any. Is it enough ??? or anything else??? Pls let me know.
Thats really a nice article.
Hi Tejaswini,
Can you give me any brief idea about IEEE 829 test plan documentation standards?
hi
cool. very good ariticle. appreciate your efforts achievments
Can you give me brief description of test planning,so what is a test plan?
What is risk management?
@ Vikas Patil
Test Plan is nothing but a brief review what we want to do in project, what methodologies we want to follow, who are the team members working and all, and do remember to update test plan if requirement changes, you can include risk assessment in Test plan itself
Let me also join in this group
Hi
It’s really interesting and seems to be motivation for testing people and like to join in this group
HI Tejaswini,
Nicely explained about the importance of the Testing documents, it’s true that we must require to prepare testing documentations before the testing task implementation for the successful testing.
Regards
Mallikarjun PATIL.
@Mallikarjun
Thank you
Nice article. Thank u…
Very Nice.. Clearly Explained with an realtime example… Thanks a Lot…..
Very Nice.. Clearly Explained with a realtime example… Thanks a Lot…..
Very Nice article….Thank You
Hi Tejaswini
I appreciate your effort in letting people who are working it IT know how important is documentation.
I work for a CRO in U.S as a Test Manager in a highly FDA regulated environment. According to FDA “Nothing will be understood as being done if you haven’t produced enough documentation around it”.
And also it is always safe and required to have appropriate documents around and about a system/software through out its life cycle for reconstruction of the scenario(s) which is in question.
Coming to creation of documentation you have to use your due diligence to include what has been done and how.
I think the process of Testing a system or a software is more contextual rather than what we can define in a generic way.
I also want to acknowledge and extend my gratitude to those who took their valuable time to add their experiences which are worth knowing and amazing.
I like to be a part of these type of eConversations as you have always much more to learn about.
Regards
Meher
Hi teja..
Its very nice about your article. Even me too faced some bad experience in my initial projects because of lack of documentation.
Thanks for this post.
Hi all,
As teja mentioned, documentation is very must.
# Apart from that we’ve to develop the good practice on maintaining documents.
Eg:
* Maintain proper folder structure,
* Maintain consistence format in naming the files,
* Placing the proper files in proper (related) folders, etc.
# Version should be maintained for all documents (possible with dates/correspondence/source/etc.)
Awesome article and thanks a lot for sharing ur huge experiance with us
Thank you all
@ Meher
Thanks for sharing your experiences
Very nice artical…keep on adding as much experience as u can..Thanx 4 sharing ur experience.
Really nice article… Thanks. Throws light on the importance of documentation in testing, really useful
Amdocs Pune is having opening for Telecom / non telecom manual testing .. exp 1-3.5 yrs
send me CV asap if u r interested
pikkuvaswani@gmail.com
Hi Tejaswini,
I have read your posts. All were so good. I am working as a Software Testing(Manual) Engineer past 3 years. still now i have never use these documents except test case reports. I am very happy to know for all these documents regarding testing. If you don’t mine can you please send me all testing related documents. I am unable to download some of the documents as you have provided. My Mail id is rajeshlab@gmail.com
Hi
Please send me risk related document on my email id
bwchaudhari@rediffmail.com
thanks
bhushan
Please let me know if there are any openings for Manual Testing. I have experiance of 1 year 9 Months.
My mail id:- mail.rakshith@gmail.com
nice article
thanks for sharing ur experience
Hi
Thanks for sharing information. I am QA in a e-publishing company. My Thought is same as above. I am agree with
this thought.
thank you so much for sharing wonderful experience
it proved to be a guideline for all those ,who are fresher in s/w testing areas like me
with regards,
tejas tare
s/w testing trainee
Hi Tejaswini,
nice article this is very true small organization do not go for documentation so what those people do who are working in small organization and hope it will help people alot. Can u please sent me some sample documents for testing.
Thanks & Regards
Kapil
Hi Tejaswini,
nice article this is very true small organization do not go for documentation so what those people do who are working in small organization and hope it will help people alot. Can u please sent me some sample documents for testing my mailid kapildevkapoor@gmail.com.
Thanks & Regards
Kapil
Hi Tejaswini,
nice article this is very true small organization do not go for documentation so what those people do who are working in small organization and hope it will help people alot. Can u please sent me some sample documents for testing my mailid
Want to know about Test Strategy? how it is different from test plan? please give me details for what test strategy documents contains possible with example?
my email id –vishu.dogra72@gmail.com
i am a software tester in small organization. i am a single person to test the applications.please send me the sample test document to my Email.
im learning testing tools,so plz send me few sample test cases & scenorios
hi sir,
I wan few sample test cases & scenarios with tips….
Hi isr,
I wan some tips for preparing manual & qtp in testing … can u send me….
Miss Tejaswini, Iam presently studying b-tech 3rd year and I am willing to get a job in the field of testing, your article is very helpful to me. you have taught me the use of documentation in every work, thank you very much for sharing your views.
hi all…
now am doing testing course in manual and automation.. may i know what r all the steps in doing manual testing projects???….. anyone know means plz tell me..
documenteation is complsory.without its u r not cover client requirement …..for covering of requirement and for thouroughly testing..documentation is complsory
hi,
how are you? thanks to you your effort.After reading your article i have got clear idea why documentation is important….
keep it up!
manual testing
I wan some tips for preparing manual & qtp in testing … can u send me….
im learning testing tools,so plz send me few sample test cases & scenorios
nice article
thanks for sharing ur experience
Its a very good artical. Thanx for sharing.
specially The artical – Why the documentation is importent in testing.
Hi Tejaswini…:)
It`s really a great article..thanks for ur efforts..
I wanna ask something, i have recently developed a project for distributed system (cluster). I want to document this project, but i dont even know about the concepts of testing and all..
So if u could help me, u can mail me at
scpby.anup@gmail.com
OR if u have any good testing documents. please send it to my id.
Thanks..
This is very good site….
Hi all
Complete tutorial about software testing
mca.siddharth@gmail.com
Contents
1. Introduction
2. Testing?
3. Software testing?
4. SDLC(Software Development Life Cycle)
5. STLC(Software testing life cycle)
6. Why Testing?
7. Who does testing?
8. When to start testing?
9. When to stop testing?
10. Type of testing
10.1 Manual Testing
10.2 Automation Testing
10.2.1 What to automate?
10.2.2 When to Automate?
10.2.3 How to Automate?
11. Testing Methods
11.1 Black Box Testing
11.2 White Box Testing
11.3 Gray Box Testing
8.1.1 Advantages
8.1.2 Disadvantages
12. Levels of Testing
12.1 Functional Testing
12.1.1 Unit Testing
12.1.2 Integration Testing
12.1.3 System Testing
12.1.4 Regression Testing
12.1.5 Acceptance Testing
12.2 Non- functional Testing
12.2.1 Performance Testing
12.2.1.1 Load Testing
12.2.1.2 Stress Testing
12.2.2 Usability Testing
12.2.3 Security Testing
12.2.4 Portability Testing
13. What is web testing?
14. Type of web testing
14.1 Functionality Testing
14.2 Usability testing
14.3 Interface testing
14.4 Database Testing
14.5 Compatibility testing
14.6 Performance testing
14.7 Security testing
14.8 Crowd Testing
15. Testing Documentation
15.1 Test Plan
15.2 Test Strategy
15.3 Test Scenario
15.4 Test Case
15.5 Test Script
15.6 Testing Environment
15.7 Test Procedure
15.8 Test Log
15.9 Traceability Matrix
16. What is Bug?
17. Bug Life Cycle
17.1 New
17.2 Open
17.3 Assign
17.4 Test
17.5 Verified
17.6 Deferred
17.7 Reopened
17.8 Duplicate
17.9 Rejected
17.10 Closed
18. Bugzilla, JIRA and Mantis
19. Bug Reporting
Testing:
Is the process of identifying defects, where a defect is any variance between actual and expected results.
“A process of analyzing a software item to detect the differences between existing and required conditions (that is defects/errors/bugs) and to evaluate the features of the software item”
DEFECT: Mismatch between the requirements, or unexpected result or wrong calculations.
BUG: Deviation from the expected result, or If something is already documented to be implemented and implementation is not consistent as per requirement then it’s a BUG.
ERRORS: When we get the wrong output i.e. syntax error, logical error, or programmatically mistake lead to error.
Failure: When everything is correct but we are not able to get result. Or When software is released, the customers found some defect or application may be crash by using any wrong functionality that is called software failure.
Software Testing
Testing with a Purpose
Software testing is performed to verify that the completed software package functions according to the expectations defined by the requirements/specifications. The overall objective to not to find every software bug that exists, but to uncover situations that could negatively impact the customer, usability and/or maintainability.
SDLC and STLC
SDLC
Requirement Phase
System Analysis
Design phase
Coding
Testing
Release
Maintenance
STLC
It involves following stages
• Preparation of the test strategy
• Preparation of the test plan
• Creation of the test environment
• Writing of the test cases
• Creation of the test scripts
• Execution of the test scripts
• Analysis of the test results
•
• Reporting of the bugs
• Performing regression testing
OR
System study
Test planning
Writing test case or script
Review test case
Executing test case
Bug tracking
Report the detect
OR
Test Planning (Test Strategy) (done by Test TL/ PM)
Test Analysis (BRS/SRS) (Test Engineer)
Test Design (Test Scenario/Test Cases/Test Data) (Test Engineer)
Test Execution (Executing test case and reporting defect) (Test Engineer)
Test Closure (Test Summary Report) (Test TL/PM/Test Engineer)
OR
Why testing is important?
Testing is about verifying that what was specified; what was delivered. In simple word software is activity to check whether the actual result match the excepted result. It is very important to understand that testing is also continuous process. Testing does not stand alone it is intimately dependent activity but it is separate process activity. Testing is the activity is the reduce the defect
Who does testing?
• Software Tester
• Software Developer
• Project Lead/Manager
• End User
Different companies have difference designations for people who test the software on the basis of their experience and knowledge such as Software Tester, Software Quality Assurance Engineer, and QA Analyst etc.
It is not possible to test the software at any time during its cycle. The next two sections state when testing should be started and when to end it during the SDLC.
When to Start Testing
An early start to testing reduces the cost, time to rework and error free software that is delivered to the client. However in Software Development Life Cycle (SDLC) testing can be started from the Requirements Gathering phase and lasts till the deployment of the software. However it also depends on the development model that is being used. For example in Water fall model formal testing is conducted in the Testing phase, but in incremental model, testing is performed at the end of every increment/iteration and at the end the whole application is tested.
Testing is done in different forms at every phase of SDLC like during Requirement gathering phase, the analysis and verifications of requirements are also considered testing. Reviewing the design in the design phase with intent to improve the design is also considered as testing. Testing performed by a developer on completion of the code is also categorized as Unit type of testing.
When to Stop Testing
Unlike when to start testing it is difficult to determine when to stop testing, as testing is a never ending process and no one can say that any software is 100% tested. Following are the aspects which should be considered to stop the testing:
• Testing Deadlines.
• Completion of test case execution.
• Completion of Functional and code coverage to a certain point.
• Bug rate falls below a certain level and no high priority bugs are identified.
• Management decision.
Testing Types
This section describes the different types of testing which may be used to test a Software
during SDLC.
1. Manual Testing
2. Automation Testing
Manual Testing:
This type includes the testing of the Software manually i.e. without using any automated tool or any script. In this type the tester takes over the role of an end user and test the Software to identify any un-expected behavior or bug. There are different stages for manual testing like unit testing, Integration testing, System testing and User Acceptance testing.
Testers use test plan, test cases or test scenarios to test the Software to ensure the completeness of testing. Manual testing also includes exploratory testing as testers explore the software to identify errors in it.
Automation Testing :
Automation testing which is also known as “Test Automation”, is when the tester writes scripts and uses another software to test the software. This process involves automation of a manual process. Automation Testing is used to re-run the test scenarios that were performed manually, quickly and repeatedly Apart from regression testing, Automation testing is also used to test the application from load, performance and stress point of view. It increases the test coverage; improve accuracy, saves time and money in comparison to manual testing.
What to automate: It is not possible to automate everything in the Software; however the areas at which user can make transactions such as login form or registration forms etc, any area where large amount of users’ can access the Software simultaneously should be automated.
When to Automate: Test Automation should be uses by considering the following for the Software:
• Large and critical projects.
• Projects that require testing the same areas frequently.
• Requirements not changing frequently.
• Accessing the application for load and performance with many virtual users.
• Stable Software with respect to manual testing.
• Availability of time.
How to Automate: Automation is done by using a supportive computer language like VB scripting and an automated software application. There are a lot of tools available which can be used to write automation scripts. Before mentioning the tools lets identify the process which can be used to automate the testing:
• Identifying areas within a software for automation.
• Selection of appropriate tool for Test automation.
• Writing Test scripts.
• Development of Test suits.
• Execution of scripts
• Create result reports.
• Identify any potential bug or performance issue.
Following are the tools which can be used for Automation testing:
• HP Quick Test Professional
• Selenium
• IBM Rational Functional Tester
• Silk Test
• Test Complete
• Testing Anywhere
• Win Runner
• Logrunner
• Visual Studio Test Professional
• WATIR
Testing Methods
1) Black Box Testing 2) White Box Testing 3) Gray Box Testing
Black Box Testing
The technique of testing without having any knowledge of the interior workings of the application is Black Box testing. The tester is oblivious to the system architecture and does not have access to the source code. Typically, when performing a black box test, a tester will interact with the system’s user interface by providing inputs and examining outputs without knowing how and where the inputs are worked upon.
Advantages:
• Well suited and efficient for large code segments.
• Code Access not required.
• Clearly separates user’s perspective from the developer’s perspective through visibly defined roles.
• Large numbers of moderately skilled testers can test the application with no knowledge of implementation, programming language or operating systems.
Disadvantages:
• Limited Coverage since only a selected number of test scenarios are actually performed.
• Inefficient testing, due to the fact that the tester only has limited knowledge about an application.
• Blind Coverage, since the tester cannot target specific code segments or error prone areas.
• The test cases are difficult to design.
White Box Testing
White box testing is the detailed investigation of internal logic and structure of the code. White box testing is also called glass testing or open box testing. In order to perform white box testing on an application, the tester needs to possess knowledge of the internal working of the code. The tester needs to have a look inside the source code and find out which unit/chunk of the code is behaving inappropriately.
Advantages:
As the tester has knowledge of the source code, it becomes very easy to find out which type of data can help in testing the application effectively.
• It helps in optimizing the code.
• Extra lines of code can be removed which can bring in hidden defects.
• Due to the tester’s knowledge about the code, maximum coverage is attained
during test scenario writing.
Disadvantages:
• Due to the fact that a skilled tester is needed to perform white box testing, the costs are increased.
• Sometimes it is impossible to look into every nook and corner to find out hidden errors that may create problems as many paths will go untested.
• It is difficult to maintain white box testing as the use of specialized tools like code analyzers and debugging tools are required.
Gray Box Testing
Grey Box testing is a technique to test the application with limited knowledge of the internal workings of an application. In software testing, the term “the more you know the better” carries a lot of weight when testing an application.
Mastering the domain of a system always gives the tester an edge over someone with limited domain knowledge. Unlike black box testing, where the tester only tests the application’s user interface, in grey box testing, the tester has access to design documents and the database. Having this knowledge, the tester is able to better prepare test data and test scenarios when making the test plan.
Advantages:
• Offers combined benefits of black box and white box testing wherever possible.
• Grey box testers don’t rely on the source code; instead they rely on interface definition and functional specifications.
• Based on the limited information available, a grey box tester can design excellent test scenarios especially around communication protocols and data type handling.
• The test is done from the point of view of the user and not the designer.
Disadvantages:
• Since the access to source code is not available, the ability to go over the code and test coverage is limited.
• The tests can be redundant if the software designer has already run a test case.
• Testing every possible input stream is unrealistic because it would take an
unreasonable amount of time; therefore, many program paths will go untested.
Levels of Testing
1) Functional Testing. 2) Non- functional Testing.
Functional Testing:
Unit Testing
Integration Testing
System Testing
Regression Testing
Acceptance Testing
Non-Functional Testing
Performance Testing
Load Testing
Stress Testing
Usability Testing
Security Testing
Portability Testing
Testing Documentation
Testing documentation involves the documentation of Software testing helps in estimating the testing effort required, test artifacts which should be developed before or during the testing of Software. Documentation for coverage, requirement tracking/tracing etc. This section includes the description of some commonly used documented artifacts related to Software testing such as:
• Test Plan
• Test Strategy
• Test Scenario
• Test Case
• Test Script
• Testing Environment
• Test Procedure
• Test Log
• Traceability Matrix
Test Plan:
Test plan is a document with information on a scope of the project, Approach, Schedule of testing activities, Resource or manpower
Test Scenario:
Test Case:
Traceability Matrix:
Traceability Matrix
The Traceability Matrix is to be able to trace from top level requirements to implementation, and from top level requirements to test. It shows the relationship between Test Requirements and Test Cases.
Using Traceability Matrix, we can check that which requirements are covered in which
Test cases and which test case covers which requirements. The same apply for the code too means which codes are covered which requirements and which requirements are covered in which section of the code too.
They are types of traceability matrix:
1. Forward traceability
2. Backward traceability
3. Bidirectional traceability
Requirement———————-?Test Case
Requirementinternet option >>General tab>>under Browsing history>>Click on setting>>there you find current location of cookies
Basically you can change your browser setting according to your need. If you want that before storing the cookies by web site it will shows prompt message or something warning so you can change the setting or Even you can disable the cookies also.
Cookie Attribute:
Cookie basically come with name-value pair, apart from that server set expiration time, path, cookie domain ,maximum age.
Now the next questing is how to test Cookie? How to do Cookie testing?
Disabling/Deleting the Cookie:
Delete the all cookie set by site which under test. One thing which is important that deleting Cookie you need to close browser so that no pre-session cookies in memory.
After that test those feature and function of the site which require the cookie ,you will come to know that those feature doesn’t work because of deleting those cookies which help to run the feature. That doesn’t mean it’s bug, it’s happened because of the disabling the cookies. It’s might possible that not every user keep the cookie enable, In that case those site require cookie need to be enabled to work the site then the site has to be send message to user that cookie need to be enable to work the site.
Reject Cookie:
You can do testing of cookie by accepting some cookies and rejecting other and check the functionality howspan style=”mso-spacerun:yes”>
iit work by rejecting cookie. For that you need to set your browser’s cookie option to prompt you whenever website attempts to set a cookie and then exercise the functionality.
Corrupt the Cookie:
For that you need to know whether the cookies are stored. Then change the cookie parameter like cookie name, expiry date, session id and all.
Check that Cookie generated by different browser:
Here the site which is under test which need to be check different browser. That different browser site generated the cookie or not .
Check the how Cookie stored sensitive Data:
It’s important that sensitive data need to be stored in encrypted format.Sensitiva data like User credit card ID,User Data etc.So even if some one open the cookies file then can not mess with the data. So that need to be check how the cookie stored the sensitive data.
Test HTML and CSS to ensure that search engines can crawl your site easily. This will include
• Checking for Syntax Errors
• Readable Color Schemas
• Standard Compliance. Ensure standards such W3C, OASIS, IETF, ISO, ECMA, or WS-I are followed.
Test business workflow- This will include
• Testing your end – to – end workflow/ business scenarios which takes the user through a series of webpage’s to complete.
• Test negative scenarios as well, such that when a user executes an unexpected step , appropriate error message or help is shown in your web application.
2) Usability testing
Usability testing has now become a vital part of any web based project. It can carried out by testers like you or a small focus group similar to the target audience of the web application.
Test the site Navigation:
• Menus , buttons or Links to different pages on your site should be easily visible and consistent on all WebPages
Test the Content:
• Content should be legible with no spelling or grammatical errors.
• Images if present should contain and “alt” text
3) Interface testing
Three areas to be tested here are – Application, Web and Database Server
• Application: Test requests are sent correctly to the Database and output at the client side is displayed correctly. Errors if any must be caught by the application and must be only shown to the administrator and not the end user.
• Web Server: Test Web server is handling all application requests without any service denial.
• Database Server: Make sure queries sent to the database give expected results.
Test system response when connection between the three layers (Application, Web and Database) can not be established and appropriate message is shown to the end user.
4) Database Testing
Database is one critical component of your web application and stress must be laid to test it thoroughly. Testing activities will include-
• Test if any errors are shown while executing queries
• Data Integrity is maintained while creating, updating or deleting data in database.
• Check response time of queries and fine tune them if necessary.
• Test data retrieved from your database is shown accurately in your web application
5) Compatibility testing
Compatibility tests ensure that your web application displays correctly across different devices. This would include-
Browser Compatibility Test:
Same website in different browsers will display differently. You need to test if your web application is being displayed correctly across browsers , JavaScript , AJAX and authentication is working fine. You may also check for Mobile Browser Compatibility.
The rendering of web elements like buttons, text fields etc changes with change in Operating System. Make sure your website works fine for various combinations of Operating systems such as Windows, Linux, Mac and Browsers such as Firefox, Internet Explorer, Safari etc.
6) Performance testing
This will ensure your site works under all loads. Testing activities will include but not limited to –
• Website application response times at different connection speeds
• Load test your web application to determine its behavior under normal and peak loads
• Stress tests your web site to determine its break point when pushed to beyond normal loads at peak time.
• Test if a crash occurs due to peak load , how does the site recover from such an event
• Make sure optimization techniques like zip compression , browser and server side cache enabled to reduce load times
7) Security testing
Security testing is vital for e-commerce website that store sensitive customer information like credit cards. Testing Activities will include-
• Test unauthorized access to secure pages should not be permitted
• Restricted files should not be downloadable without appropriate access
• Check sessions are automatically killed after prolonged user inactivity
• On use of SSL certificates , website should re-direct to encrypted SSL pages.
8) Crowd Testing
You will select a large number of people (crowd) to execute tests which otherwise would have been executed a select group of people in the company. Crowd sourced testing is an interesting and upcoming concept and helps unravel many a unnoticed defects.
BUG
Bug can be defined as the abnormal behavior of the software. No software exists without a bug. The elimination of bugs from the software depends upon the efficiency of testing done on the software. A bug is a specific concern about the quality of the Application under Test (AUT)
Bug Life Cycle:
In software development process, the bug has a life cycle. The bug should go through the life cycle to be closed. A specific life cycle ensures that the process is standardized. The bug attains different states in the life cycle. The life cycle of the bug can be shown diagrammatically as follows:
The different states of a bug can be summarized as follows:
1. New
2. Open
3. Assign
4. Test
5. Verified
6. Deferred
7. Reopened
8. Duplicate
9. Rejected and
10. Closed
Description of Various Stages:
1. New: When the bug is posted for the first time, its state will be “NEW”. This means that the bug is not yet approved.
2. Open: After a tester has posted a bug, the lead of the tester approves that the bug is genuine and he changes the state as “OPEN”.
3. Assign: Once the lead changes the state as “OPEN”, he assigns the bug to corresponding developer or developer team. The state of the bug now is changed to “ASSIGN”.
4. Test: Once the developer fixes the bug, he has to assign the bug to the testing team for next round of testing. Before he releases the software with bug fixed, he changes the state of bug to “TEST”. It specifies that the bug has been fixed and is released to testing team.
5. Deferred: The bug, changed to deferred state means the bug is expected to be fixed in next releases. The reasons for changing the bug to this state have many factors. Some of them are priority of the bug may be low, lack of time for the release or the bug may not have major effect on the software.
6. Rejected: If the developer feels that the bug is not genuine, he rejects the bug. Then the state of the bug is changed to “REJECTED”.
7. Duplicate: If the bug is repeated twice or the two bugs mention the same concept of the bug, then one bug status is changed to “DUPLICATE”.
8. Verified: Once the bug is fixed and the status is changed to “TEST”, the tester tests the bug. If the bug is not present in the software, he approves that the bug is fixed and changes the status to “VERIFIED”.
9. Reopened: If the bug still exists even after the bug is fixed by the developer, the tester changes the status to “REOPENED”. The bug traverses the life cycle once again.
10. Closed: Once the bug is fixed, it is tested by the tester. If the tester feels that the bug no longer exists in the software, he changes the status of the bug to “CLOSED”. This state means that the bug is fixed, tested and approved.
Realyyyy very very helpful,,,,,,,,,,,,,,,
Hi Siddharth,.
Its very great by providing the article on software testing.
Its very helpful.
Can any one help in providing in test scenarios / cases for work flow management (e-mail tracking, issuer etc).
My mail id is sreelud9@gmail.com
Thanks a lot
Sree
.i learnt some thing from ur experience and used to know importance of documentation. Recently I have started my testing carrer.keep posting like this articles.it was very helpful
Hi mam,
i m working as a software tester in a com.can you pls tell me more about test plan ,test execution?
i ‘ve got a new project in php.There is many field to add a user and patient.
Shall i test each field to prepare the test plan?
Nice article, thanks
Hi Tejaswini,
It’s a great experience to read this article. I havn’t
imagine that Test Doc. has so much importance.
Your experience proved a saying that “If it’s not written down, it doesn’t exist”.Thanks a lot.
Hi Tejaswini,
Good article many times our team faced this issue. I also faced the same issue in my project. But learnt to document all the things what I work at least in notepad. Will be waiting for 2nd article
@ Siddharth Shukla
Thank you for sharing.
still have a confusion on Defect/Bug, Failure/Error
For Defect you have given the “unexpected result” For Bug “Deviation from the expected result” i think these two meaning are same right. can please give me some examples
Thank u for sharing Sidharth.
Coud u please help me in the Question,
How do u produce testing documents?
What is the best answer for it.
Please let me know if there are any openings for Manual Testing. I have experiance of 1 year 9 Months.
My mail id:- mail.rakshith@gmail.com
Very Helpful Article…we surely make a practice of it and will surely inform my Software Testing Colleagues regarding this Article….:)
Nice work!
You narrated very well about the importance of documentation. But, about the situtaion you had with your angry client, I don’t think that the proof of requirements document saved you (or QA team) from getting blamed. Almost every process oriented 0rganization maintains designing the requirements document. In your case, instead of searching through all the requirements documents, the best practice could be looking at Traceability Matrix document, through which you could easily identify if there is any such requirement from the customer and if so, have we addressed it or not.
I also feel that, requirements document is not a work product of tester’s effort. However, tester can verify it for the correctness of same.
As I said earlier, you did a great job on posting this article. I also agree that most of them in the team have an impression that the documentation is a free time work. To vanish this myth, tester should be more creative to prepare effective documentation. To derive effective scenarios they should speak to SMEs/BAs/Sr.Developers to understand the customers business process well. In our agile process, we use user stories in preparing test cases.
Once again, thanks for sharing the info and keep up the good work.
Best Regards,
Asha
very nice article…and needed information for everyone who is looking forward to explore their carrier in testing field
hi everyone my name is sumanth iam looking for a job in testing field i have experience of 1.5 years please if u knw any… let me knw…
thanking u
mail id:sumanthpathi2016@gmail.com
hi everyone my name is sumanth iam looking for a job in testing field i have experience of 1.5 years please if u knw any… let me knw…
thanking u
mail id:sumanthpathi2016@gmail.com
can you please guide me how to create a test plan on client requirement doc
It is good and informative.
Very useful with key pints indeed a Good read.
Thank you for sharing.
great information on Software Testing Documentation Guide