What To Do When There Isn’t Enough Time To Test?

Part way through your test cycle, do you often realize you do not have enough time to test? You had it all under control, to begin with, but soon you are reaching the contingency plan’s “What to do when there isn’t enough time to test?” section.

I have been there too and it is not fun. :)

I thought about this long and hard. How can something that started so well, go down so badly, so quickly. And, here is my analysis. 

Where Did My Testing Time Go?

Not Enough Time To Test

Firstly, Why does this happen? Many reasons – some of which are:

#1) Incorrect Estimation:

If you started with an inaccurate expectation, things are bound to fail. A good test estimate must take the following into account:

  • Time for preparatory tasks – We are talking about tasks such as:
    • Identifying and putting together a regression suite
    • Creating Test data
    • Time to determine test readiness (E.g.: Smoke/Sanity Test), etc.
  • Test case maintenance: Test cases are long-term usage assets. They are sure to undergo minor updates during execution. It is recommended that for new products up to 30% of your test execution time should be allocated for these minor maintenance tasks. All teams and projects might not need 30%, but do allocate some time and effort for this task.
  • Ad-hoc/Exploratory testing – The count of scripted tests is a major denominator for test estimation numbers. However, no test team in this world will deny exploring your software even if the model is dominantly scripted.
  • Reporting/Communication – This includes triage/stand up meetings, updating work management tools etc.
  • Contingency factor: Standards recommend 25-30% buffer to your original estimates. But teams can rarely afford it. Even then, leave a little breathing room, when possible.
  • Team and its capabilities: If you have a new team or if they are using a tool for the first time, you might need to set some time aside for training. Tailor your estimates based on your team you are working with.

Recommended read => Check this for more information on test estimation success and methods

#2) Unstable builds and other technical problems:

  • Smoke/Sanity test failure: When the basic tests on the AUT fail after deployment into QA environment there is pretty much nothing the QA team can do towards test execution. It is true that we can work on other tasks while this happens, but it still will not fill the test cycle time. So, this is a major contributor to time wasted.
  • Test data unavailable: Production-like data is a must for every testing project. Not getting this into the QA environment on time is also another blocking factor. Sometimes testers can work around this by creating and managing their own test data, but it is time-consuming and might not always be on-point.
  • Environment issues – The build failing deployments, the server keeps getting timed out, many more such issues eat away your test cycle. This probably stems from the fact that, some companies (not all) undermine the importance of a good, live-like environment for effective QA. They often try to get away low-capacity servers and make-do set ups. This is really a short-time fix and does nobody any favors. In fact, it could cost them the quality of testing and loss of valuable test time.

#3) Lack of agreement between all parties involved:


This might be a rare problem with teams following Agile or SAFe due to the close circles they work in, but many teams still suffer from disagreement or miscommunication as to when Dev, Ops, and QA is supposed to receive deliverables from one another. Hence, delays.

To understand communication subtleties, check this => How Business, Development and QA Can Work Together to Get the Project Completed

Now that we know the problems, here are some ways to fix it.

How can testers get enough time for testing?

#1) Estimate accurately. When in doubt over-estimate by a reasonable margin, but not underestimate. Don’t forget to make estimate adjustments based on your team, tools and processes. When done, seek official sign off so everyone is aware and is in kept in the loop.

#2) Take historical data into consideration – The Test Management tool is your best friend.

  • How long did the earlier release test cycles take?
  • What kind of issues caused interruptions to the previous test cycle?
  • How many runs did most test cases take before they passed?
  • What defects were reported?
  • What defects caused the testing to be interrupted?

#3) Ask these questions and plan accordingly in crunch time:

  • Find out Important functionality is your project?
  • Find out High-risk module of the project?
  • Which functionality is most visible to the user?
  • Which functionality has the largest safety impact?
  • Which functionality has the largest financial impact on users?
  • Which aspects of the application are most important to the customer?
  • Which parts of the code are most complex, and thus most subject to errors?
  • Which parts of the application were developed in rush or panic mode?
  • What do the developers think are the highest-risk aspects of the application?
  • What kinds of problems would cause the worst publicity?
  • What kinds of problems would cause the most customer service complaints?
  • What kinds of tests could easily cover multiple functionalities?

Considering these points, you can greatly reduce the risk of project releasing under less time constraint.

#4) Use a Test Management tool. This will significantly reduce the amount of preparation, reporting and maintenance time and effort.

=> For the list of the most popular test management tool choice, check out here:

#5) There is not much we can do about incorrect builds/technical issues, but the one thing that can help is looking at the Unit test results. This will give us an idea as to whether the build was a success or not and what kind of tests did it fail – so we don’t reinvent the wheel.

If your Test Management Tool supports CI integration, you have that information available without any fuss so you understand the stability of the application better.

#6) Measure your productivity and progress often. Don’t let status reports be a deliverable just for the benefit of the external teams. Make sure you are closely monitoring your daily targets and your ability to accomplish them.

Also, be sure to not get into the classic conundrum of ‘Velocity vs. Quality’. Because, when you report, say, 50 bugs a day, it might appear as if you are being super productive. But if most of them are coming back as invalid ones, you have got yourself a problem.

So monitor, monitor and monitor a little more :)

Conclusion:

Finally, despite all the precautions and measures if you still find yourself crunched for time, ask help.

Most teams are willing to participate in a war room session to get things back on track.

About the author: These helpful testing tips are provided by STH team member Swati S.

Now, what are your tricks to stay on time and deliver a quality testing service? Also, what points in the above article resonate with you?

We appreciate your feedback and cherish your readership. Thank you for reading!




Recommended reading

62 comments ↓

#1 rakesh

thanks for the tips

#2 rakesh

What kinds of problems would cause the worst publicity?
i dint get it , can u tell me in brief please

#3 Raja

hi,
can u give me a sample use case and test case.
Same as template also ok…….

#4 Tanvi

I was asked in an interview, ”When do u stop testing?”
What should be a probable answer to that?

#5 Shahanaz R

I was asked for an interview on What all types of Test Estimation models are there and to describe each one?

#6 Shahanaz R

I was even asked What all types of Risks are there in Software Testing?

#7 Shahanaz R

Suppose i have a project when clicking English tab, all are displayed in English- inclusive of date and currency(Eg:- dd/mm/yy and $) and When clicking French all are in French, ie date and currency becomes (Eg:- mm/dd/yy and Euro). Can you please tell me what is Localization testing and Internationalization testing in the above case?

When a project is made purely in French? Is Localization testing done there?

#8 sagar

Can u please post me test case sample of Localisation testing , How to conduct the localisation testing?

#9 divya

Can you please tell me the tools which are used for Internationlization testing and risks involved.

#10 Sonia

Hi,

I have to start writing manual test cases . I am not sure if manual test cases shoul dbe written for fixed data or we should use parameterization. I would also like to know if it is a good practice to make small test cases and combine them into a test set or make end to end test cases

#11 Raja

Hi,

If we using functional test case then we use fixed data and if necessary in concentrate unit test case then we go for parameterization.
in automation it is must to go for parameterization and data driven.

#12 Sonia

Hi ,

Thanks. Another question , how can I structure the tests so that we can re use the cases even if the form changes for eg , for a web site that does the following :
Login , collect information about a persons( personal, credit card , company , team , event , misc information),
should there be only one end to end test or many small tests for each related group of information .

Again the combination of data against the entire end to end process . how will that work ?

Thanks

#13 Raja

Hi,
Now,Collecting information through web page is done by validation itself(BY program),so v have 2 check whether the system response correctly or not.In this case functional testing(end to end testing) is more than enough.
But, the same case login page followed by shopping page,where the same data required in that situation unit test case (individual module) should be tested.because the data has to be validate against the bank details and process him to next steps.so,we concentrate more on this page.
The same process but differ from application to application,so based on the functionality of the project v have to choose the test cases.
Question 2:
The Combination of data can be modified to use for functional test case.if u want to learn more than practice with two small application having same functions.

#14 Raja murugesh

Hi,
In testing anything is possible.Before writing end to end or many small test cases,understand the flow of application because with minimum input v have to obtain maximum output.
Now,Collecting information through web page is done by validation itself(BY program),so v have 2 check whether the system response correctly or not.In this case functional testing(end to end testing) is more than enough.
But, the same case login page followed by shopping page,where the same data required in that situation unit test case (individual module) should be tested.because the data has to be validate against the bank details and process him to next steps.so,we concentrate more on this page.
The same process but differ from application to application,so based on the functionality of the project v have to choose the test cases.
Question 2:
The Combination of data can be modified to use for functional test case.if u want to learn more than practice with two small application having same functions.

Regards,
Raja

#15 anish

Too good. Thank for the tips

#16 anish

Thax for tip

#17 Suresh Balakrishnan

Hi Shahanaz R,

This is Suresh Working as test Engineer i have mentioned Types of Risks as below:
1.Software Risk
2.business Risk
3.Schedule Risk
4.Teechnical Risk
5.Programatic Risk
6.budget Risk

Thanks
Suresh
bsuresh7282@gmail.com

#18 Suresh Balakrishnan

Hi Tanvi,
I have mentioned Reasons for to stop testing as below:

1.Try continuing testing and logging the critical defects
2.all quality goals defined at the start of the project have been met.
3.After complete Alpha and Beta Testing
4.Successful use of specific test case design methodologies
5.A percentage of coverage for each coverage category
6.Rate of error detection falls below a specified threshold.
Thanks,
Suresh
bsuresh7282@gmail.com

#19 Sonia

Hi Suresh,

You have mentioned the different types of risks, would you please explain each risk. Alternatively could you suggest a book on Risk Testing .

Thanks

#20 Suresh Balakrishnan

Hi Sonia,

I have Mentioned Types of Risks with Explanations as below:
Risk Identification
1.Software Risks – Knowledge of the most common risks associated with software development, and the platform you are working on.

2.Testing Risks – Knowledge of the most common risks associated with software testing for the platform you are working on, tools being used, and test methods being applied.

3.Premature Release Risk – Ability to determine the risk associated with releasing unsatisfactory or untested software products.

4.Business Risks – Most common risks associated with the business using the software.

5.Risk Methods – Strategies and approaches for identifying risks or problems associated with implementing and operating information technology, products, and processes; assessing their likelihood, and initiating strategies to test for those risks.

Managing Risks
6.Risk Magnitude – Ability to rank the severity of a risk categorically or quantitatively.

7.Risk Reduction Methods – The strategies and approaches that can be used to minimize the magnitude of a risk.

8.Contingency Planning – Plans to reduce the magnitude of a known risk should the risk event occur.
Thanks,
Suresh
bsuresh7282@gmail.com

#21 Sonia

Thank you for the valuable information Suresh

#22 Hari

Hi,

The article was good. Could you pls send me some testing resume format? Also some ideas about use case…

#23 parul

hi…………………..i wanted to know about test cases with example

#24 Raja

Hi Parul,

Below is the sample test case

Project Form UIN:
System Name: Module Name:
Test Sub-Module :

Tester Name Signature Date/Time
TM Rajamurugesh

Test Procedures:
Hardware: NA
Software: NA
Communications: NA
Database:
Server Configuration:
Client Configuration: NA
Test Tools/Techniques/Methodologies: Manual
Test Location:
Dev Environment : Dev URL
Test Environment : test URL

Test Cases & Reports:
SNo-Objective-Test Case- Navigation-Inputs-ExpectedOutputs-Test Result (Pass/Fail)

#25 Raja

SNo-Objective-Test Case- Navigation-Inputs-ExpectedOutputs-Test Result (Pass/Fail)

1-aim-steps/procedure-link/event-data-what we except-pass/Fail

#26 SHIVALI

May i know when to stop testing? Can i get some testing resumes?

Thank you

#27 Raja

Hi SHIVALI ,

Send me a test mail, i will reply to that mail with testing resume, test cases and some testing white papers.

My mail id:
rajamurugesh.b@gmail.com

#28 shivali

Can you plz tell me how to prepare and clear ISTQB foundation level paper?

#29 Jagdish R

Hi SHivali….

When to stop testing?depends on the below:

1. When all the test cases are exected successfully
2. When the budjet allocation is over
3. When the time/scheduled for testing is met
4.When the delivery date is met to deliver the application

#30 shivali

Can you please tell me if i can get some projects of software testing which i can do at home?

#31 G.Mallikarjun

Shivali, plz read this article to get ur answer.
Use risk analysis to determine where testing should be focused.
Since it’s rarely possible to test every possible aspect of an application, every possible combination of events, every dependency, or everything that could go wrong, risk analysis is appropriate to most software development projects. This requires judgement skills, common sense, and experience. (If warranted, formal methods are also available.) Considerations can include:
• Which functionality is most important to the project’s intended purpose?
• Which functionality is most visible to the user?
• Which functionality has the largest safety impact?
• Which functionality has the largest financial impact on users?
• Which aspects of the application are most important to the customer?
• Which aspects of the application can be tested early in the development cycle?
• Which parts of the code are most complex, and thus most subject to errors?
• Which parts of the application were developed in rush or panic mode?
• Which aspects of similar/related previous projects caused problems?
• Which aspects of similar/related previous projects had large maintenance expenses?
• Which parts of the requirements and design are unclear or poorly thought out?
• What do the developers think are the highest-risk aspects of the application?
• What kinds of problems would cause the worst publicity?
• What kinds of problems would cause the most customer service complaints?
• What kinds of tests could easily cover multiple functionalities?
• Which tests will have the best high-risk-coverage to time-required ratio?

#32 Indu Sharma

Its good ..i enjoyed reading this. I will try to incorporate this in my testing….:)

#33 krishna

can you please tell me and send the documents on how to calculate test estimation

#34 Indu Sharma

Effort estimation is very simple:

>>1st u have to find out all the possible scenarios in all features.
>>Then Make a traceability matrix i.e. for each requirements how many test cases are required.
>>Suppose if u have 100 testcases, then calculate how many test cases one person can execute in a day(here u have to consider difficult and easy both types of test cases)
>>If one person can do 5 test cases in a day then in 20 days all test can be executed.i.e effort=20PD or 4PW.
>> Now u have to add following facters:
1. What if 30% Testcases failed in Run1, then u have to rerun all those test cases. So u have to consider 35 testcases failure i.e 7PD or 1.4 PW
2. What if say 10% testcases failed in Run2 then i have to re-run 10 testcases again. so its 2PD or 0.4 PW

Now, Total effort= 4PW+1.4PW+0.4PW=5.8PW(Person week)

This is just a example i gave… u can add if there are any opther variants which impact ur execution.

regards,
Indu

#35 mayuri

HI!!!!
I am looking job in QA .I am Brainbench certified,but doesnt have real time experience.I saw and study your post.Thanks for such important information.
I want to know within 1 project how many test cases we have to write?
Simply,If I want to justify my 1 year of experience in HOME INSURANCE company on 1 project,so how can I put points,or justify my experience?
I will have to give some scenario or anything i am damn confuse.Can you please help me out by giving any example in HOME INSURANCE application.
Thank you!!!
mail id-mayuri.s2010@gmail.com

#36 pavan kumar

can you tell me what’s the meant of elapsed time and it’s related topics

#37 Sandy

Our company is on an On Demand Application and has not control over fixes (service packs) and releases. For Service Packs the vendor actually loads our test AND production systems and we have literally no time to test those beofre our European Users sign on. Even if we find a bug, we can only submit an SR as it is already in Production. A Servcie Pack is typically 30 to 60 bug fixes that are usually minor. While we rarley, never? have catastrophic failures and SP will often cause minor problems. We have talked about doing “regression testing” for this before we test the actual changes that impact our system. It seems to me that is overkill and Regression should be done after we test the specific changes. Perhaps we need some kind of Smoke / Build Verifcation Testing as soon as we get the Servcie Pack but again, we are not sure to what level detail. Perhaps only test things that we have actually changed from the vanilla system?

Your thoughts

#38 Monalisa

Its really interesting..
and helpful

#39 Gayathri

Hi Vijay,
Please answer below question
What if all features/functionalities are critical and we don’t have enough time to test them thoroughly. Can we say we are confident to go with release and it will be bug free.

Thanks

#40 Deepa

How do we ensure if a requirement has been tested thoroughly, covering all possible scenarios? Do we have any tools?

#41 Ranjith

Hi Team,

I got a different question in robert boash

1) how will you say the test scenarios are enough to validate the functionality
2) which technique are used for preparing for test scenarios.
3) if your team leader said praparied test scenarios are not enough , then he said you how to prepare some moore scenarios then what will you do.

#42 Prajna Priyadarsini

HI,

can you help me with the resume tips ?

#43 Sampath Gsj

Sooooper

#44 naren

I am not able answer QA questions for interview.Please offer some help.

#45 sowmya

hi,,
I need examples on high,medium and low severity,
other then company logo

#46 Pratik

Dear Sir,
I have some questions which had been asked me in the interview and I answered but they said I am wrong, so it would be very helpful for all of us to get an idea about these kind of queries..
The questions were:

1. If I give you a Login page which has fields like Username, Password, Login and Cancel button.. How would you start writing test cases..(I explained I will start with Smoke testing i.e. with Valid username and valid password, but they said NO, you will first check it with UI testing like Alignment, Page color,etc..)
So sir, is it really like that? Please answer

2. This room is constructed with the combined work of a contractor, carpainter,plumber,etc that means they are the developers and you are a tester… Find the defects in this room.
Sir I am really waiting to know your answers…
Thank you in advance

#47 Vijay

@rakesh – here is an example of the problem that would cause the worst publicity:
“Bill Gates crashes Windows 98 during live demo” :)
https://www.youtube.com/watch?v=yHxj-47csUU

#48 Vijay

@Hari – we have many articles on how to prepare QA testing resume with sample resume format as well. Check these at:
http://www.softwaretestinghelp.com/category/software-testing-resume/

#49 Ancy jose

good tips

#50 Samuel Garratt

For problems such as Smoke/Sanity test failure, could you say service virtualisation could be used to save time? If a certain aspect is not available you can set up an automated tests that run against a virtual back end (based on expected behaviour) and then switch their destination to the real system once it’s online.

#51 Steven Demetrius

For those people needing to perform temporal testing (time travel testing) where you are considering changing the system clock forward to back in time to validate / verify applications, you should consider using a tool that makes temporal testing easy and efficient.

#52 Swati Seela

@ Samuel Garratt: Sure, for early testing, virtualization is great.

#53 Poonkavithai.K

Hi Swati S,

This article is great. You have put the points precisely.
In the section “#1) Incorrect Estimation:” we can add one more point on Test Prioritization.
Improper Test Prioritization would lead to wastage of effort on irrelevant test case and missing of business critical issues

In section #2) Unstable builds and other technical problems: we can add points on
Improper Release Document and Misunderstanding/unavailable upstream and/or downstream applications.

Regards
Poonkavithai.K

#54 Sowjanya

Hi Friends,

Can u please send me some sample insurance projects where i can explain in the interviews.I even want some sample resumes if u can do so.

Thank You,
Sowjanya

#55 Swati Seela

@Poonkavithai.K: Absolutely! Thank you for adding your contribution to this article.

#56 Varun Dutta

One way to meet our release date is by working for long hours and sometimes even working on weekends.This is what I am doing in my current company.And I guess this is the case with most companies based out of India.Working late and on weekends is a culture that has been nurtured in India and I don’t see it changing anytime soon.

#57 Mayur Patil

Sir in last interview interviewer asked me one question. If you are working with 3 team members if they are going to leave and there 3000 test cases then how you handle that?

#58 mahesh

Hi i’m mahesh,I’m B-Tech student 2013 pass out.i have done my testing course.I don’t have any working experience i’m still in search of job..if any openings are there please let me know…and as i’m 2013 passout should i try by putting exp or i can try as a fresher..please help me out… mail id;maheshsaky9@gmail.com

#59 w3softech

hii..
You are providing wonderful tips,when Isn’t have time,It is very useful to who is looking for testing side,You given very useful tips,Thank you for this great blog.

#60 sweety

Hii..
You are providing wonderful information about software testing,I got good clarification on testing,These tips are very useful to every software tesing side people,Thank you for sharing this blog.

#61 Ashish

Hi,

We didn’t receive answer of main question here.. what shall we do if we don’t have time?

#62 Dinesh Jain

I have a situation based question:
Suppose your team can execute 10 tests/day, you have 2 days time to go and the number of tests to be executed are 35. How will you ensure that you will cover all 35 test combination within the remaining days.

Assume that it’s a web application which need to be tested on certain combination of OS and Browser.

Certain not so good things to note:
1. You are testing this application for the first time. So no historical data available to take some wise decision by taking some risks.
2. The test combinations (35) have come after applying Risk Based Technique.
3. All are P1
4. No automation in place
5. You do not have option to ask your team to stretch
6. No possibility of getting any additional tester to help

What are the test execution techniques to be applied to have un-compromised test coverage and stringent timeline?

Leave a Comment