How Test Planning has Upper Hand than Test Execution Phase?


Software test planning reserves far better scope comparatively in STLC phase. The delivery of quality software is ensured by the testing phase. What has to be done in testing is actually decided in the test planning phase.

What you will learn in this tutorial?

This tutorial will provide overview and illustrations on the importance of the test planning and the execution phase. After reading this tutorial you will understand the significant importance of the planning phase compared to the execution phase with more live examples and case studies for illustrations.

test planning and execution

Test Planning:

Essential things to be noted while Test Planning

Planning the test is a core important section in testing cycle. Outcome of the testing phase will be determined by the quality and the scope of the planning that has been done for the testing.

Test Planning 1

Planning the test usually occurs during the time of development phase in order to save lead time for test execution upon mutual agreement from all parties involved.

There are few things to be noted much importantly as follows:

  • Test Planning must be started in parallel to development provided the requirements have been frozen.
  • All the stake holders like designers, developers, clients and testers needed to be involved while finalizing the test planning.
  • Test planning cannot be worked out for unconfirmed or any unapproved business needs.
  • Similar test plans will be applied to the new requirements business will require.

Example #1

Development team is working on software XYZ after getting few requirements from the clients. Almost the testing team have started their preparation for the test defining or planning phase. The test planning has to be designed to address the initial requirements quoted by the clients. This has been done by the testing team. Neither of the other stakeholders was involved during this phase and the planning has been frozen.

Development team now have done some changes in the business flow in order to address few issues in their work with clients ‘approval. Now the software has come to the testing team for testing. Having the test plan as per the old business flow, the testing team have started their round of testing. This impacted the testing deliverables with much delay as the modified business flow was not shared to the testing team.

Observation from example 1:

There are some observations or few essential aspects during the test planning from the above example. They are:

  • Understanding the new business flow consumed time
  • Delays in project deliverables
  • Reworking on test planning and other tasks in the phase

All these observations have to be converted into essential needs for an effective testing deliverable.

Few other major components in a test planning are as follows:

Test strategy – This is one of the most important section that can explain the strategy that will be used while testing

Test coverage – This is essentially required which will do conformance mapping of the business needs and the test cases so that one can ensure the entire software has been tested.

Test cycles and durations – This can become very critical depending upon the rounds of development and their time for completing each round.

Pass/fail criteria – Much needed one where the pass and fail criteria is defined. Fewer times this will also be defined by the clients.

Business and technical requirements – Need to have the software and the purposes they will serve will be clearly defined along with the low level explanations.

Limitations

There are few things that can actually control the software testing phase especially the planning phase.

Following are such few areas:

  • Features to be and not to be tested – This will clearly point out what has to be tested and what should not be.
  • Suspension criteria and resumption requirements – This is decision maker on the software developed and criteria defined to suspend the testing or to resume the testing.
  • Responsibilities – The tester will have multiple responsibilities in ensuring the issues, bugs and defect in the software under test. Additionally the bugs have to be validated with the developers for them to fix.
  • Risks and contingencies – Risks associated during the testing should be clearly mentioned and proper contingencies during the time have to be defined very clearly.

Test Planning 2

Case study #1

The development team from the example #1 is planning to release the software xyz in 2 phases. Phase 1 has got many features to be tested and few not to be tested. Again not keeping the testing team informed about the features yet to be developed, the software has been released to test. Now the testing team starts their execution based on the test plans they have worked out. They come up with large number of bugs. After validation from the development team most of them goes invalid.

The observations from the above case study:

------------

  • Development team to release the software to testing team with release notes and requirements coverage notes (release notes)
  • Features to be tested and not to be tested have to be factored based on the released software before testing.
  • Suspension and resumption criteria for the testing have to be defined properly.
  • Risk and the contingency plans for the unavailability of the software have to be pictured perfectly.

Also read => How to Manage Risks During Test Planning Phase

Test execution

Execution of the test cases is one of the steps in the STLC phase. This will have to perform in accordance to the test plans worked out earlier. Hence the test planning always keeps dominating the whole of the testing phase. Below is an example where the testing team gets impacted for the changes in test plans.

Example #2

Testing the software A was started based on the plan 1 worked out by the team. Later due to the business needs and the changes the test plan has to undergo some changes. This in turn has forced the test cases or the execution to be changed.

Observations:

  • Test plan will determine the test cases execution
  • Execution part varies as per the plan.
  • As long as the plan and the requirements are valid the test cases are valid.

Test execution 1

How to overcome problems while execution

Testers will more often come across various scenarios while they perform the test execution. This is when the testers will have to understand and know the ways to resolve the problem or to at least find a work around for the issue.

Example #3

During the test case execution of the software B, the testing team comes across multiple issues. Few of them are show stoppers. They require the developers in helping to overcome the issue. This has happened several times and the outcome of it is delay in testing deliverables.

Observations:

  • There is dependency for overcoming environmental problems and issues.
  • Proper understanding of environment is required for testers.
  • Frequently occurring and known issues have to be documented for overcoming in the future.

Version controlling and management

Version Controlling and management of test plans and the test cases are really important to show case the timely deliverables. This is being more significant and often done with the help of a version controlling tool. The version control tool not only helps them control the test plans but also the defects management. When there are testing projects with multiple cycles and releases, these tools can really help a lot in bringing down metrics for supporting the testing deliverables.

Also read => Risk Management at Test Execution Phase

Test Planning vs Test Execution:

Following are the few important areas which will point on how test planning will differ from Test execution phase.

Comparison areaTest PlanningTest Execution
Person responsibleThe test manager will be preparing the Test plan and will be sharing to all the stake holders for their review.This will be normally done by tester keeping in mind that the test cases prepared has been approved and signed off.
Main focusThe Test plan focus areas are how the testing should be carried out, what should be considered and what not to, environment that can be used, Test schedules etc.The Test execution focuses mainly on the execution of the test cases provided to be tested on the software.
Recurring or iterative modeThis is a single time activity. Having said that it may or may not require modifications for the future releases of the software.There are 3 parts in this area when we talk about iteration.
1. Functional testing.
2. Regression testing.
3. Re-testing.
InputsThe inputs for creation of a test plan is really required and to be provided by business analysts, Architect, clients etc.,The test case document is the major input.
Period when it can be startedIt has to be started along with the development cycle for effective outcome and to save time. But there are few models like water fall model where in the testing phase will start only after the development phase has been completed.Execution has to be started strictly after the development of the software has been done.
Closure periodThe test plan will have no such closure period. Generally a sign off from all interested parties for the software will be provided.Execution for a specific release or cycle will be considered to be closed when all of the test cases have been executed against the software.
Deliverable positioningTest plan will be considered as a major deliverable for the testing activity. This will be done as the first step in testing process.This will be coming as a last bench member in the testing phase. Post execution the defects/bugs status along with the test case execution status will be shared as one of the testing deliverables
Tools usageThere will not be many tools used as the planning activity will be more of discussion and documentation. To keep track of any changes to the plan, the test managers will normally use any version control tool like VSS or something else.It will depend on the mode of execution. In case of manual no tool will be used for execution. But for logging the defects and managing, some tools will be used.
In case of automation testing, the execution will be done with the help of tools like QTP, SELENIUM etc.
Impacts on the deliverablesThis will impact all of the testing phases in a larger mannerThis will impact the subsequent cycle or release to be tested.

Conclusion

The above illustrations might have explained in a better shape over the importance of the test planning activities than the test execution. By some means the execution phase is kind of subset of the test plan.

Based on the test strategy, approach and other things the test plan have higher probability of getting modified to give room to the changes. It is a definite thing that Test execution depends on the test cases. Test cases are based upon the plans. Hence changes in plans will ensure changes in the test cases. But conversely changes in test cases need not mandatorily look for changes.

This is one of the main reasons the test planning keeps up compared to the test execution phase.

Abut the author: This is a guest post by Nagarajan. He is working as test lead with over 6 years of Testing experience in various functional areas like Banking, Airlines, Telecom in terms of both manual and automation.


The Best Software Testing Training You'll Ever Get!

software testing QA training

11 comments ↓

#1 Krishna bh on 07.17.14 at 12:59 pm

test planning is important as in general it goes like – plan properly and it will help for execution!

#2 Claudiu Draghia on 07.17.14 at 1:07 pm

A lot of interesting information. Especially the examples. There is however one thing I have to disagree with: “The delivery of quality software is ensured by the testing phase. ” Quality is not something you add like salt at the end. The quality of the software is verified by the testing phase in some aspects but quality as whole has many attributes (eg. the comments in the code are one quality attribute that usually testing does not check; this is a valid attribute for maintainability; just as a example)

#3 Bianca Tanase on 07.17.14 at 9:13 pm

Yes, quality is not something you add, but something you check during and at the end of development to ensure that the product is in line with the specifications aproved by the client.

#4 Bianca Tanase on 07.17.14 at 9:15 pm

Nice article guys…. keep up the good work

#5 Nagarajan on 07.18.14 at 8:52 am

@Krishna bh/Claudiu Draghia/Bianca Tanase,

Thanks for the comments.

@Claudiu Draghia,

Quality of software is ensured by the testing phase in this context refers to the whole of the testing cycle. Whether or not the Software has a better quality, the softwaretesting phase will pick them and identify the areas needed to be monitored/fixed/resolved/corrected.

On the whole the point you have mentioned here has been interpreted in a different manner :)

feel free to put on more questions!!!

-Nagarajan

#6 Pankaj on 07.20.14 at 12:42 pm

Nice Article.I feel the test execution is part of test planing i.e. when exeution will start and end. The points mentioned in above article are fit only when we have a standard process in place or SDLC model in place.

Correct me if I wrong guys :)

Cheers,
Pankaj

#7 Nagarajan on 07.21.14 at 3:52 pm

@ Pankaj,

You are absolutely right. When we have 2 entities Test planning and execution, there must be a proper Software model in place. Else the facts provided above will become un feasible.

-Nagarajan

#8 karuvarasan on 07.24.14 at 11:07 am

Thanks nagarajan for this useful article

#9 swati on 08.01.14 at 7:03 am

@ Vijay

Do we require design documents for the preparation of test cases?

#10 Nagarajan on 08.01.14 at 4:33 pm

@Swati,

Design document will be hardly useful for Testers. This is because, it will have high level flow of data across the business architecture. Hence there is another deliverable called “System Requirement specification document” or “Low Level Design document” that can explain at very low level that can be understood by the testers to come up with the test plans and other testing deliverables.

-Nagarajan

#11 Tarun sharma on 09.18.14 at 6:05 pm

How can I make my test plan more effective without documentation of mobile android testing??

Leave a Comment