Test Execution – How to Deal with this Most Important and Happening Phase in the STLC – QA Training on a Live Project Day 5

Today, in our software testing mini training course, we are progressing into the last stage of the STLC, which is the test execution.

You can check out the list of all tutorials posted in this free QA training series on this page: End to End software testing training on a live project.

Test execution is without doubt the most important and ‘happening’ phase in the STLC and also the entire development life cycle. The reason is – every team/team member’s contribution and work gets validated here:

  • Has the Business Analyst interpreted the requirements correctly?
  • Has the development team translated the business requirements to functional requirement and eventually to code correctly?
  • Has the data architect and DBAs design the right back end systems?

Well, test execution is where all the answers to these questions would be found. That does make us, QAs the heroes of the entire software building process, doesn’t it? :)

test execution STLC

Test Execution is also the “Test” part of the SDLC.

Once the test cases are written, shared with the BAs and Dev team, reviewed by them, changes are notified to the QA team (if any), QA team makes necessary amends- Test design phase is complete. Now getting the Test cases ready does not mean we can initiate the test run. We need to have the application ready as well among other things.

Test Execution Guidelines:

Let us now make a list of all things that are important to understand Test Execution phase:

#1. The build (the code that is written by the dev team is packaged into what is referred to a build- this is nothing but an installable piece of software (AUT), ready to be deployed to QA environment.) being deployed (in other words, installed and made available) to the QA environment is one of the most important aspects that needs to happen for the test execution to start.

#2. Test execution happens in the QA environment. To make sure that the dev team’s work on code is not in the same place, where the QA team is testing, the general practice is to have dedicated Dev and QA environment. (There is also a production environment to host the live application). This is basically to preserve the integrity of the application at various stages in the SDLC life cycle. Otherwise, ideally, all the 3 environments are identical in nature.

#3. Test team size is a not constant from the beginning of the project. When the test plan is initiated the team might just have a Team lead. During the test design phase, a few testers come on board.  Test execution is the phase when the team is at its maximum size.

#4. Test execution also happens in at least 2 cycles (3 in some projects). Typically in each cycle, all the test cases (the entire test suite) will be executed. The objective of the first cycle is to identify any blocking, critical defects, and most of the high defects. The objective of the second cycle is to identify remaining high and medium defects, correct gaps in the scripts and obtain results.

#5. Test execution phase consists of- Executing the test scripts + test script maintenance (correct gaps in the scripts) + reporting (defects, status, metrics, etc.) Therefore, when planning this phase schedules and efforts should be estimated taking into consideration all these aspects and not just the script execution.

#6. After the test script being done and the AUT is deployed – and before the test execution begins, there is an intermediary step. This is called the “Test Readiness Review (TRR)”.  This is a sort of transitional step that will end the test designing phase and ease us into the test execution.

For information on this step and a sample “Test readiness review checklist”, check out this link: Software testing Checklist

#7. In addition to the TRR, there are few more additional checks before we ensure that we can go ahead with accepting the current build that is deployed in the QA environment for test execution.

Those are the smoke and sanity tests. Detailed information on what these are is at: What is Smoke and Sanity Test?

#8. On the successful completion of TRR, smoke and sanity tests, the test cycle officially begins.

#9. Exploratory Testing would be carried out once the build is ready for testing. The purpose of this test is to make sure critical defects are removed before the next levels of testing can start. This exploratory testing is carried out in the application without any test scripts and documentation. It also helps in getting familiar with the AUT.

------------

#10. Just like with the other phases of the STLC, work is divided among team members in the test execution phase also. The division might be based on module wise or test case count wise or anything else that might make sense.

#11. The primary outcome of the test execution phase is in the form of reports – primarily, defect report and test execution status report. The detailed process for reporting can be found at: Test executions reports

New Columns in Test Cases Document:

The test case document now gets to be expanded with the following two columns – Status and Actual result.

(Note – For live project test execution, we have added and updated these columns with test execution results in the test cases spreadsheet provided for download below)

Status column:

Test execution is nothing but, using the test steps on the AUT, supplying the test data(as identified in the test case document) and observing the behavior of the AUT to see if it satisfies the expected result or not. If the expected result is not met, it can be construed as a defect. And the status of the test case becomes “Fail” and if the expected result is met, the status is “Pass”. If the test case cannot be executed because of any reasons (an existing defect or environment not supporting) the status would be “Blocked”. The status of a test case that is yet to be run can be set to No run/unexecuted or can be left empty.

  • For a test case with multiple steps, if a certain step’s (in the middle of the test case steps) expected result is not met, the test case status can be set to “Fail” right there and the next steps need not be executed.
  • The status “Fail” can be indicated in red color, if you would like to draw attention to it immediately.

Actual result column:

This is a space where we testers can record what the deviation in the expected result is. When the expected result is met (or a test case whose status is “Pass”) this field can be left empty. Because, if the expected result is met it means the actual result=expected result, which means rewriting it in the actual result column will be a repetition and redundancy.

A screenshot of the deviation can be attached in this column for enhanced clarity of what the problem is.

Test Execution Results for OrangeHRM Live Project:

Let us now get OrangeHRM and carry out the test execution based on the above guidelines listed. Here are a few points to note:

  • The extended test case template.
  • Exploratory testing as indicated is to be carried out without test scripts. So please feel free to test the application in parallel as you see fit.
  • Due to the limitations that we have in presenting the live project in the form of readable content- only a limited amount of test cases/functionality of the OrangeHRM application is shown in the sample test execution template. Again, please feel to work on more for the most practical experience.
  • The sanity and smoke test suites are also added to the document, to give you an idea about what kind of test cases are considered for these stages.
  • Defects are not logged yet, even though the status of some test cases is set to “Fail”. This is because, logging the defects is the next most important/commonly worked on aspect of our life as testers. So, we want to deal with it in detail in the next article.

Test Cases with Execution Results:

=> Click here to download the test case execution document.

It Contains – Test cases execution result, smoke tests, sanity tests, exploratory test – spreadsheets

Lastly, if a test management tool was used for creating and maintaining the test case, the same can be used for test execution as well. The use of a tool makes reporting easier, but otherwise the process of running the test cases is the same.  Please check out this article to get an idea on how to use HP ALM for test case execution.

(Click on image to enlarge)

ALM for test excution result

This brings us to the end of another interesting segment of the testing process. In the next and last article of this free online Software Testing QA training mini course, we will look into defects in detail; wrap up topics like “when to stop testing”, metrics and QA sign off.

=> QA Training Day 6: Bug Tracking, Test Metrics, and Test Sign off

Please let us know how we are doing and stay tuned for the next article.

Recommended reading

27 comments ↓

#1 Tushar

Very informative artical

#2 Poovarasan Murugan

Clearly said… and great real time example of Smoke, Sanity, Exploratory Test and Functional Test.

#3 Ganesh

got clear picture of STLC. waiting for the conclusing tutorial now.

#4 baza

ive really learned a lot from this project. looking forward to the next phase of defects.

#5 venkatakrishna

Nice, Thank you

#6 Ritesh

Great.. Thanks..

#7 srinivas kadiyala

Do we need to submit our test cases to developers for review?
Why it is necessary to do it for change ?

#8 srinivas kadiyala

If the testing environment also contains the same softwares installed of developers before, does testing effort effects the results?

#9 Nitin D.

Great work …plz continue

#10 nandini

Hi Vijay,
thanks for the articles about live testing.

can you also provide some real time communication mails within team members.
Exactly how to write mails in between team.

thanks

#11 dora

Could you please explain me on How to write test scripts
for this Project? i would really appreciate your example test scripts for this.

Thanks

#12 dora

Are test scripts and test cases same? I am sorry if it is silly question to ask here. I have just started this testing course.

Thanks

#13 Ahmed fathi

really so valued training

#14 Irfan

Exploratory testing is not testing without test scripts and documentation. Ad-hoc means testing with out any scripts or test cases.

#15 Swati

@srinivas kadiyala: The reason why we share the test cases with the Dev team is because- the dev team creates the code for the AUT and they know it very well. So, by sharing the cases that we intend to test with them, we are seeking their opinion, asking them “Have we taken everything into consideration”. Also, sometimes, dev team can provide useful insights into areas of AUT that will need more concentration. (For example, if a certain module was developed last and did not have enough time to Unit or integration tested extensively, in that case, dev team can point specific tests that will help us fill that gap)

I hope this helps!

#16 Swati

@dora: Test cases and test scripts are terms that can be used interchangeably. Test script, at times, is a term that is used more in the context of automation.

#17 Gaurav Thakkur

Yes, I am agree with irfan. In adhoc testing, test case is not required. There we used to do random testing. But in exploratory testing, first we need to explore the application to get the idea about application and its flow which will help to prepare test cases.

#18 Deyan

Great Stuff..
Wish you all the best :)

#19 vignesh

Wow…It’s Awesome:)

#20 Praveen kumar Reddy

HI ,

Nice blog……………

#21 Sunny

Hi, I thank this blog writer and others for coming up with such a great work. For a beginner like me, this website has made everything available, good understanding and learning. I was looking for projects and I am glad I found it here, I am truly thankful to you all. Please don’t stop this good work. My best regards…

#22 Rian

Very informative.. I am a developer with 3.6 yrs of experience and want to switch to manual testing.. Please guide me on how to achieve it….

#23 Yasir Arafat

Hi software Testing Help Team ,I really appreciate your work and effort ,for whole web site and spacialy in 06 tutorials ,these are amazing and you guys are simply awesome,I learned a lot from these tutorial ,and you guys made it.
thank you so much
from Toronto Canada

#24 kali damburu reddy

it is very very very nice article sir real

#25 kali damburu reddy

sir one more ?
i am fresher what type of ? ask hr manager please few q and answer tell me sir

#26 kali damburu reddy

thank q sir good 9i8

#27 gaurav

pls help me im still confuse about which test cases are sanity and smoke test cases in the test case document

Leave a Comment