What is Build Verification Testing (BVT)?
Build Verification Test is a set of tests run on every new build to verify that the build is testable before it is released to the testing team for further testing.
These test cases are core functionality test cases that ensure that the application is stable and can be tested thoroughly. Typically the BVT process is automated. If BVT fails, then that build will again get assigned to a developer for the fix.
What You Will Learn:
Build Verification Testing (BVT Testing)
BVT is also called as Smoke Testing or Builds Acceptance Testing (BAT).
New Build is checked mainly for two things:
- Build Validation
- Build Acceptance
BVT Basics
- This is a subset of tests that verify the main functionalities.
- The BVT’s are typically run on daily builds and if the BVT fails the build is rejected and a new build is released after the fixes are done.
- The advantage of BVT is that it saves the efforts of a test team to set up and test a build when major functionality is broken.
- Design BVTs carefully to cover basic functionality.
- Typically BVT should not run for more than 30 minutes.
- BVT is a type of Regression Testing, done on each and every new build.
BVT primarily checks for the project integrity and checks whether all the modules are integrated properly or not. Module integration testing is very important when different teams develop project modules.
We have heard of many cases of application failure due to improper module integration. Even in the worst cases, the complete project gets scrapped due to failure in module integration.
What is the Main Task in Build Release
Obviously file ‘check-in’ i.e. to include all new and modified project files associated with respective builds.
BVT was primarily introduced to check initial build health i.e., to check whether – all new and modified files are included in the release, all file formats are correct, and every file version, language & flags associated with each file.
These basic checks are worth before build release to test team for testing. You will save time and money by discovering the build flaws at the very beginning using BVT.
Which Test Cases Should Be Included in BVT
This is a very tricky decision to make before automating the BVT task. Keep in mind that the success of BVT depends on which test cases you include in BVT.
Here are some simple tips to include in Test Cases in your BVT Automation Suite:
- Include only critical test cases in BVT.
- All test cases included in the BVT should be stable.
- All test cases should have known the expected results.
- Make sure that all included critical functionality test cases are sufficient for application test coverage.
Also, do not include modules in BVT, which are not yet stable. Due to some under-development features, you can’t predict expected behavior as these modules are unstable and you might know some known failures before testing for these incomplete modules. There is no point in using such modules or test cases in BVT.
You can make this critical functionality test case inclusion task simple by communicating with all those involved in the project development and testing life cycle. Such a process should negotiate BVT test cases, which ultimately ensure BVT success.
Set some BVT quality standards and these standards can be met only by analyzing the major project features and scenarios.
For Example, Test cases to be included in BVT for Text editor application (some sample tests only):
- Test case for creating the text file.
- Test cases for writing something into the text editor.
- Test case for copy, cut, and paste functionality of text editor.
- Test cases for opening, saving, and deleting text files.
These are some sample test cases which can be marked as “critical” and for every minor or major change in the application, these basic critical test cases should be executed. This task can be easily accomplished by BVT.
BVT automation suits need to be maintained and modified from time-to-time. E.g. include test cases in BVT when there are new stable project modules available.
What Happens When BVT Suite Runs
Say Build verification automation test suite executed after any new build.
- The results of the BVT execution will be sent to all the email ID’s associated with the project.
- The BVT owner (person executing and maintaining the BVT suite) inspects the result of BVT.
- If BVT fails then BVT owner diagnoses the cause of failure.
- If the cause of failure is a defect in the build, then all the relevant information with failure logs will be sent to the respective developers.
- Developer on his initial diagnostic replies to the team about the failure cause. Is this really a bug? If it’s a bug then what will be his bug-fixing scenario?
- On the bug fix, once again the BVT test suite is executed and if the build passes BVT, the build is passed to the test team for further detailed functionality, performance, and other tests.
This process gets repeated for every new build.
Why did BVT or Build Fail?
BVT breaks sometimes and this doesn’t mean that there is always a bug in the build.
There are a few other reasons to build fail like test case coding errors, automation suite errors, infrastructure errors, hardware failures etc.
You need to troubleshoot the cause of the BVT break and need to take proper action after diagnosis.
Tips for BVT Success
- Spend considerable time writing BVT test case scripts.
- Log as much detailed info as possible to diagnose if the BVT passes or fails as a result. This will help the developer team to debug and quickly understand the failure cause.
- Select stable test cases to include in BVT. For new features, if a new critical test case passes consistently on a different configuration then promote this test case in your BVT suite. This will reduce the probability of frequent build failures due to new unstable modules and test cases.
- Automate the BVT process as much as possible. Right from the build release process to the BVT results – automate everything.
- Have some penalties for breaking the build ;-) Some chocolate or team coffee party from a developer who breaks the build will do.
Conclusion
BVT is nothing but a set of regression test cases that are executed each time for the new build. This is also called a smoke test. The build will not be assigned to the test team unless and until the BVT passes.
BVT can be run by developers or testers and BVT results are communicated throughout the team and immediate action is taken to fix the bug if BVT fails. BVT processes are typically automated by writing scripts for test cases.
Only critical test cases are included in BVT. These test cases should ensure application test coverage. BVT is very effective for daily as well as long-term builds. This saves significant time, cost & resources and after all no frustration of the test team for the incomplete build.
If you have some experience in the BVT process then please share it with our readers in the comments below.
Awesome explanation…… please do keep posting with more examples……
Hi,
There is as such no testing process in my organisation but what i do is …creating test scenarios and test cases based on that…try to cover all client requirement ..only tester for 5-6 developers team..have test their individual module as well as integrated system ..
but after doing this for long time i found this very time consuming and i have to put more extra efforts to get that quality which i want in my products.
can any one suggest me what is the good option in that case to achieve a quality build with minimum amount of efforts ..
i am not saying that i don’t want to work hard to achieve my goals all I want is smart way to get it ..
i know automation is an option and i am looking for that trying selenium and little bit qtp..
but automation is applicable after system is testable .
Thanks
waiting for rep
superb article
Hi Vijay,
If we release a new feature for application, how to include automation test case in BVT for the new feature when the feature just release but we does not implement auto script yet?
Hey, BVT is not for your new feature.After introducing your new features, it should not break core functionality.Our module or functionality should be stable, your new feature may have some bugs.Please correct me I am wrong.
Thanks,
Sandeep Khopade
very well written
@Sourabh
kindly provide your contact detail so we discuss about Selenium and other automation tools.
Hi,
Thanks for the post. It is very useful.
Very good writeup. Covers many major aspects of BVT.
Very well explained…….is their any connection with Jenkins for BVT??
Awesome explanations!
why do you call it a Smoke test that belongs to the regression technique while the linked page about “Smoke test” denotes it as an acceptance group?
Hi Vijay,
I am immensely pleased to know the detailed explanation of BVT but although we went through such a process.
I have a question here? Where do we store Build Versions and what is the standard format?
Best Regards,
Maqsood Syed
awesome no words to explain
every line is written very precisely
Thanks