What is Build Verification Testing (BVT)?
Build Verification Test is a set of tests run on every new build to verify that the build is testable before it is released to the testing team for further testing.
These test cases are core functionality test cases that ensure that the application is stable and can be tested thoroughly. Typically the BVT process is automated. If BVT fails, then that build will again get assigned to a developer for the fix.
What You Will Learn:
Build Verification Testing (BVT Testing)
BVT is also called as Smoke Testing or Builds Acceptance Testing (BAT).
New Build is checked mainly for two things:
- Build Validation
- Build Acceptance
BVT Basics
- This is a subset of tests that verify the main functionalities.
- The BVT’s are typically run on daily builds and if the BVT fails the build is rejected and a new build is released after the fixes are done.
- The advantage of BVT is that it saves the efforts of a test team to set up and test a build when major functionality is broken.
- Design BVTs carefully to cover basic functionality.
- Typically BVT should not run for more than 30 minutes.
- BVT is a type of Regression Testing, done on each and every new build.
BVT primarily checks for the project integrity and checks whether all the modules are integrated properly or not. Module integration testing is very important when different teams develop project modules.
We have heard of many cases of application failure due to improper module integration. Even in the worst cases, the complete project gets scrapped due to failure in module integration.
What is the Main Task in Build Release
Obviously file ‘check-in’ i.e. to include all new and modified project files associated with respective builds.
BVT was primarily introduced to check initial build health i.e., to check whether – all new and modified files are included in the release, all file formats are correct, and every file version, language & flags associated with each file.
These basic checks are worth before build release to test team for testing. You will save time and money by discovering the build flaws at the very beginning using BVT.
Which Test Cases Should Be Included in BVT
This is a very tricky decision to make before automating the BVT task. Keep in mind that the success of BVT depends on which test cases you include in BVT.
Here are some simple tips to include in Test Cases in your BVT Automation Suite:
- Include only critical test cases in BVT.
- All test cases included in the BVT should be stable.
- All test cases should have known the expected results.
- Make sure that all included critical functionality test cases are sufficient for application test coverage.
Also, do not include modules in BVT, which are not yet stable. Due to some under-development features, you can’t predict expected behavior as these modules are unstable and you might know some known failures before testing for these incomplete modules. There is no point in using such modules or test cases in BVT.
You can make this critical functionality test case inclusion task simple by communicating with all those involved in the project development and testing life cycle. Such a process should negotiate BVT test cases, which ultimately ensure BVT success.
Set some BVT quality standards and these standards can be met only by analyzing the major project features and scenarios.
For Example, Test cases to be included in BVT for Text editor application (some sample tests only):
- Test case for creating the text file.
- Test cases for writing something into the text editor.
- Test case for copy, cut, and paste functionality of text editor.
- Test cases for opening, saving, and deleting text files.
These are some sample test cases which can be marked as “critical” and for every minor or major change in the application, these basic critical test cases should be executed. This task can be easily accomplished by BVT.
BVT automation suits need to be maintained and modified from time-to-time. E.g. include test cases in BVT when there are new stable project modules available.
What Happens When BVT Suite Runs
Say Build verification automation test suite executed after any new build.
- The results of the BVT execution will be sent to all the email ID’s associated with the project.
- The BVT owner (person executing and maintaining the BVT suite) inspects the result of BVT.
- If BVT fails then BVT owner diagnoses the cause of failure.
- If the cause of failure is a defect in the build, then all the relevant information with failure logs will be sent to the respective developers.
- Developer on his initial diagnostic replies to the team about the failure cause. Is this really a bug? If it’s a bug then what will be his bug-fixing scenario?
- On the bug fix, once again the BVT test suite is executed and if the build passes BVT, the build is passed to the test team for further detailed functionality, performance, and other tests.
This process gets repeated for every new build.
Why did BVT or Build Fail?
BVT breaks sometimes and this doesn’t mean that there is always a bug in the build.
There are a few other reasons to build fail like test case coding errors, automation suite errors, infrastructure errors, hardware failures etc.
You need to troubleshoot the cause of the BVT break and need to take proper action after diagnosis.
Tips for BVT Success
- Spend considerable time writing BVT test case scripts.
- Log as much detailed info as possible to diagnose if the BVT passes or fails as a result. This will help the developer team to debug and quickly understand the failure cause.
- Select stable test cases to include in BVT. For new features, if a new critical test case passes consistently on a different configuration then promote this test case in your BVT suite. This will reduce the probability of frequent build failures due to new unstable modules and test cases.
- Automate the BVT process as much as possible. Right from the build release process to the BVT results – automate everything.
- Have some penalties for breaking the build ;-) Some chocolate or team coffee party from a developer who breaks the build will do.
Conclusion
BVT is nothing but a set of regression test cases that are executed each time for the new build. This is also called a smoke test. The build will not be assigned to the test team unless and until the BVT passes.
BVT can be run by developers or testers and BVT results are communicated throughout the team and immediate action is taken to fix the bug if BVT fails. BVT processes are typically automated by writing scripts for test cases.
Only critical test cases are included in BVT. These test cases should ensure application test coverage. BVT is very effective for daily as well as long-term builds. This saves significant time, cost & resources and after all no frustration of the test team for the incomplete build.
If you have some experience in the BVT process then please share it with our readers in the comments below.
thanks a lot Vijay for the post.
i got really good idea of BVT. Was eagar to know abt BVT since long and now your post helped me a lot.
hi,
thanking fr the more information refgarding th bvt thanks a lot
with best regards,
G.Viswanath
Thank you………Vijay
I really got a new idea for my testing process, which I can implement in my testing
Hi Vijay,
This is fantastic, good tips for beginners. My Expectation from your side is new articles daily.
Regards,
R.Prakash.
Hi,
Thanx for your more valueable information for smoke testing.
i would like to know about sanity testing.
thanks a lot ……ur articles are very helpful….
Hi Sonali,
as far as I know, “sanity test” is just another expression for smoke test.
Good work Vijay.
BR
Jens
Nice Article!
Nice Article!
hi ,
can you please tell how to test Reports generated by sql server reporting services ?
or provide any links where we can refer to acquire knowledge about testing reports.
Thanks,
Pradeep.
Hi vijay,
really a helpful article for the beginners, now here goes my query realted to BVT to u ,
most of the time Devloper does the BVT and the build team modifies the database changes and implement them during the next version build,
now for the companies who deals in pure manual testing how can we assure the BVT took place and its working ?
Is there anyway through which we can make same thing working in manual testing?
Thanks & Regards,
Avinash.
Thnank U Vijay……..
Good Article
thanx
thanx nice
hi,
i didnt see any article on performace testing on this site, can you give it so that it will be very helpful for us.
thnaks,
Pradeep.
Thanks a lot Vijay for this article on BVT.
This kind of summarizes the main ideas we should know about BVT very well.
Thats a lot and looking forward for more such articles.
-subha
Great vijay,
you had given me the key to achieve my destination.
thanks a lot.
hi,
thanks for this information!!!!!!
Nice Information.
It really helped a lot.
Developers migh be doing the unit and intregration testing and not necessarily the BVT. BVT is most of the times done by test engineer. Once the build team deploys the test build on the test environments its the job of test engineer to perform BVT (sniff, sanity smoke etc..)
If you are able to test the applicaiton and are executing the test cases, its natural that the BVT has been passed. Else you wouldn’t have been able to test the applciation… It dosent mattter if the testing is done manually or is automated using any tool. If the build has got a new feature how can you automate it?
Following is the process that you might follow:
Developer initiates a mail to build team (also marked to test team with the description of what to be tested in the new build) to make the build >> Build team makes the build and deploys it on test machines and replies all asking the test team to continue testing, else if build fails he replies so in the mail >> If the BVT fails tester replies to mail stating that the BVT failed along with the logs whatever are available, else continues testing.
Regards, Suhas M
*the post above is marked to Avinash.
Hi,
I found this article very interesting, thanks for the information. While testing web modules we frequently come across schema hardcoding done by developers, can it also be a part of BVT, even if it is not related to any specific module.
it is very nice article…Thanks a lot
hi frends
i want to know possible test cases for health care system.
please reply be asap bcoz i have an interview.
Thanks a lot Vijay and all the related guys and gals.
Hello Vijay,
Talking about BVT, I am sure you would agree that this is one of the most confused and most misunderstood concepts in S/W Testing. Few weeks ago one of my blog readers had asked me a plain vanilla definition of BVT. But unlike you (giving away my version of definition of BVT), I opted for hearing the different versions of definitions circulating in the Testosphere! And the result was shocking! Well, here is the result of the Case Study in case anybody wants to take a look.
Happy Testing…
-Debasis
Hi Suhas ,
Now its too much clear to me wht exactly BVT the author want to tell us in this nice Post.Well you are absolutely right for the Procedure that we are following for testing. Anyways thanks sir for the useful information and really appreciated your reply.
Thanks N Regards,
Avinash.
Thats a good article by vijay but as pointed out by suhas, it depends from organisation to organisation. Its not necessary that BVT is done by developers. Once the build is ready it is released to the testing team who does the smoke testing and confirms for failure or success of the same. Thanks for suhas’s comment.
Hi,
what is the diff b/w embedded software and application software…
Hi,
It is a good article.
Very clear description is given about BVT.
Hi frnds,
recently i given a test in one company, i didnt got selected in HR..
well in that written test they asked abt “Cyclometric” wt it mean?
wts the use of it?
anybody know?
Hey L V Prasad,
Could you please clearly mention what exactly the question is?
is it Cyclometric / Psychometric?
Did they ask this question….
What is Cyclometric complexity? (is this the question?)
if Yes : – Then:- The cyclomatic complexity is a software metric that provides a quantitative measure of the logical complexity of a program.
hai all,
i want details about ISTQB & STUDY MATERIALs. If
u’ve kindly fwd to my mail. my mail id is gsheela80@yahoo.com
Thank u very much.
Hi,
Thanx for the info on BVT. Got a pratical insight as to how it is done. Can u pls recomend how to go about DB testing with some sample testcases?
Thanx,
Meera
Amazing blog!
Was wondering, such an amazing blogger like you should visit Manipal on 18th and 19th April to join Manipal Bloggers’ Meet and Manipal Media Students’ Convention…
what say?
Please visit the site for more info. http://www.mmsc-manipal.com
hi……….
This is really very helpfull aricle and i have learnt a lot from this article
Looking forward to see such types of article more
Thank you very much
@meera
database testing is more related with white box testing stuff ..Actualli i mean to say u need to concentrate on followings
1>varify dsn is correct or not
2>varify weather we are referring correct table or not
3>varify in inline qurey/sp/trigger
4>varify data in the table is updated or latest or not..
this is all very very important prerequisite that tester shold know like developer..since most of invalid bug is because of this…
if anything more ping me back..rahulkumarbangalore@yahoo.co.in
@sonali
I have given clear difference b/n smoke and sanity here 1 more time
Thi sis most mcommon thing weather folks get confused..but i will give more clear example
SMOKE–it checks health of build..weather health i sstable or not ..which decide weather further testing would be carried out or not..we check weather build is crashing or not..it is more systematic than SANITY..
SANITY–it checks functionality of build at higher level..we carried out one teansection related to each module hare..it is less formal and less planned then SMOKE.
if smoke fails sanity stops.Sanity tell weather it is “reasonable” enough to carry out testing or not..
i hope this will clarify major doubt .
Rahul!!!
more info ping back rahulkumarbangalore@yahoo.co.in
@Deepak
i think it would be Cyclomatic Complecity (cc)which is no. of binary decision+1
i.e. cc=binary decision+1
used for test coverage..
more info ping back rahulkumarbangalore@yahoo.co.in
HI
Nice site, have a lots of information.
Thanks
Iam not able to understand what is this APACHE AND TOMCAT SERVER is all about..are they different ?
Some say they are not and they are called as webapplication servers.By the way do application server and web server reside on the same machine or in different machines.How they are configured.
How a application server contacts webserver or vice versa..again are they configured on differnet machines .I am totally confused please help me understand this.Respond me to this e mail with your answers norman_g40@yahoo.com
HI Jen
Sanity testing is come under the same ambit but situation is different.
After a final round of testing then we push produt to client side, before that will do a quick round of testing before First UAT is called Sanity.
I think is right.
Thanks
what is the build and release?
Exactly what is the difference between new build and modified build?
Can we process an xml file in QTP like database files,if possible,how can we process it
hello friends/sir/mam
Its Pankaj Bhatt,completed graduation in Commerce.I know JAVA Language.Looking for Software Testing.Need help from u.
I have make career in this field,so what step i have to follow?
plzs reply at: pankaj.bhatt@hotmail.com
Hi,
I am vishvas tyagi working for Keane India.Can you please tell me about V model testing and what’s the need of this model?
Hi All,
If any of u r tester from Kolkata then pls reply for this thread.
Thanks
Vishal
Call: 9830704327
mail: bishalchaperia@gmail.com
Hi All,
Here are the differences you can see,
SMOKE TESTING:
1) Smoke testing originated in the hardware testing practice of turning on a new piece of hardware for the first time and considering it a success if it does not catch fire and smoke. In software industry, smoke testing is a shallow and wide approach whereby all areas of the application without getting into too deep, is tested.
2) A smoke test is scripted–either using a written set of tests or an automated test
3) A Smoke test is designed to touch every part of the application in a cursory way. It’s is shallow and wide.
4) Smoke testing will be conducted to ensure whether the most crucial functions of a program work, but not bothering with finer details. (Such as build verification).
5) Smoke testing is normal health check up to a build of an application before taking it to testing in depth.
SANITY TESTING:
1) A sanity test is a narrow regression test that focuses on one or a few areas of functionality. Sanity testing is usually narrow and deep.
2) A sanity test is usually unscripted.
3) A Sanity test is used to determine a small section of the application is still working after a minor change.
4) Sanity testing is a cursory testing; it is performed whenever a cursory testing is sufficient to prove the application is functioning according to specifications. This level of testing is a subset of regression testing.
5) sanity testing is to verify whether requirements are met or not, checking all features breadth-first.
Regards,
VijayD
hello sir or madam
today i went to interview.
in that interviewer asked abt qtp and winrunner ,which one is best.
on that time what i told is now every one is using qtp,
is it correct or not.i dont know.
and another question ,we ar ready to test linux ,which one is best again he asked.
what i told is in this testing we ar testing the functionality.
is it correct or not.
he asks how to test the linux.
just i completed the testing course.
regards
bharghava
Where can I download “Mercury Quality Center Connectivity Add-in”? Can anyone provide the link.
hi deepak
cyclometic complexity is one of metric which isused to measure the logical complexity of program
to calculate CYCLOMETIC COMPLEXITY
V(G)=E-N+2
Here
E————>NO OF EDGES
N————>NO OF NODES
THANX KS THANGAVEL
hello,
by this article can we conclude tht BAT, BVT and Smoke test’ all are same kind of testing, or any difference s thr……..pls tel me……
Hi Beena,
You are right. We can conclude that BAT, BVT and Smoke test are same.
Thank U Anand.
By the by, r u working? In which city u r working?
Yes, I am working in Mumbai.
hi,
I recently joined a MNC as a test engineer and i was put into an onsite-offshore model project. I am facing prb regd executing testcases as the application crashes/gives error msg at diff. pts while entering the data due to which i cannot set p/f status to any of the testcases.i do not know where the prb is? And my teamlead does not care abt this and says that i have to finish atleast some testcases before the end of the day as she has to report to onsite lead. I do not know what to do. where is the prb i do not understand. can any one help me.
hi,
I recently joined a MNC as a test engineer and i was put into an onsite-offshore model project. I am facing prb regd executing testcases as the application crashes/gives error msg at diff. pts while entering the data due to which i cannot set p/f status to any of the testcases.i do not know where the prb is? And my teamlead does not care abt this and says that i have to finish atleast some testcases before the end of the day as she has to report to onsite lead. I do not know what to do. where is the prb i do not understand. can any one help me.
didn’t get an answer for this yet. Hoping for an imm. reply folks
Hi Friends,
Currently i am working in Non-IT field and planned to switch my career to testing.I had learned the basics of testing,and now i am very eager to learn testing practically with real time projects.
If any one interested to teach testing personally with real time projects , i will be more grateful, or else show me the right place in chennai.
Thanks & regards
selvam.ba@rediffmail.com
Hi
This is suresh, do you have any idea about build verifcation testing in linux.
A compete essay on this.
Hi Vijay,
Thanks for posting this article. Can you please let me know more about Automation with respect to Silk Test? I am working on that and your points would be of a great help to me and my team.
It’s a good article.Keep posting them.
Hi Vijay,
Thanks for the article, as far for others who think Sanity and smoke testing are same , then this is for them
Sanity testing:
Once a new build is obtained with minor revisions, instead of doing a through regression, a sanity is performed so as to ascertain the build has indeed rectified the issues and no further issue has been introduced by the fixes. Its generally a subset of regression testing and a group of test cases are executed that are related with the changes made to the app.
Generally, when multiple cycles of testing are executed, sanity testing may be done during the later cycles after through regression cycles.
Smoke Testing:
When a build is received, a smoke test is run to ascertain if the build is stable and it can be considered for further testing.
Smoke testing can be done for testing the stability of any interim build.
Smoke testing can be executed for platform qualification tests.
Hope this answers the doubt about smoke and sanity testing.
Thanks & regards
Sarath S Pillai
@ Vijay
Thanks to Sarath S Pillai.I will just add my note here mite b helpful to others.
sanity and smoke there is no much difference and moverover its company specif wt have they have given.
BUT if u go by books dffination
SANITY–is build acceptence test used to certify the build where build is stable or not and we decide here is it worth time spending for further testing or not..
Most important thing is taht we have seperate test cases for SANITY.some company have sanity check list.its one and same thing
SMOKE–as i told there is no much difference between sanity and smoke.However we focus testing on more critical and important test path.basically we check depth wise while in sanity we check bredth wise.
I hope i added some point mite b helpful
How BVT is a type of “Regression Test”? can anyone explain this.
Thanks in advance
This is for the first time i had read about BVT and blive me i dint find any problem in understanding the concept,this matter is so transparent this even i can visualize the BVT process…and which help me lot to keep the concept of BVT in my mind for a long time…
Thanx & Regards…
Hi Shweta…
Actully when we find any bug in any module thru BVT it assigned to the respective developer for fix and after fixing it,we again test the same application and its impact on others thru BVT,and which is the same thing we do in Regression Testing…
Is BVT or Smoke testing functional or non-functional testing technique??
Also can you identify for me about
Sanity, Re-testing,RegressionUsability & User interface testing??
Will appreciate a quick response.
Hi Addy…
Both BVT & smoke testing are functional testing techniques,
Sanity testing is vry similar to smoke testing,in this we test the basic functionality of the component.
Re-testing is to re test the component after fixing the bug.
Regression testing is test the impact of the fixed bug on the whole project.
& UIT ithink it similar to UAT.
hi….
tanks u for information about d testing process
hi sir,
good article
Very Nice and very helpful… Thanks.
Very nice article…. kip posting n cheers to all for the discussion…
Hello Vijay,
Question on BVT- who actually do BVT,Is it developer or tester,as it is done before assigning to testing team ?
Hi Puneet,
Answer is in ur question itself, BVT is kind of white box testing and also a component testing so it is done at developer’s site before assingning it to testing team.
what are the metric involved in test plan
its shantha
what are the metric involved in test plan
I think this excellent answer.
thanks & Regards
Thanks a lot, I got the idea about BVT
Hi Vijay
U have done wonderful job on this site. Its very useful to learn about BVT clearly. Thanks a lot. Thank you a lot.
Thanks a lot for giving information about BVT, I have some doubt about BVT. know i got clear information about that.This site very help to the bigner’s.
Thnank U Vijay……..
@Rahul
Can you do a sanity test of your sentences before you replies in any forum. It’s really hard to understand when someone makes so much mistakes in english.
peace..
thanks so much
its very helpful for me thanks
I had doubt about Selenium :
When i generate any script it takes normal time depends upon internet speed , but when i run it at High speed it hardly takes 1 min to execute test case with many transactions.
Why is that?
how can it be so fast ?
Thanks for making BVT Clear….
Thanks for creating clear picture of BVT ….
Hey, Do you really practice the script execution for the Smoke testing. And also in many cases of keyword driven testing, the flows are not very much organised to suit the smoke tetsing suite, as oer my knowledge. SO how is that we proceed with the same in case it has to be practised.
hi vijay ,i am sandya.now i am studying mphil (C.A).i select software testing for my research topic.my guide told me to take that topic is ” how to enhance performance on regression testing?”.but i do not know how 2 do that?pls send some related heading to my mail id.my mail id is sandyaselvam@gmail.com
Hi Vijay ..
The details about BAT is really very helpful and Could I know which automation testing tool would be used to test the Build Installation.
Please do get me some idea for this as I am assigned to google this in my concern.
I am satisfy with this answer and I hope this type of answering will be continue for every question.
Thanks
hi…this is nice one topic…its really help those ..who have only therotical knowledge…practically..how testing performed in company environment…this is good one post..i like it..
i leant sumthg abt bvt testing by using dis article …
thanx yo….
Hi,
I am working as a release team member , daily i installed my latest build and execute some set of test cases, it is very painful job to do daily. can you please suggest me is dar any tool for this type of snoke testing??
Good Site But Give real example .
Hi Vijay,,,can you tell me after completing of smoke testing for lateral builds we perform sanity testing….
Hi Vijay, Thanks a lot for this excellent article. You covered all the necessary points very nicely.
Well,really a good article to read and I would appreciate such initiatives.I am manual test engineer and have overall 2.6 years of experience guys can you share your respective email Id,s to me or let me know if there are any opening on pratapvarun93@gmail.com,I,ll be highly thankful to you all.
Awesome explanation…… please do keep posting with more examples……
Hi,
There is as such no testing process in my organisation but what i do is …creating test scenarios and test cases based on that…try to cover all client requirement ..only tester for 5-6 developers team..have test their individual module as well as integrated system ..
but after doing this for long time i found this very time consuming and i have to put more extra efforts to get that quality which i want in my products.
can any one suggest me what is the good option in that case to achieve a quality build with minimum amount of efforts ..
i am not saying that i don’t want to work hard to achieve my goals all I want is smart way to get it ..
i know automation is an option and i am looking for that trying selenium and little bit qtp..
but automation is applicable after system is testable .
Thanks
waiting for rep
superb article
Hi Vijay,
If we release a new feature for application, how to include automation test case in BVT for the new feature when the feature just release but we does not implement auto script yet?
Hey, BVT is not for your new feature.After introducing your new features, it should not break core functionality.Our module or functionality should be stable, your new feature may have some bugs.Please correct me I am wrong.
Thanks,
Sandeep Khopade
very well written
@Sourabh
kindly provide your contact detail so we discuss about Selenium and other automation tools.
Hi,
Thanks for the post. It is very useful.
Very good writeup. Covers many major aspects of BVT.
Very well explained…….is their any connection with Jenkins for BVT??
Awesome explanations!
why do you call it a Smoke test that belongs to the regression technique while the linked page about “Smoke test” denotes it as an acceptance group?
Hi Vijay,
I am immensely pleased to know the detailed explanation of BVT but although we went through such a process.
I have a question here? Where do we store Build Versions and what is the standard format?
Best Regards,
Maqsood Syed
awesome no words to explain
every line is written very precisely
Thanks