Entries Tagged 'Testing Concepts' ↓

This is a guest post by Meenal Balajiwale.

“If QA (Quality Assurance) is done then why do we need to perform QC (Quality Control)?”, this thought may come to our mind some times and looks a valid point too.  This means if we have followed all the pre-defined processes, policies and standards correctly and completely then why do we need to perform a round of QC?

In my opinion QC is required after QA is done. While in ‘QA’ we define the processes, policies, strategies, establish standards, developing checklists etc. to be used and followed through out the life cycle of a project. And while in QC we follow all those defined processes, standards and policies to make sure Continue reading →

Topics we will cover in this article:
- Application Testing
- Categories of Applications
- Application Testing Methodologies
- Application Testing Tools
- Software Test Plan
- Application Testing Cycles
- Application Testing – Best Practices

Application Testing is an activity that every software tester performs daily in his career. These two words are extremely broad in practical aspect. However, only the core and most important areas will be discussed here. The purpose of this article is to touch all the primary areas so that the readers will get all the basic briefing at a single place.

Categories of Applications

Whether it is small calculator software with only the basic arithmetic Continue reading →

What is Agile Testing?
“Agile testing involves testing from the customer perspective as early as possible, testing early and often as code becomes available and stable enough from module/unit level testing.” – A wikipedia definition.

Why Need of Innovations in the Age of Agile Testing?

Global Recession/Economic downtime effect
Current Events are not Current Trends –

When global downturns hit, there is certain inevitability to their impact on information technology and Finance Sectors. Customers become more reluctant in giving software business. Some customers are withdrawing their long term projects and some customers using the opportunities in quoting low price. Many projects that dragged much longer than Continue reading →

Boundary value analysis and Equivalence partitioning, explained with simple example:

Boundary value analysis and equivalence partitioning both are test case design strategies in black box testing.

Equivalence Partitioning:

In this method the input domain data is divided into different equivalence data classes. This method is typically used to reduce the total number of test cases to a finite set of testable test cases, still covering maximum requirements.

Continue reading →

Despite of hundreds of web articles on Smoke and sanity testing, many people still have confusion between these terms and keep on asking to me. Here is a simple and understandable difference that can clear your confusion between smoke testing and sanity testing.

Here are the differences you can see:

SMOKE TESTING:

• Smoke testing originated in the hardware testing practice of turning on a new piece of hardware for the first time and considering it a success if it does not catch fire and smoke. In software industry, smoke testing is a shallow and wide approach whereby all areas of the application without getting into too deep, is tested.
• A smoke test is scripted, either using a written set of tests or an automated test
• A Smoke test is designed to touch every part of the application in a cursory way. It’s shallow and wide.
• Smoke testing is conducted to ensure whether the most crucial functions of a program are working, but not bothering with finer details. (Such as build verification).
• Smoke testing is normal health check up to a build of an application before taking it to testing in depth.

SANITY TESTING:

• A sanity test is a narrow regression test that focuses on one or a few areas of functionality. Sanity testing is usually narrow and deep.
• A sanity test is usually unscripted.
• A Sanity test is used to determine a small section of the application is still working after a minor change.
• Sanity testing is a cursory testing, it is performed whenever a cursory testing is sufficient to prove the application is functioning according to specifications. This level of testing is a subset of regression testing.
• Sanity testing is to verify whether requirements are met or not, checking all features breadth-first.

Hope these points will help you to clearly understand the Smoke and sanity tests and will help to remove any confusion.

Thanks to VijayD for answering this question in simple way for our readers.

If you have more points on smoke and sanity testing to elaborate on, please comment below.

What is BVT?

Build Verification test is a set of tests run on every new build to verify that build is testable before it is released to test team for further testing. These test cases are core functionality test cases that ensure application is stable and can be tested thoroughly. Typically BVT process is automated. If BVT fails that build is again get assigned to developer for fix.

BVT is also called smoke testing or build acceptance testing (BAT)

New Build is checked mainly for two things:

• Build validation
• Build acceptance

Some BVT basics:

• It is a subset of tests that verify main functionalities.
• The BVT’s are typically run on daily builds and if the BVT fails the build is rejected and a new build is released after the fixes are done.
• The advantage of BVT is it saves the efforts of a test team to setup and test a build when major functionality is broken.
• Design BVTs carefully enough to cover basic functionality.
• Typically BVT should not run more than 30 minutes.
• BVT is a type of regression testing, done on each and every new build.

BVT primarily checks for the project integrity and checks whether all the modules are integrated properly or not. Module integration testing is very important when different teams develop project modules. I heard many cases of application failure due to improper module integration. Even in worst cases complete project gets scraped due to failure in module integration.

What is the main task in build release? Obviously file ‘check in’ i.e. to include all the new and modified project files associated with respective builds. BVT was primarily introduced to check initial build health i.e. to check whether – all the new and modified files are included in release, all file formats are correct, every file version and language, flags associated with each file.
These basic checks are worth before build release to test team for testing. You will save time and money by discovering the build flaws at the very beginning using BVT.

Which test cases should be included in BVT?

This is very tricky decision to take before automating the BVT task. Keep in mind that success of BVT depends on which test cases you include in BVT.

Here are some simple tips to include test cases in your BVT automation suite:

• Include only critical test cases in BVT.
• All test cases included in BVT should be stable.
• All the test cases should have known expected result.
• Make sure all included critical functionality test cases are sufficient for application test coverage.

Also do not includes modules in BVT, which are not yet stable. For some under-development features you can’t predict expected behavior as these modules are unstable and you might know some known failures before testing for these incomplete modules. There is no point using such modules or test cases in BVT.

You can make this critical functionality test cases inclusion task simple by communicating with all those involved in project development and testing life cycle. Such process should negotiate BVT test cases, which ultimately ensure BVT success. Set some BVT quality standards and these standards can be met only by analyzing major project features and scenarios.

Example: Test cases to be included in BVT for Text editor application (Some sample tests only):
1) Test case for creating text file.
2) Test cases for writing something into text editor
3) Test case for copy, cut, paste functionality of text editor
4) Test case for opening, saving, deleting text file.

These are some sample test cases, which can be marked as ‘critical’ and for every minor or major changes in application these basic critical test cases should be executed. This task can be easily accomplished by BVT.

BVT automation suits needs to be maintained and modified time-to-time. E.g. include test cases in BVT when there are new stable project modules available.

What happens when BVT suite run:
Say Build verification automation test suite executed after any new build.
1) The result of BVT execution is sent to all the email ID’s associated with that project.
2) The BVT owner (person executing and maintaining the BVT suite) inspects the result of BVT.
3) If BVT fails then BVT owner diagnose the cause of failure.
4) If the failure cause is defect in build, all the relevant information with failure logs is sent to respective developers.
5) Developer on his initial diagnostic replies to team about the failure cause. Whether this is really a bug? And if it’s a bug then what will be his bug-fixing scenario.
6) On bug fix once again BVT test suite is executed and if build passes BVT, the build is passed to test team for further detail functionality, performance and other testes.

This process gets repeated for every new build.

Why BVT or build fails?
BVT breaks sometimes. This doesn’t mean that there is always bug in the build. There are some other reasons to build fail like test case coding error, automation suite error, infrastructure error, hardware failures etc.
You need to troubleshoot the cause for the BVT break and need to take proper action after diagnosis.

Tips for BVT success:
1) Spend considerable time writing BVT test cases scripts.
2) Log as much detailed info as possible to diagnose the BVT pass or fail result. This will help developer team to debug and quickly know the failure cause.
3) Select stable test cases to include in BVT. For new features if new critical test case passes consistently on different configuration then promote this test case in your BVT suite. This will reduce the probability of frequent build failure due to new unstable modules and test cases.
4) Automate BVT process as much as possible. Right from build release process to BVT result – automate everything.
5) Have some penalties for breaking the build Some chocolates or team coffee party from developer who breaks the build will do.

Conclusion:
BVT is nothing but a set of regression test cases that are executed each time for new build. This is also called as smoke test. Build is not assigned to test team unless and until the BVT passes. BVT can be run by developer or tester and BVT result is communicated throughout the team and immediate action is taken to fix the bug if BVT fails. BVT process is typically automated by writing scripts for test cases. Only critical test cases are included in BVT. These test cases should ensure application test coverage. BVT is very effective for daily as well as long term builds. This saves significant time, cost, resources and after all no frustration of test team for incomplete build.

If you have some experience in BVT process then please share it with our readers in comments below.

If you like this post do consider subscribing to our email newsletter to ensure you will not miss any article on software testing! Click here to get article updates by email.