This is in fact most crucial test for time critical applications such as stock management systems which gets refreshed every minute creating huge load on web site as there are thousands of users accessing the web site at the same moment.

For such applications stress testing with more than 10,000 users is a basic test. The WAPT Pro comes with default “Load Agents” functionality to test average load on any web site. But for high capacity test with more than 10,000 users we can now leverage the power of x64 Load Engine extension.

x64 Load Engine is similar to Load Agents feature on WAPT Pro installation. The main difference is in the ability of load engine to generate huge load using 64 bit Windows systems. Using one 64 bit server you can dramatically increase the load testing capacity on web site under test. The x64 Load Engine can be configured in such a way that using slightly high end hardware you can easily generate more than 100k virtual users load on web site.

Assume that you want to stress test your web site with 100,000 concurrent virtual users. You can achieve this using 4 servers each with 25,000 virtual users. Setup this load engine on 4 systems and use them concurrently to achieve desired web site load. This powerful x64 Load Engine effectively utilizes the available memory resources on 64 bit system architecture.

WAPT Pro x64 Load Engine Installation:
- You can install x64 Load Engine on 64 bit version of Windows XP/2003/Vista/2008/Win7 OS.
- x64 Load engine works best on following hardware configuration:
Core i5/Phenom or CPU better than this, 8+ GB RAM and Gigabit Ethernet

How to install:    
x64 Load Engine is a WAPT Pro extension so it runs on WAPT Pro tool similar to Load Agents. To use this load engine you must have WAPT Pro installed first. You can download and install x64 Load Engine from below mentioned link. After installation you can start using load engine using the WAPT pro workplace itself.
Download WAPT Pro x64 Engine.

x64 Load Engine Pros:
- You can generate almost unlimited virtual users (test load) on Windows 64 system.
- Easy to install, learn and configure
Check out below screenshot to know the simplicity of x64 Load Engine Manager:

- No restrictions on the size of used virtual memory

x64 Load Engine Cons:
- No evaluation period available
- Works with WAPT Pro only

As said earlier in my comment, this tool is very useful for performance, stress & load testers. Other testers can get a glimpse of the performance testing scope which is very interesting to work on!

Please ask your queries in below comment section. Your feedback is always appreciated!

---------------------
I just wanted to thank you for subscribing to SoftwareTestingHelp.com. This blog is a success because of you and your support. Here's to your success!

As always, I appreciate your valuable comments. Visit below link to post your comment:
Want to Stress Test Website with 10,000+ Users? Use WAPT Pro x64 Load Engine

Thank you so much! ---------------------
Recommended: TestLodge - Online test case management tool - Web based test case management software allowing you to manage your test plans, requirements, test cases and test runs with ease.

How loadUI works:

LoadUI consist of building blocks called Components that you drag from the Component Toolbar to the Project you’re working on. These components connect to each other by wires and there is no limit for how many components that you can create and connect.

This way of drag-and-drop testing is especially useful during the test, since you can do complex load tests. The overview that you get makes it easier to try different combinations during the actual test and react to them immediately. In loadUI, Instant feedback is king.

Creating load tests in loadUI:

In order to run a test in loadUI you need at least two components (and an empty Project to work in);

- A generator
Which creates load, for example 100 requests per second.
- A Runner
That runs a certain type of test, for example a web test.

So let’s do it; drag out a Fixed Rate Generator and a Web Page Runner.
You can’t run them yet though first you need to connect them. You do this by connecting the nodes, or terminals, located above and below the components to each other with connections.

So now connect the Fixed Rate Generator to the Web Page Runner.

Now that everything is connected you should enter what website you want to load test into the Web Page Runner. Your test is now ready to run! Click the play button in the top menu to make your generator start sending requests to your site.

You can now follow, in real-time, how the test is going by clicking the Results Workbench button. Once there, double click on your Current Run and, from the left hand menu, drag out Response Time to the rack.

Congratulations! You have now created your first load test in loadUI. Play around with the widgets you have on your dashboard, experiment by adding some more, and try tracking other values in the Results Workbench. A good next step would to add an Assertion component and connecting it to the Web Page Runner. Then you can, for example, test how many times you your requests come through while following certain rules you set up (like taking less than 10 milliseconds).

Pro Tips:

Let a manager test!
This is the modern way of gorilla testing.  Let someone less proficient do the testing for you and see what breaks your service. It will also create an understanding and passion for testing within the management. A usual comment is “This is fun! But hard…”

It’s interactive, use it!
LoadUI is highly interactive and configurable in real-time, so take advantage of that.

Create a second test in real time against the same server with a different Generator and see if the system shows weaknesses when exposes to chaotic, life-like, behavior. Change this in real time and experiment.

Record your tests
Since loadUI is so interactive, reproducing the tests can be difficult. Use a screen recorder like Wink (which is free) to be able to replay your tests and see what caused the service to crash.

Work structured
LoadUI has support for test cases. Break up your tests into smaller test cases; you can always start them all at once.

Don’t double-work
If you have created functional tests in soapUI, you can simply run these in loadUI with the soapUI component. This way you don’t have to redefine the test another time in loadUI.

Write your own component
LoadUI is not only free; it’s open-source, too. That means that you can write your own components, in Groovy, if you wish. Only you set the limits for what is possible in LoadUI! Read more about creating your own components here.

Read more about loadUI, and download it here: Latest loadUI version

**********************************
This is a guest article by Henrik Olsson.
About Author:
Henrik is a software developer for loadUI from Stockholm, Sweden with an academic background in Distributed Systems and a passion for usability. He is experienced in teaching and thinks that most things can be easily explained, if well modeled and accurately targeted.

Editor’s Note:
I’m glad to share this open source load testing tool with my readers. LoadUI and soapUI are the best open source software testing tools from trusted company eviware (now SmartBear). If you are thinking to introduce load testing tool on your project try loadUI and soapUI (functional testing tool).  Also I personally used soapUI  for many projects and will soon cover it in a separate article. I love the power and beauty of these open source tools!

If you used this tool and have some tips to share, please do it in below comment section. Also we would be happy to hear and answer your queries on loadUI.

---------------------
I just wanted to thank you for subscribing to SoftwareTestingHelp.com. This blog is a success because of you and your support. Here's to your success!

As always, I appreciate your valuable comments. Visit below link to post your comment:
Load Testing Using loadUI – A Free and Open Source Load Testing Tool

Thank you so much! ---------------------
Recommended: TestLodge - Online test case management tool - Web based test case management software allowing you to manage your test plans, requirements, test cases and test runs with ease.

Does that mean you should wait till your stakeholder report the performance glitches in web application you developed? Definitely not. Many testers are good at testing websites manually and they report almost every defect while testing under standard tests. BUT, when same tester performs load or stress tests they stuck either at resource (required tools) or skill level.

I suggest not to take any risk if you are committed to defect free service. Ask for required tools and train your staff for necessary skills. Today, I’m going to review load, stress and performance testing tool for websites. The tool is called WAPT – Web Application Load, Stress and Performance Testing – a cost effective and easy to learn web load testing tool.

WAPT allows you to perform website load and performance testing by creating heavy load from a single or multiple workstations. When you set and run your tests with this tool within a matter of minutes you can get performance report of your website or web application. WAPT uses powerful virtual users same as the real world users with full control over how to customize these virtual users.

Measuring website performance:

Did you ever wonder?
- How many users can work simultaneously on your website with acceptable quality of service?
- How many visitors your website can handle by day or hour?
- What is your website response time under load?

These all questions are nothing but the measure of website “performance characteristic”.

Getting Started With WAPT:

WAPT – website performance tool performs test by emulating activity of many virtual users. Each virtual user can have its own profile settings. You can have thousands of virtual users acting simultaneously on your website performing any activity like reading or writing with your web server. Once you set number of virtual users to act on your website you have option to run your tests for specified time or specified user sessions.

Analyzing the test report:
Test result consists of charts updated in real time which you can monitor when your tests are running. The final comprehensive report is provided at the end of the tests.

Here are the important parameters to be monitored on the test report:
Error Rate: Failure rate against total number of tests run. The error may be due to the high load on server or due to the network problems and timeouts.

Response Time: Obviously a great parameter to check when you run tests for website performance. This response time indicates time required by server to provide correct reply to the request.

Number of pages per second: Number of page requests successfully completed by server per second.
How to conclude performance tests?
These performance criteria change during each test-pass with different load conditions. You need to conclude what is your acceptable load limit and whether your server is able to serve this load.

E.g.: If you expect your server to handle 100 requests successfully per second then anything below this will be failure of your server which needs to be tackled.

How to Record tests:

WAPT works like any other record and playback tool but the real strength is behind it’s parametrization where you can configure any parameter from website url or user session to act as a real user.

Testing with WAPT in simple 5 steps:
Record->Configure->Verify->Execute->Analyze

WAPT uses inline Microsoft internet explorer which is used to record your interaction with website. When you record your test all dynamic parameters are recorded as static values which can be configured later while test execution. You then need to configure each user with different settings like unique sessions, number of virtual users, values for dynamic parameters etc. Once you done with recording and configuration just verify your test if it’s ready to run and then execute performance tests if everything looks ok. Finally analyze reports to decide website performance test as accepted or failed against your set of defined standards. That’s it.

WAPT is available in two versions
- Standard version (Latest WAPT 7.1)
- Professional version of this stress and performance testing tool (Latest WAPT Pro 2.1)

What WAPT Pro can do for you?
- Use several computers to generate load on website
- Measure web server performance in terms of CPU, RAM or network usage
- You can include the execution of a JavaScript code into virtual user profiles.

If you don’t want to specify every parameter manually you can use some technology specific modules to significantly improve your test experience.

Following additional modules can be downloaded and installed along with standard or professional version of WAPT:
- Module for ASP.NET testing
- Module for Adobe Flash testing
- Module for JSON format

Finally, any review can’t be complete without the list of Pros and cons.

WAPT Pros:

- Easy to install – Takes only 5 minutes to install
- Easy to use with very short learning curve
- You get run-time reports so that you can decide whether to continue the test or not, saving your big time.
- Detailed test report with graphical representation.
- Supports secure HTTPS protocol.
- 30 days free trial available!

WAPT Cons:

- Only windows platform supported to install this tool. (But you can test your website running under any OS and technology)
- No scripting ability
- It’s not free

How to try this tool?
You can download 30 day trial version of WAPT from here.

That being said WAPT makes website load, stress and performance testing super easy.

Over to You!

Which performance testing tool do you use?
Ask your queries related to WAPT tool or performance testing in comments below.

---------------------
I just wanted to thank you for subscribing to SoftwareTestingHelp.com. This blog is a success because of you and your support. Here's to your success!

As always, I appreciate your valuable comments. Visit below link to post your comment:
Web Application Load, Stress and Performance Testing Using WAPT

Thank you so much! ---------------------
Recommended: TestLodge - Online test case management tool - Web based test case management software allowing you to manage your test plans, requirements, test cases and test runs with ease.

1)  What Is KDT (Keyword Driven Testing)?
2)  Limitations of Automation testing – Why do Test Automation projects fails?
3)  How to use Keyword Driven Testing method to overcome Automation testing barriers?
4)  What is Test Language?
5)  How to use Test Language for Keyword Driven Automation Testing?
6)  Responsibilities of Functional Testers and Test Automation experts in KDT.

Please follow below link to download this free article in PDF file format:

Keyword Driven Testing – How to Overcome Barriers of Automation Testing

---------------------
I just wanted to thank you for subscribing to SoftwareTestingHelp.com. This blog is a success because of you and your support. Here's to your success!

As always, I appreciate your valuable comments. Visit below link to post your comment:
How to Use Keyword Driven Testing (KDT) to Overcome Barriers of Automation Testing

Thank you so much! ---------------------
Recommended: TestLodge - Online test case management tool - Web based test case management software allowing you to manage your test plans, requirements, test cases and test runs with ease.

Abstract
Recently, when I wanted to start my new Automated Testing Project with four resources, I thought of applying any one of the Agile methodologies. But I was not able to continue because, a series of questions were raised inside my mind. The questions are like “Is it possible to use Agile methodologies in Automated Testing?”, “Can I use traditional tools”, “Should I have to go for open-source tools”, “What are the challenges I have to face if I am implementing automation in Agile Environment”. In this article let us analyze some of challenges we face while implementing Automation with Agile methodologies. Automated testing in the Agile environment stands a risk of becoming chaotic, unstructured and uncontrolled.

Agile Projects present their own challenges to the Automation team; Unclear project scope, Multiple iterations, Minimal documentation, early and frequent Automation needs and active stakeholder involvement all demand lot of challenges from the Automation Team. Some of these challenges are:

Challenge 1: Requirement Phase
Test Automation developer captures requirements in the form of “user stories”, which are brief descriptions of customer-relevant functionality.

Each requirement has to be prioritized as follows:

High: These are mission critical requirements that absolutely have to be done in the first release
Medium: These are requirements that are important but can be worked around until implemented.
Low: These are requirements that are nice-to-have but not critical to the operation of the software.

Once priories are established, the release “iterations” are planned. Normally, each Agile release iteration takes between 1 to 3 months to deliver. Customers/software folks take liberty to make too many changes to the requirements. Sometimes, these changes are so volatile that the iterations are bumped off. These changes are greater challenges in implementing Agile Automation testing process.

Challenge 2: Selecting the Right Tools

Traditional, test-last tools with record-and-playback-features force teams to wait until after the software is done. More over, traditional test automation tools don’t work for an Agile context because they solve traditional problems, and those are different from the challenges facing Agile Automation teams. Automation in the early stages of an agile project is usually very tough, but as the system grows and evolves, some aspects settle and it becomes appropriate to deploy automation. So the choice of testing tools becomes critical for reaping the efficiency and quality benefits of agile.

Challenge 3: Script Development Phase

The Automation testers, developers, business analysts and project stakeholders all contribute to kick-off meetings where “user-stories” are selected to next sprint. Once the “user-stories” are selected for the sprint, they are used as the basis for a set of tests.

As functionality grows with each iteration, regression testing must be performed to ensure that existing functionality has not been impacted by the introduction of new functionality in each iteration cycle. The scale of the regression testing grows with each sprint and ensures that this remains a manageable task the test team use the test automation for the regression suite.

Challenge 4: Resource Management

The Agile approach requires a mixture of testing skills, that is, test resource will be required to define unclear scenarios and test cases, conduct manual testing alongside developers, write automated regression tests and execute the automated regression packages. As the project progresses, specialist skills will also be required to cover further test areas that might include integration and performance testing. There should be an appropriate mix of domain specialist who plan and gather requirements. The challenging part in the Resource management is to find out test resources with multiple skills and
allocate them.

Challenge 5: Communication

Good communication must exist among Automation testing team, developers, business analysts and stake holders. There must be highly collaborative interaction between client and the delivery teams. More client involvement implies more suggestions or changes from the client. It implies more bandwidth for communication. The key challenge is that the process should be able to capture and effectively implement all the changes and data integrity needs to be retained. In traditional testing, developers and testers are like oil and water, but in agile environment, the challenging task is that they both must work together to achieve the target.

Challenge 6: Daily Scrum Meeting

Daily Scrum Meeting is one of the key activities in Agile Process. Teams do meet for 15 minutes stand up sessions. What is the effectiveness of these meetings? How far these meetings help Automation practice Developers?

Challenge 7: Release Phase

The aim of Agile project is to deliver a basic working product as quickly as possible and then to go through a process of continual improvement. This means that there is no single release phase for a product. The challenging part lies in integration testing and acceptance testing of the product.

If we can meet these challenges in a well optimized manner, then Automated Regression Testing in Agile environment is an excellent opportunity for QA to take leadership of the agile processes. It is better placed to bridge the gap between users and developers, understand both what is required, how it can be achieved and how it can be assured prior to deployment. Automation practice should have a vested interest in both the how and the result, as well as continuing to assure that the whole evolving system meets business objectives and is fit for purpose.

About the Author:
J.B.Rajkumar has more than 15 years of experience in both Academics and Software Testing. He has worked as Corporate Trainer, Test Lead, QA Manager and QC Manager. He is a frequent speaker for International Conferences, Colleges, Universities and Software Industries. Presently he is with Automation Practice, in one of the top MNCs.

---------------------
I just wanted to thank you for subscribing to SoftwareTestingHelp.com. This blog is a success because of you and your support. Here's to your success!

As always, I appreciate your valuable comments. Visit below link to post your comment:
Automated Regression Testing Challenges in Agile Environment

Thank you so much! ---------------------
Recommended: TestLodge - Online test case management tool - Web based test case management software allowing you to manage your test plans, requirements, test cases and test runs with ease.

Learning Basics of  Rational Robot (7.0)

1. Features of Rational Robot
Rational Robot is an automated functional, regression testing tool for automating Windows, Java, IE and ERP applications under windows platform.  Rational Robot provides test cases for common objects such as menus, lists, bitmaps and specialized test cases for objects specific to the development environment. It integrates with tools like Rational Test Manager, Rational Clearquest and Requisite Pro in the Rational Unified Processor for Defect Tracking, Change Management and Requirement Traceability. It also supports UI technologies like Java, the Web, all VS.NET controls, Oracle Forms, Borland Delphi and Sybase Power Builder applications.

2. Rational Administrator
It is a tool for managing associations between Rational artifacts such as Test Datastores, Requisite Pro projects and Rose models.

• Rational Projects are created using Rational Administrator
• Users and Groups can be maintained
• Project assets can be upgraded

3. Recording Options
Using Object oriented technology, Robot identifies an object by its name property not by its location coordinates. There are two different options

• GUI  -   Functional Testing
• VU   -   Performance Testing

4. SQABasic language
SQABasic is similar to Microsoft Visual Basic. All the scripts will be in scriptname.rec format.  When you playback the script, Robot automatically compiles and runs the script, which repeats your actions and executes the verification points.

5    Shell Scripts
It is a master script that calls other automated scripts and plays them back in sequence. “callscript  test1” is a command to call script named test1. Combined into a single shell script, scripts can run in unattended mode and perform comprehensive test coverage.  It centralizes test results into one test log.

6    Low level Recording
Turn “Low Level Recording On”  in Robot during recording, mouse and keyboard actions are automatically stored in an external file.

7    Verification Points
Verification points verify that a certain action has taken place, or verify the state of an object. There are 11 Verification points in Robot

• Alpha-numeric : Verifies alphanumeric data.  Used for edit boxes, pushbuttons,    labels, text fields, etc.,
• Object Properties: Tests object attributes such as color, font and position.
• Menu: Verifies the menu values and optionally their state (enabled or disabled) of a window
• Clip Board: Verifies the contents of the windows clipboard
• Window Existence:  Tests to see if a particular window does or does not exist on the screen.
• Region Image: Graphically compares an area of the screen you specify
• Window Image: Graphically compares an entire window such as a window box.
• Object Data: Test data contents of objects(eg. Dropdown)
• File Comparison: Compares the contents of the two files (size and the contents)
• File Existence: Checks for the existence of a specified file
• Module Existence: Used to verify whether a specified module is loaded into a specified context, or loaded anywhere in memory.

When you are creating verification points, there will be two options – Wait State and expected Results.
Wait states are useful when AUT requires an unknown amount of time to complete a task. Using a wait state keeps the verification point form failing if the task is not completed immediately or if the data is not accessible immediately.
Expected Results – Click Pass or Fail in the Verification Point Name dialog box.

8    Variable Window
During debugging, if you want to examine variable and constant values, you can variables window. View->Variables.

9.    Object Mapping
If AUT contains a custom object or any object that Robot does not recognize, you
can create a custom object mapping before start recording.  By adding the object’s
class to the list of classes that Robot recognizes, and then associating the class to a
standard object type. Robot saves this custom class/object type mapping in the
project and uses it to identify the custom object during playback.

10    Debug Tools

Animate(F11) – Animation mode allows you to see each line of script as it executes.
Step Over(F10) – Use to execute a single command line within a script
Step Into(F8) – Use to being single step execution
Step Out(F7) – Use to step out of the called script and return to the calling script.
Go Until Cursor(F6) – Use to play back the active GUI script, stopping at the text cursor location.

11    Library Files and Header Files
Header files have .sbh extensions and contain the procedure declarations and global variables referred to in your script files.  There are two types of library files. Those with .sbl extensions can’t have verification points. Those with .rec extensions are stored in the project and can have verification points. Both Header and library are in \SQABAS32 in the project directory.

12    Image Masks used for dynamic objects
Image masks are used to hide an area of the screen. When you play back a script that contains an Image VP and a mask, Robot ignores the masked area when comparing actual results to the recorded baseline.

13    Data Pool
A Datapool is a test dataset that supplies data variables in a test script during playback.  Using datapools allows you to run multiple iterations of a script using different data each time.  It can be created and managed using Test Manager for data driven tests.

 14. Important Web Site for Rational Robot trial version download and Rational Robot tutorial:
http://www.ibm.com/developerworks/rational/downloads/

Hope this is a good start for those who want to learn Rational Robot.  If you have any questions please feel free to comment below.

---------------------
I just wanted to thank you for subscribing to SoftwareTestingHelp.com. This blog is a success because of you and your support. Here's to your success!

As always, I appreciate your valuable comments. Visit below link to post your comment:
Learning Basics of Rational Robot – IBM Test automation tool

Thank you so much! ---------------------
Recommended: TestLodge - Online test case management tool - Web based test case management software allowing you to manage your test plans, requirements, test cases and test runs with ease.

This is not the case always. Some times testers may add complications in testing process due to their unskilled way of working. In this post I have added most of the testing challenges created due to testing staff, developing staff, testing processes and wrong management decisions.

So here we go with the top challenges:

1) Testing the complete application:
Is it possible? I think impossible. There are millions of test combinations. It’s not possible to test each and every combination both in manual as well as in automation testing. If you try all these combinations you will never ship the product

2) Misunderstanding of company processes:
Some times you just don’t pay proper attention what the company-defined processes are and these are for what purposes. There are some myths in testers that they should only go with company processes even these processes are not applicable for their current testing scenario. This results in incomplete and inappropriate application testing.

3) Relationship with developers:
Big challenge. Requires very skilled tester to handle this relation positively and even by completing the work in testers way. There are simply hundreds of excuses developers or testers can make when they are not agree with some points. For this tester also requires good communication, troubleshooting and analyzing skill.

4) Regression testing:
When project goes on expanding the regression testing work simply becomes uncontrolled. Pressure to handle the current functionality changes, previous working functionality checks and bug tracking.

5) Lack of skilled testers:
I will call this as ‘wrong management decision’ while selecting or training testers for their project task in hand. These unskilled fellows may add more chaos than simplifying the testing work. This results into incomplete, insufficient and ad-hoc testing throughout the testing life cycle.

6) Testing always under time constraint:
Hey tester, we want to ship this product by this weekend, are you ready for completion? When this order comes from boss, tester simply focuses on task completion and not on the test coverage and quality of work. There is huge list of tasks that you need to complete within specified time. This includes writing, executing, automating and reviewing the test cases.

7) Which tests to execute first?
If you are facing the challenge stated in point no 6, then how will you take decision which test cases should be executed and with what priority? Which tests are important over others? This requires good experience to work under pressure.

8 ) Understanding the requirements:
Some times testers are responsible for communicating with customers for understanding the requirements. What if tester fails to understand the requirements? Will he be able to test the application properly? Definitely No! Testers require good listening and understanding capabilities.

9) Automation testing:
Many sub challenges – Should automate the testing work? Till what level automation should be done? Do you have sufficient and skilled resources for automation? Is time permissible for automating the test cases? Decision of automation or manual testing will need to address the pros and cons of each process.

10) Decision to stop the testing:
When to stop testing? Very difficult decision. Requires core judgment of testing processes and importance of each process. Also requires ‘on the fly’ decision ability.

11) One test team under multiple projects:
Challenging to keep track of each task. Communication challenges. Many times results in failure of one or both the projects.

12) Reuse of Test scripts:
Application development methods are changing rapidly, making it difficult to manage the test tools and test scripts. Test script migration or reuse is very essential but difficult task.

13) Testers focusing on finding easy bugs:
If organization is rewarding testers based on number of bugs (very bad approach to judge testers performance) then some testers only concentrate on finding easy bugs those don’t require deep understanding and testing. A hard or subtle bug remains unnoticed in such testing approach.

14) To cope with attrition:
Increasing salaries and benefits making many employees leave the company at very short career intervals. Managements are facing hard problems to cope with attrition rate. Challenges – New testers require project training from the beginning, complex projects are difficult to understand, delay in shipping date!

These are some top software testing challenges we face daily. Project success or failure depends largely on how you address these basic issues.

For further reference and detailed solutions on these challenges refer book “Surviving the Top Ten challenges of Software Testing” written by William E. Perry and Randall W. Rice.

Over to you:
Many of you are working in manual and/or automation testing field. Though I have addressed many of above challenges in our previous articles, I want your views on handling these software testing challenges.  Feel free to express your views in comment section below.

---------------------
I just wanted to thank you for subscribing to SoftwareTestingHelp.com. This blog is a success because of you and your support. Here's to your success!

As always, I appreciate your valuable comments. Visit below link to post your comment:
Manual and Automation testing Challenges

Thank you so much! ---------------------
Recommended: TestLodge - Online test case management tool - Web based test case management software allowing you to manage your test plans, requirements, test cases and test runs with ease.

Build Verification test is a set of tests run on every new build to verify that build is testable before it is released to test team for further testing. These test cases are core functionality test cases that ensure application is stable and can be tested thoroughly. Typically BVT process is automated. If BVT fails that build is again get assigned to developer for fix.

BVT is also called smoke testing or build acceptance testing (BAT)

New Build is checked mainly for two things:

• Build validation
• Build acceptance

Some BVT basics:

• It is a subset of tests that verify main functionalities.
• The BVT’s are typically run on daily builds and if the BVT fails the build is rejected and a new build is released after the fixes are done.
• The advantage of BVT is it saves the efforts of a test team to setup and test a build when major functionality is broken.
• Design BVTs carefully enough to cover basic functionality.
• Typically BVT should not run more than 30 minutes.
• BVT is a type of regression testing, done on each and every new build.

BVT primarily checks for the project integrity and checks whether all the modules are integrated properly or not. Module integration testing is very important when different teams develop project modules. I heard many cases of application failure due to improper module integration. Even in worst cases complete project gets scraped due to failure in module integration.

What is the main task in build release? Obviously file ‘check in’ i.e. to include all the new and modified project files associated with respective builds. BVT was primarily introduced to check initial build health i.e. to check whether – all the new and modified files are included in release, all file formats are correct, every file version and language, flags associated with each file.
These basic checks are worth before build release to test team for testing. You will save time and money by discovering the build flaws at the very beginning using BVT.

Which test cases should be included in BVT?

This is very tricky decision to take before automating the BVT task. Keep in mind that success of BVT depends on which test cases you include in BVT.

Here are some simple tips to include test cases in your BVT automation suite:

• Include only critical test cases in BVT.
• All test cases included in BVT should be stable.
• All the test cases should have known expected result.
• Make sure all included critical functionality test cases are sufficient for application test coverage.

Also do not includes modules in BVT, which are not yet stable. For some under-development features you can’t predict expected behavior as these modules are unstable and you might know some known failures before testing for these incomplete modules. There is no point using such modules or test cases in BVT.

You can make this critical functionality test cases inclusion task simple by communicating with all those involved in project development and testing life cycle. Such process should negotiate BVT test cases, which ultimately ensure BVT success. Set some BVT quality standards and these standards can be met only by analyzing major project features and scenarios.

Example: Test cases to be included in BVT for Text editor application (Some sample tests only):
1) Test case for creating text file.
2) Test cases for writing something into text editor
3) Test case for copy, cut, paste functionality of text editor
4) Test case for opening, saving, deleting text file.

These are some sample test cases, which can be marked as ‘critical’ and for every minor or major changes in application these basic critical test cases should be executed. This task can be easily accomplished by BVT.

BVT automation suits needs to be maintained and modified time-to-time. E.g. include test cases in BVT when there are new stable project modules available.

What happens when BVT suite run:
Say Build verification automation test suite executed after any new build.
1) The result of BVT execution is sent to all the email ID’s associated with that project.
2) The BVT owner (person executing and maintaining the BVT suite) inspects the result of BVT.
3) If BVT fails then BVT owner diagnose the cause of failure.
4) If the failure cause is defect in build, all the relevant information with failure logs is sent to respective developers.
5) Developer on his initial diagnostic replies to team about the failure cause. Whether this is really a bug? And if it’s a bug then what will be his bug-fixing scenario.
6) On bug fix once again BVT test suite is executed and if build passes BVT, the build is passed to test team for further detail functionality, performance and other testes.

This process gets repeated for every new build.

Why BVT or build fails?
BVT breaks sometimes. This doesn’t mean that there is always bug in the build. There are some other reasons to build fail like test case coding error, automation suite error, infrastructure error, hardware failures etc.
You need to troubleshoot the cause for the BVT break and need to take proper action after diagnosis.

Tips for BVT success:
1) Spend considerable time writing BVT test cases scripts.
2) Log as much detailed info as possible to diagnose the BVT pass or fail result. This will help developer team to debug and quickly know the failure cause.
3) Select stable test cases to include in BVT. For new features if new critical test case passes consistently on different configuration then promote this test case in your BVT suite. This will reduce the probability of frequent build failure due to new unstable modules and test cases.
4) Automate BVT process as much as possible. Right from build release process to BVT result – automate everything.
5) Have some penalties for breaking the build Some chocolates or team coffee party from developer who breaks the build will do.

Conclusion:
BVT is nothing but a set of regression test cases that are executed each time for new build. This is also called as smoke test. Build is not assigned to test team unless and until the BVT passes. BVT can be run by developer or tester and BVT result is communicated throughout the team and immediate action is taken to fix the bug if BVT fails. BVT process is typically automated by writing scripts for test cases. Only critical test cases are included in BVT. These test cases should ensure application test coverage. BVT is very effective for daily as well as long term builds. This saves significant time, cost, resources and after all no frustration of test team for incomplete build.

If you have some experience in BVT process then please share it with our readers in comments below.

If you like this post do consider subscribing to our email newsletter to ensure you will not miss any article on software testing! Click here to get article updates by email. 

---------------------
I just wanted to thank you for subscribing to SoftwareTestingHelp.com. This blog is a success because of you and your support. Here's to your success!

As always, I appreciate your valuable comments. Visit below link to post your comment:
What you need to know about BVT (Build Verification Testing)

Thank you so much! ---------------------
Recommended: TestLodge - Online test case management tool - Web based test case management software allowing you to manage your test plans, requirements, test cases and test runs with ease.

Quick Test Professional: Interview Questions and answers.

1. What are the features and benefits of Quick Test Pro(QTP)?

1. Key word driven testing
2. Suitable for both client server and web based application
3. VB script as the script language
4. Better error handling mechanism
5. Excellent data driven testing features

2. How to handle the exceptions using recovery scenario manager in QTP?

You can instruct QTP to recover unexpected events or errors that occurred in your testing environment during test run. Recovery scenario manager provides a wizard that guides you through the defining recovery scenario. Recovery scenario has three steps
1. Triggered Events
2. Recovery steps
3. Post Recovery Test-Run

3. What is the use of Text output value in QTP?

Output values enable to view the values that the application talks during run time. When parameterized, the values change for each iteration. Thus by creating output values, we can capture the values that the application takes for each run and output them to the data table.

4. How to use the Object spy in QTP 8.0 version?

There are two ways to Spy the objects in QTP
1) Thru file toolbar: In the File ToolBar click on the last toolbar button (an icon showing a person with hat).
2) Thru Object repository Dialog: In Objectrepository dialog click on the button “object spy…” In the Object spy Dialog click on the button showing hand symbol. The pointer now changes in to a hand symbol and we have to point out the object to spy the state of the object. If at all the object is not visible or window is minimized then hold the Ctrl button and activate the required window to and release the Ctrl button.

5. What is the file extension of the code file and object repository file in QTP?
File extension of
Per test object rep: filename.mtr
Shared Object rep: filename.tsr
Code file extension id: script.mts

6. Explain the concept of object repository and how QTP recognizes objects?

Object Repository: displays a tree of all objects in the current component or in the current action or entire test( depending on the object repository mode you selected).
we can view or modify the test object description of any test object in the repository or to add new objects to the repository.
Quicktest learns the default property values and determines in which test object class it fits. If it is not enough it adds assistive properties, one by one to the description until it has compiled the unique description. If no assistive properties are available, then it adds a special Ordianl identifier such as objects location on the page or in the source code.

7. What are the properties you would use for identifying a browser and page when using descriptive programming?

“name” would be another property apart from “title” that we can use. OR
We can also use the property “micClass”.
ex: Browser(“micClass:=browser”).page(“micClass:=page”)

8. What are the different scripting languages you could use when working with QTP?

You can write scripts using following languages:
Visual Basic (VB), XML, JavaScript, Java, HTML

9. Tell some commonly used Excel VBA functions.

Common functions are:
Coloring the cell, Auto fit cell, setting navigation from link in one cell to other saving

10. Explain the keyword createobject with an example.

Creates and returns a reference to an Automation object
syntax: CreateObject(servername.typename [, location])
Arguments
servername:Required. The name of the application providing the object.
typename : Required. The type or class of the object to create.
location : Optional. The name of the network server where the object is to be created.

11. Explain in brief about the QTP Automation Object Model.

Essentially all configuration and run functionality provided via the QuickTest interface is in some way represented in the QuickTest automation object model via objects, methods, and properties. Although a one-on-one comparison cannot always be made, most dialog boxes in QuickTest have a corresponding automation object, most options in dialog boxes can be set and/or retrieved using the corresponding object property, and most menu commands and other operations have corresponding automation methods. You can use the objects, methods, and properties exposed by the QuickTest automation object model, along with standard programming elements such as loops and conditional statements to design your program.

12. How to handle dynamic objects in QTP?

QTP has a unique feature called Smart Object Identification/recognition. QTP generally identifies an object by matching its test object and run time object properties. QTP may fail to recognize the dynamic objects whose properties change during run time. Hence it has an option of enabling Smart Identification, wherein it can identify the objects even if their properties changes during run time.
Check out this:
If QuickTest is unable to find any object that matches the recorded object description, or if it finds more than one object that fits the description, then QuickTest ignores the recorded description, and uses the Smart Identification mechanism to try to identify the object.
While the Smart Identification mechanism is more complex, it is more flexible, and thus, if configured logically, a Smart Identification definition can probably help QuickTest identify an object, if it is present, even when the recorded description fails.

The Smart Identification mechanism uses two types of properties:
Base filter properties – The most fundamental properties of a particular test object class; those whose values cannot be changed without changing the essence of the original object. For example, if a Web link’s tag was changed from to any other value, you could no longer call it the same object. Optional filter properties – Other properties that can help identify objects of a particular class as they are unlikely to change on a regular basis, but which can be ignored if they are no longer applicable.

13. What is a Run-Time Data Table? Where can I find and view this table?

In QTP, there is data table used, which is used at runtime.
-In QTP, select the option View->Data table.
-This is basically an excel file, which is stored in the folder of the test created, its name is Default.xls by default.

14. How does Parameterization and Data-Driving relate to each other in QTP?

To data driven we have to parameterize. i.e. we have to make the constant value as parameter, so that in each interaction(cycle) it takes a value that is supplied in run-time data table. Through parameterization only we can drive a transaction (action) with different sets of data. You know running the script with the same set of data several times is not suggested, and it’s also of no use.

15. What is the difference between Call to Action and Copy Action.?

Call to Action: The changes made in Call to Action, will be reflected in the original action (from where the script is called). But where as in Copy Action , the changes made in the script ,will not effect the original script(Action)

16. Explain the concept of how QTP identifies object.

During recording qtp looks at the object and stores it as test object. For each test object QT learns a set of default properties called mandatory properties, and look at the rest of the objects to check whether this properties are enough to uniquely identify the object. During test run, QTP searches for the run time objects that matches with the test object it learned while recording.

17. Differentiate the two Object Repository Types of QTP.

Object repository is used to store all the objects in the application being tested.
Types of object repository: Per action and shared repository.
In shared repository only one centralized repository for all the tests. where as in per action for each test a separate per action repository is created.

18. What the differences are and best practical application of Object Repository?

Per Action: For Each Action, one Object Repository is created.
Shared: One Object Repository is used by entire application

19. Explain what the difference between Shared Repository and Per Action Repository

Shared Repository: Entire application uses one Object Repository , that similar to Global GUI Map file in WinRunner
Per Action: For each Action, one Object Repository is created, like GUI map file per test in WinRunner

20. Have you ever written a compiled module? If yes tell me about some of the functions that you wrote.

Sample answer (You can tell about modules you worked on. If your answer is Yes then You should expect more questions and should be able to explain those modules in later questions): I Used the functions for Capturing the dynamic data during runtime. Function used for Capturing Desktop, browser and pages.

21. Can you do more than just capture and playback?

Sample answer (Say Yes only if you worked on): I have done Dynamically capturing the objects during runtime in which no recording, no playback and no use of repository is done AT ALL.
-It was done by the windows scripting using the DOM(Document Object Model) of the windows.

22. How to do the scripting. Are there any inbuilt functions in QTP? What is the difference between them? How to handle script issues?

Yes, there’s an in-built functionality called “Step Generator” in Insert->Step->Step Generator -F7, which will generate the scripts as you enter the appropriate steps.

23. What is the difference between check point and output value?

An output value is a value captured during the test run and entered in the run-time but to a specified location.
EX:-Location in Data Table[Global sheet / local sheet]

24. How many types of Actions are there in QTP?

There are three kinds of actions:
Non-reusable action – An action that can be called only in the test with which it is stored, and can be called only once.
Reusable action – An action that can be called multiple times by the test with which it is stored (the local test) as well as by other tests.
External action – A reusable action stored with another test. External actions are read-only in the calling test, but you can choose to use a local, editable copy of the Data Table information for the external action.

25. I want to open a Notepad window without recording a test and I do not want to use System utility Run command as well. How do I do this?

You can still make the notepad open without using the record or System utility script, just by mentioning the path of the notepad “( i.e. where the notepad.exe is stored in the system) in the “Windows Applications Tab” of the “Record and Run Settings window.

Don’t want to miss any software testing tip! Subscribe via Email.

---------------------
I just wanted to thank you for subscribing to SoftwareTestingHelp.com. This blog is a success because of you and your support. Here's to your success!

As always, I appreciate your valuable comments. Visit below link to post your comment:
Learning basics of QTP automation tool and preparation of QTP interview questions

Thank you so much! ---------------------
Recommended: TestLodge - Online test case management tool - Web based test case management software allowing you to manage your test plans, requirements, test cases and test runs with ease.

Here I am reviewing HP- QTP (QuickTest Pro) tool for Automation testing.

QTP is widely used test automation tool mainly for functional testing. QTP has many more advanced options and HP recommends that all existing and new users should begin with QTP instead of WinRunner.

Features of QTP:

• Ease of use.
• Simple interface.
• Presents the test case as a business workflow to the tester (simpler to understand).
• Uses a real programming language (Microsoft’s VBScript) with numerous resources available.
• QuickTest Pro is significantly easier for a non-technical person to adapt to and create working test cases, compared to WinRunner.
• Data table integration better and easier to use than WinRunner.
• Test Run Iterations/Data driving a test is easier and better implement with QuickTest.
• Parameterization easier than WinRunner.
• Can enhance existing QuickTest scripts without the “Application Under Test” being available; by using the ActiveScreen.
• Can create and implement the Microsoft Object Model (Outlook objects, ADO objects, FileSystem objects, supports DOM, WSH, etc.).
• Better object identification mechanism.
• Numerous existing functions available for implementation – both from within QuickTest Pro and VBScript.
• QTP supports .NET development environment
• XML support
• The Test Report is more robust in QuickTest compared to WinRunner.
• Integrates with TestDirector and WinRunner (can kick off WinRunner scripts from QuickTest).

I am getting more and more questions on QTP. So I have approached some QTP experts to answer all our readers questions. If you have any query related to QTP implementation or execution you can comment it below. All questions will be answered in coming posts.

---------------------
I just wanted to thank you for subscribing to SoftwareTestingHelp.com. This blog is a success because of you and your support. Here's to your success!

As always, I appreciate your valuable comments. Visit below link to post your comment:
QuickTest Pro – QTP Functional testing tool review

Thank you so much! ---------------------
Recommended: TestLodge - Online test case management tool - Web based test case management software allowing you to manage your test plans, requirements, test cases and test runs with ease.