Ad-hoc Testing: How to Find Defects Without a Formal Testing Process

Ad-hoc testing is vital for comprehensive testing and helps determine the test bucket’s effectiveness. Discover the technique for Finding Defects Without a Formal Testing Process in this tutorial.

The very term ad hoc implies the lack of structure or something that is not methodical. When you talk about ad hoc testing, it means that it is a form of a black box or behavioral testing performed without any formal process in place.

The formal process here means having documentation like requirement documents, Test Plans, test cases, and proper test planning in terms of the schedule and order of performed tests. Also, any actions performed during the testing are not documented typically.

Ad-hoc Testing – Complete Guide

The main purpose of this is to uncover defects or flaws that cannot be captured through traditional or formal processes followed during the testing cycle. 

As already understood, the essence of this testing lies in not having a formal or structured way of testing. When such kind of random testing techniques are performed, it is apparent that the testers perform this without any particular use case in mind to break the system.

Hence, it is even more obvious that such intuitive or creative testing methodology requires the tester to be extremely skilled, capable, and have in-depth know-how of the system. Ad hoc testing ensures that the testing performed is complete and is very useful in determining the effectiveness of the test bucket.

Recommended reading => Exploratory Testing – How to Think Beyond Traditional Testing Boundaries?

An Ad-hoc Testing Example

Here is an example of how we can perform this testing for UI Wizard.

Let’s say you need to create a plan or a template for some kind of task to be performed using this UI wizard. The wizard is a series of panes that have user input, like name, description, etc.

As the wizard progresses: say on one pane, user data is to be entered which involves the UI wizard throwing a context pop-up box that adds the associated data to complete the wizard and deploy/activate the wizard.

To test this tester does his regular testing, such as:

• Complete the wizard successfully with all the valid data and create the plan.
• Cancel the wizard mid-way.
• Edit a created plan through the wizard.
• Delete the created plan and see that there is no residue of it.
• Enter a negative value in the wizard and see the error messages are seen.

Now, for the above example, here are some test cases for ad hoc tests that could be performed to uncover as many defects as possible:

• While trying to add negative data, we should add certain special characters that are not restricted to see if the system handles them properly. For Example, sometimes wizards don’t restrict {or [ braces, but in certain situations, this can conflict with code based on the language it is written in, and cause very unreliable behavior.
• Another test is specifically regarding pop-ups. A user can cause the pop-up to launch and then try to press the backspace button on the keyboard. Many times I have observed that doing so makes the background wizard completely disappear and the entire user data that was entered until the point the pop-up was launched is lost!

Characteristics of Ad-hoc testing:

If you note the scenarios above, you will see some very distinct characteristics of this type of testing.

They are:

• They are always in line with the test objective. However, there are certain drastic tests performed with the intent of breaking the system.
• The tester needs to have complete knowledge and awareness about the system being tested. The result finds bugs that attempt to highlight the loopholes in the testing process.
• When considering the two tests above, the logical response is that this type of test should only be done once unless there is a related defect that requires a re-test.

When Do We Do Ad-hoc Testing

A million-dollar question indeed!

Most of the time, test teams are always burdened with too many features to test within limited timelines. In that limited time, there are lots of testing activities that are derived from the formal process that must also be completed. In such situations, ad hoc testing finding its way into the testing is slim.

However, from my experience, one round of ad hoc testing can do wonders for product quality and raise many design questions. 

Since ad hoc testing is more of a “wild-child” testing technique that doesn’t have to be structured, the general recommendation is that it must be performed after the execution of the current test bucket is done. Another point of view is that this could be done when detailed testing cannot be performed because of less time.

I would say that ad hoc testing can be done almost any time – in the beginning, towards the middle, and at the end! It just finds its place at any time. However, when ad hoc testing must be done to bring out maximum value is best judged by an experienced tester who has in-depth knowledge of the system being tested. 

When not to execute?

If the previous question was worth a million dollars, this should be worth a billion!

While we’ve established how effective and fruitful ad hoc testing can be, as a skilled and capable tester, we also need to decipher when not to invest in this type of testing. Although it is at the discretion of the tester, here are some recommendations/examples when it might not be necessary. 

• Avoid this testing when there is a test case for which a defect exists. In such a situation, there is a need to document the test case failure point and then verify/re-test the defect when it is fixed. Hence, it won’t be applicable here.
• There may also be certain scenarios where customers or clients may be invited to test the beta version of the software. In such cases, this testing should not be conducted.
• Another scenario is when there is a very simple UI screen that is added. Traditional positive and negative testing should suffice here to bring out maximum defects.

Types Of Ad-hoc Testing

There are three categories below in which we can categorize ad-hoc testing:

#1) Buddy Testing

In this form of testing, there will be a test member and a development member who will be chosen to work on the same module. Just after the developer completes the unit testing, the tester and developer sit together and work on the module. This kind of testing enables the feature to be viewed in a broader scope for both parties.

The developer will gain a perspective of all the different tests the tester runs and the tester will gain a perspective of how the inherent design is which will help him avoid designing invalid scenarios, thereby preventing invalid defects. It will help one think like to think the other.

#2) Pair Testing

In this testing, two testers work together on a module with the same test setup shared between them. The idea behind this form of testing is to have the two testers brainstorm ideas and methods to have several defects. Both can share the work of testing and make necessary documentation of all observations made.

#3) Monkey Testing

Testers mainly perform this testing at the unit testing level. The tester parses data or tests completely randomly to ensure that the system can withstand any crashes. Testers further classify this testing into two categories.

Ad-hoc Testing Benefits

Testing warrants the tester with a lot of power to be as creative as necessary.

This increases the testing quality and efficiency as below:

• The biggest advantage that stands out is that a tester can find the number of defects than in traditional testing because of the various innovative methods they can apply to test the software.
• This form of testing can apply to any stage of the SDLC. It’s not only restricted to the testing team. The developers can also conduct this testing, which would help them code better and also predict what problems might occur.
• It can be coupled with other testing to get the best results, which can sometimes cut short the time needed for regular testing. This would enable better quality test cases to be generated and better quality of the product on the whole.
• It doesn’t mandate any documentation to be done which prevents the extra burden on the tester. A tester can concentrate on actually understanding the underlying architecture.
• When there is not much time available to test, this can prove very valuable in terms of test coverage and quality.

Ad-hoc Testing Drawbacks

There are also a few disadvantages to ad hoc testing. Now, let’s examine some of the notable drawbacks.

Since it’s not very organized and there is no documentation mandated, the most evident problem is that the tester has to remember and recollect all the details of the ad hoc scenarios in memory. This can be even more challenging, especially in scenarios where there is a lot of interaction between different components.

• Following from the first point, this would also result in not being able to recreate defects in the subsequent attempts if asked for information.
• Another very important question this brings to light is the effort of accountability. Since there is no planning or structure, it becomes impossible to account for the time and effort invested in this kind of testing.
• A very knowledgeable and skilled tester in the team must be the one to perform ad-hoc testing, as it requires being proactive and intuitive in terms of foreseeing potential defect-ridden areas.

Tips For Maximizing the Effectiveness of This Testing

We’ve discussed at length the strengths and weaknesses associated with this testing.

Ideally, ad hoc testing should find its place in the SDLC, however, if not approached appropriately, it can prove costly and a waste of valuable testing time. So, given below are a few pointers to make ad hoc testing effective:

#1) Identify Defect prone areas

When you have a good hold over testing a particular piece of software, you agree that there will be certain features that are more prone to errors than others. If you’re new to the system, then check the features v/s defects opened against them.

The number of defects in a particular feature will show you it’s sensitive and you should precisely choose that very area to perform ad hoc testing. This proves to be a very time-efficient way of exposing some serious defects.

#2) Building expertise

Undoubtedly, a tester who has more experience is more intuitive and can guess where the errors might be when compared to someone who has not much experience. I would say, experienced or not, it’s up to the individual to take the plunge and build expertise on the system that is being tested.

Yes, experienced testers have an edge as their skills built up over the years come in handy, but the new testers should use this as a platform to gain as much knowledge as possible to design better ad hoc scenarios.

#3) Create test categories

Once you are aware of the list of features to be tested, set aside a few minutes to decide how you would categorize those features and test them. For example, decide to test features that are most visible and most commonly used by the user before anything else, as these would seem critical to the software’s success.

Then you could categorize them functionality/priority-wise and test them segment by segment.

Another example where this is particularly very important is if there is integration between components or modules. In these cases, there can be a lot of abnormalities that can occur. Using categorization would help touch upon this kind of test at least once or twice.

#4) Have a rough plan

Yes, this point might confuse you a bit as we described ad hoc testing as testing which should have no planning or documentation. The idea here is to stick to the essence of ad hoc testing, but still have some rough pointers jotted down on how you plan to test.

A very basic example is that sometimes you may just not be able to remember all the tests you intend to perform. So jotting them down would ensure you don’t miss out on anything.

#5) Tools

Let’s take an example faced commonly by all of us. A lot of times if you observe, the testing of the functionality in itself is successful with no discrepancy reported in its behavior. However, the testers could miss some exceptions reported in the logs behind the scenes, as it doesn’t affect the test objective.

These could be even higher in severity. Hence, we need to learn tools that will help pinpoint this immediately.

#6) Document for more defects

Once more, this might cause some skepticism. Documentation doesn’t have to be detailed, but just a small note for your reference of all the different scenarios covered, deviation in steps involved, and record those defects for the particular test feature category.

This will help you improve the overall test bucket as well, whereby you could decide how to improvise existing test cases or add more if necessary.

Conclusion

We’ve discussed ad hoc testing techniques- their strengths, weaknesses, and situations where it would and would not be beneficial.

This is one testing technique that guarantees to cater to and satisfy a tester’s creativity to the maximum. In my entire test career, I have gained the utmost satisfaction from ad hoc testing as there’s no limit to innovating and you only end up being more knowledgeable.

The main thing to take back from all the above information would be to determine how to tap into ad hoc testing strengths and make it add value to the overall test process and product quality. 

About the author: This is a guest article by Sneha Nadig. She is working as a Test lead with over 7 years of experience in manual and automation testing projects.

Do you perform ad hoc testing on your project? What are your suggestions to make Ad hoc testing successful?