Ad-hoc Testing: How to Find Defects Without a Formal Testing Process

The very term ad-hoc implies the lack of structure or something that is not methodical. When you talk about ad-hoc testing, it means that it is a form of a black box or behavioural testing performed without any formal process in place.

The formal process here means having the documentation like requirement documents, test plan, test cases and proper test planning in terms of its schedule and order of performed tests. Also, any actions performed during the testing are not typically documented.

This is mainly done with the aim of trying to uncover defects or flaws which cannot be captured through traditional or formal processes followed during the testing cycle. 

ad-hoc testing

As already understood, the essence of this testing lies in not having a formal or structured way of testing. When such kind of random testing techniques is performed, it is apparent that the testers perform this without any particular use case in mind with the aim to break the system.

Hence it definitely is even more obvious that such intuitive or creative testing methodology requires the tester to be extremely skilled, capable and have in-depth know-how of the system. Ad-hoc testing ensures that the testing performed is complete and is particularly very useful in determining the effectiveness of the test bucket.

Recommended reading => Exploratory Testing – How to Think Beyond Traditional Testing Boundaries?

Let’s start with an Ad-hoc testing example: 

Here is an example of how we can perform this testing when it comes to UI Wizard.

Let’s say you need to create a plan or a template for some kind of task to be performed using this UI wizard. The wizard is a series of panes which has user input like name, description, etc. As the wizard progresses: say on one of the panes, user data is to be entered which involves the UI wizard to throw a context pop-up box which adds the associated data to complete the wizard and deploy/activate the wizard.

To test this tester does his regular testing such as:

  • Complete the wizard successfully with all the valid data and create the plan.
  • Cancel the wizard mid-way.
  • Edit a created plan through the wizard.
  • Delete the created plan and see that there is no residue of it.
  • Enter a negative value in the wizard and see the appropriate error messages are seen.

Now, for the above example here are some test cases for ad-hoc tests that could be performed to uncover as many defects as possible:

  • While trying to add negative data, add certain special characters which are not restricted to see if they are handled properly. For example, sometimes wizards don’t restrict {or [ braces but in certain situations, this can conflict with code based on the language it is written in, and cause very unreliable behaviour.
  • Another test is specifically with respect to pop ups. A user can cause the pop up to launch and then try to press the backspace button on the keyboard. Many times I have observed that doing so, makes the background wizard completely disappear and the entire user data that was entered until the point the pop-up was launched, is lost!

Characteristics of ad-hoc testing:

If you note the scenarios above, you will see something very distinct characteristics about this type of testing.

They are:

  • They are always in line with the test objective. However, they are certain drastic tests performed with an intent to break the system.
  • The tester needs to have complete knowledge and awareness about the system being tested. The result of this testing finds bugs that attempt to highlight loopholes of the testing process.
  • Also looking at the above two tests, the natural reaction to it would be that – this kind of tests can be performed just once as it’s not feasible for a re-test unless there is a defect associated.

When do we do ad-hoc testing?

A million dollar question indeed!

Most of the time’s test teams are always burdened with too many features to test within limited timelines. In that limited time-span, there are lots of testing activities that are derived from the formal process that must also complete. In such situations ad-hoc testing finding its way into the testing is slim. However, from my experience, one round of ad-hoc testing can do wonders to the product quality and raise many design questions. 

Since ad-hoc testing is more of a “wild-child” testing technique that doesn’t have to be structured, the general recommendation is that it must be performed after the execution of the current test bucket is done. Another point of view is that this could be done when detailed testing cannot be performed due to less time.

In my view, I would say that ad-hoc testing can be done almost any time – in the beginning, towards the middle and towards the end! It just finds its place at any given time. However, when ad-hoc testing must be done to bring out maximum value is best judged by an experienced tester who has in-depth knowledge about the system being tested. 

When not to execute?

If the previous question was worth a million dollars, this should be worth a billion!

While we’ve established how effective and fruitful ad-hoc testing can be, as a skilled and capable tester we also need to decipher when not to invest in this type of testing. Although it is at the discretion of the tester, here are some recommendations/examples when it might not be necessary. 

  • Avoid this testing when there is a test case for which a defect exists. In such a situation there is a need to document the test case failure point and then verify/re-test the defect when it is fixed. Hence it won’t be applicable here.
  • There may also be certain scenarios where customers or clients may be invited to test the beta version of the software. In such cases, this testing should not be conducted.
  • Another scenario is when there is a very simple UI screen that is added. Traditional positive and negative testing should suffice here to bring out maximum defects.

Types of Ad-hoc testing:

Ad-hoc testing can be categorized into three categories below: 

#1. Buddy testing:

In this form of testing, there will be a test member and a development member that will be chosen to work on the same module. Just after the developer completes the unit testing, the tester and developer sit together and work on the module. This kind of testing enables the feature to be viewed in a broader scope for both parties. The developer will gain a perspective of all the different of tests the tester runs and the tester will gain a perspective of how the inherent design is which will help him avoid designing invalid scenarios, thereby preventing invalid defects. It will help one think like think the other.


#2. Pair testing:

In this testing, two testers work together on a module with the same test setup shared between them. The idea behind this form of testing to have the two testers brainstorm ideas and methods to have a number of defects. Both can share the work of testing and make necessary documentation of all observations made.

#3. Monkey testing:

This testing is mainly performed at the unit testing level. The tester parses data or tests in a completely random way to ensure that the system is able to withstand any crashes. This testing can be further classified into two categories:

Ad-hoc testing benefits:

It testing warrants the tester with a lot of power to be as creative as necessary.

This increases the testing quality and efficiency as below:

  • The biggest advantage that stands out is that a tester can find number of defects than in traditional testing because of the various innovative methods they can apply to test the software.
  • This form of testing can be applied anywhere in the SDLC; it’s not only restricted to the testing team. The developers can also conduct this testing, which would help them code better and also predict what problems might occur.
  • Can be coupled with another testing to get best results which can sometimes cut short the time needed for the regular testing. This would enable better quality test cases to be generated and better quality of the product on the whole.
  • Doesn’t mandate any documentation to be done which prevents the extra burden on the tester. A tester can concentrate on actually understanding the underlying architecture.
  • In cases when there is not much time available to test, this can prove to be very valuable in terms of test coverage and quality.

Ad-hoc testing drawbacks:

Ad-hoc testing also has a few drawbacks. Let’s take a look at some of the drawbacks that are pronounced:

Since it’s not very organized and there is no documentation mandated, the most evident problem is that the tester has to remember and recollect all the details of the ad-hoc scenarios in memory. This can be even more challenging especially in scenarios where there is a lot of interaction between different components.

  • Followed from the first point, this would also result in not being able to recreate defects in the subsequent attempts if asked for information.
  • Another very important question this brings to light is the effort accountability. Since this is not planned/ structured, there is no way to account for the time and effort invested in this kind of testing.
  • Ad-hoc testing has to only be performed by a very knowledgeable and skilled tester in the team as it demands being proactive and intuition in terms foreseeing potential defect ridden areas.

Best practices to make this testing more effective:

We’ve discussed at length the strengths and weaknesses associated with this testing.

Ideally, ad-hoc testing should find its place in the SDLC, however, if not approached in the appropriate manner it can prove to be costly and a waste of valuable testing time. So given below are a few pointers to make ad-hoc testing effective:

#1. Identify Defect prone areas:

When you have a good hold over testing a particular piece of software, you will agree that there will be certain features which are more prone to errors than the others. If you’re new to the system, then go ahead and check the features v/s defects opened against them. The number of defects in a particular feature is will show you that it’s sensitive and you should precisely choose that very area to perform ad-hoc testing. This proves to be a very time efficient way of exposing some serious defects.

#2. Building expertise:

Undoubtedly, a tester who has more experience is more intuitive and can guess where the errors might be, when compared to someone who has not much experience. I would say, experienced or not, it’s up to the individual to take the plunge and build expertise on the system that is being tested. Yes, experienced testers have an edge as their skills built up over the years come in handy, but the new testers should use this as a platform to gain as much knowledge as possible to design better ad-hoc scenarios.

#3. Create test categories:

Once you are aware of the list of features to be tested, set aside a few minutes to decide how you would categorize those features and test. For example, you should decide to test features that are most visible and most commonly used by the user before anything else, as these would seem critical to the software’s success. Then you could categorize them functionality/ priority wise and test them segment by segment.

Another example where this is particularly very important is if there is integration between components or modules. In these cases, there can be a lot of abnormalities that can occur. Using categorization would help touch upon this kind of tests at least once or twice.

#4. Have a rough plan:

Yes, yes this point might confuse you a bit as we described ad-hoc testing as testing which should have no planning or documentation. The idea here is to stick to the essence of ad-hoc testing, but still, have some rough pointers jotted down on how you plan to test.

A very basic example is that sometimes you may just not be able to remember all the tests you intend to perform. So jotting them down would ensure you don’t miss out on anything.

#5. Tools:

Let’s take an example faced very commonly by all of us. A lot of times if you observe, the testing of the functionality in itself is successful with no discrepancy reported in its behaviour. However, the logs behind the scenes could be reporting some exceptions seen which could be missed by the testers as it doesn’t hamper the test objective in any way. These could be even high in severity. Hence it’s very important for us to learn and tools which will help pinpoint this immediately.

#6. Document for more defects:

Again, I understand that this may raise some eyebrows again. Documentation doesn’t have to be detailed, but just a small note for your own reference of all the different scenarios covered, deviation in steps involved and record those defects for the particular test feature category. This will help you improve the overall test bucket as well whereby you could decide how to improvise existing test cases or add more if necessary.

Conclusion:

We’ve discussed in detail about ad-hoc testing techniques- its strengths, weaknesses, situations where it would and would not be beneficial.

This is one testing technique that guarantees to cater to and satisfy a tester’s creativity to the maximum. In my entire test career, I gain utmost satisfaction from ad-hoc testing as there’s no limit to being innovating and you only end up being more knowledgeable.

Having said that, the main thing to take back from all the above information would be to determine how to tap into ad hoc testing strengths and make it add value to the overall test process and product quality. 

About the author: This is a guest article by Sneha Nadig. She is working as a Test lead with over 7 years of experience in manual and automation testing projects.

Do you perform ad-hoc testing on your project? What are your suggestions to make ad-hoc testing successful?




Recommended reading

9 comments ↓

#1 karunaS

hi, good systematic explanation for ad hoc testing :)

#2 Henkn

very useful test technique. plays important role in testing cycle. the must thing for success is – tester’s expertise.

#3 Santosh

Good one Very usefull, by seeing this article i came to know in adhoc also there are types.

#4 Malvika

What are the two categories of Monkey Testing ?

#5 Jaya

This site is very nice. I am eager to learn lot many things.

Thanks a lot Softwaretestinghelp.com

#6 kumar amitav

i want to learn ecommerce domain knowledge with real time experience tester…please help me to guide in this…

#7 John

I suggest you get an english dictionary and find out what the term “ad hoc” really means!

#8 Kapil

Really good example,but I m not understand,plz easy example of the ad-hoc tesing..i m confuse..

#9 vishwa

if we find defects while performing adhoc testing.what we should do

Leave a Comment