Context Driven Testing: 7 Basic Principles with an Example

Let me explain the 7 Basic Principles of Context-Driven Testing with an Example:

When I drive to the airport, I usually take the freeway that will get me there in the least amount of time and avoid tolls. But one day, I took a longer local road route with a toll. I did it because I wanted some extra time with my friend on the drive, who had travelled a very long distance to spend the weekend with our family.

The ‘bad’ choice turned out to be the best one in this case and everything worked out well.

However, we should also consider this scenario. What if I was low on gas? What if I was low on cash? I would choose a different option. Why? The context.

Principles of Context Driven Testing

[image credit]

When you make decisions based on the following aspects, it is a context-driven decision:

• The people involved
• Circumstances
• Goals
• Available options
• Emotions, etc.

So, what is context-driven testing?

Context Driven Testing is a mindset shift (or School of testing) developed by Cem Kaner, James Bach & Bret Pettichord. You can read about it in detail in their famous book: Lessons Learned in Software Testing.

There are 7 basic principles to it. The following are directly picked from their book:

#1) The value of any practice depends on its context.

#2) There are good practices in context, however there are no best practices.

#3) People working together are the most important part of any project’s context.

#4) Projects unfold over time in ways that are often not predictable.

#5) The product is a solution. If the problem isn’t solved, the product doesn’t work.

#6) Good software testing is a challenging intellectual process.

#7) Only through judgement skills, exercised cooperatively throughout the entire project, are we able to do the right things at the right times to effectively test our products.

I am not going to explain each one of them because it is done for us by the experts themselves here.

I will do a scenario based explanation of my take on Context-Driven Testing.

An Example of Context Driven Testing:

Let us say that I am starting a testing project – Project A which includes end-to-end testing for a web based application.

What would be my strategy?

Based on the standard process, this will be the sequence of events:

• Gather requirements and understand the application
• Create a Test plan
• Create Test documentation – Test scenarios, Test cases, Traceability Matrix, etc.
• Have all the documentation reviewed and approved
• Set up QA environment and Test data
• Perform Test execution
• Create bug reports
• Generate and share test execution status reports, etc.

If I ask myself the question, “How did I decide this is what I need to be doing?” My answer would be, best practices, QA standards, Industry guidelines, past experience baselines, etc., right?

I am repeating what I was taught to do or what I have seen others do.

Now, is there something wrong with that? Not at all. This could even work too because there is a certain sense of repeatability and time-testedness to this approach. However, does it pave the way for optimal results?

That is Doubtful. Why?

Because with every project you are going to deal with different circumstances:

• Documented vs. undocumented requirements
• Closely working vs. geographically distributed teams
• Development and Testing teams belonging to the same company vs. competition
• Sufficient time vs.Tight schedules
• The composition of your team– Newcomers vs. experienced ones. Trained vs. untrained ones.
• Availability of tools – Manual vs. Test management tool usage
• Type of the project: Needs strict adherence to rules (FDA or banking) vs. experimental (like social media)
• Project technology. For example: you would not test the web and a windows app the same way
• Requirements for the client (some want daily detailed reports, some want just the highlights)
• Process followed (Agile vs. Traditional and scripted vs. exploratory testing)

This list is not exhaustive and you know as well as I do that there are many variables to each project.

Context-driven testing is when you let these circumstances decide your test practices, techniques, and even definitions rather than standard, industry-perceived ‘best practices’.

Now, let’s say these are the details I am working with for Project A:

• I am working with a team of 5- 4 newcomers and 1 experienced tester.
• I don’t have any documented requirements.
• My team is in India and the development team is in the US, so we work opposite time zones.
• Client wants a daily detailed status report
• We use a web based bug tracking tool, such as Mantis or Bugzilla, and that is all we have.
• I have to do 2 rounds of testing in 10 days with 3 days for test documentation

Here is a rough game plan:

1) Since lots of newcomers are on the team, we need plenty of peer reviews.

2) We also need at least 2 requirement discussion meetings with BA and Dev team. This has to be formal because the teams are located elsewhere and there is little scope for me to walk up to them with questions.

3) This is an aggressive testing timeline for documentation. The more documentation we write, the more reviews we need, which equates to more time. We will have to keep minimum documentation. We are just going to document the main End-to-End TCs and the rest will be tested exploratorily.

4) Daily status reports during test execution will be created and sent EOD daily.

5) Most testing is exploratory, so try to make a brief outline of every test executed. This way we know what is tested and what is not.

6) Defects will be reported real-time into Mantis. Since the team is working in a different time zone, they might have to wait a whole day before they hear from the QA team, in case they need a clarification. Therefore, set up an everyday call at a convenient team, where QA team will demonstrate the recreation of bugs. That way, there will be no need to wait around or follow up.

And so on.

Once you have an overall strategy, write a basic test plan explaining these points. Now, you are all set to get into a testing project after careful consideration and custom made a strategy for success.

In Summary:

This is Context-Driven testing; making your circumstances (not the standards) the primary inputs and influencers for your test strategy. We urge you to look around and take everything around you into account.

Personally, I am in love with this concept because too often testing practices are perceived as rigid and imitation-based. I will also do it as someone did it and it was successful. This is the kind of image that scares people from trying and staying in a testing career.

However, there is plenty of scope for creative thinking, analytical skills, and decision making. To learn more about this, read-up on the topic in the links provided above.

Happy context-driven-testing!

We would love to hear from you. Feel free to provide your feedback in the comments section below.