Agile Testing On The Rise – Boon or Bane?

Agile – has become the trend of IT. Most of the projects are AGILE nowadays and the role of a tester is very significant.

Even before we touch on Agile Testing let us understand the concept of ‘AGILE’.

Agile is a methodology that evolved from our Iterative or Incremental methodology of development – Almost 2+ decades of Agile Manifesto. Over these decades, Agile has a very comfortable place in the hearts of Developers, PM’s and Customers. 

  • PM’s are happy as they feel it is a transparent approach and they are able to deliver what is committed in small sprints/periods
  • Customers are happy as they see their product growing in increments. They are able to change the requirements and develop what they desire to. It is flexible to allow for experimentation and faster time-to-market.
  • Developers are happy as they estimate a story, they deliver with minimum monitoring

agile testing boon or bane

Recommended read => Getting Started with Agile Scrum Methodology – The Complete Guide

To better understand the pleasant and unpleasant side of Agile, let’s look at the following financial models in which Agile usually works:

  • Variable price with fixed scope – This works well with Agile. Reason being, we can deliver the fixed scope in multiple sprints – but no customer would like it. :)
  • Fixed price with variable scope – This works best if we prioritize the product backlog with business value and agree to get valuable delivery with fixed price.
  • Fixed price, fixed scope – This is more towards the “Death-March project”.

Whether it is agreed or not, the bottom line is that it is always a fixed price project with Agile.

Let us now move on to the role of a tester in Agile:

The role of a tester in Agile is like “Expecting the Impossibilities”.

Traditionally, the target of a tester was to test the modules functionally in an independent phase provided exclusively to a tester. This was the time testers used, to explore the product and determine its stability/problems in it. They were simpler times, even though there were a few challenges in some companies.

A tester needed the following things to succeed:

  • Good interpersonal skills
  • Able to collaboratively work with cross-functional teams
  • Provide test results to make clear decisions
  • Analytical and logical thinking

While these are the common skills, Agile demands the following extra skills from a ‘TESTER’:

  • Provide speedy response to changes without business impact or delivery delays
  • Understand the user stories and their acceptance criteria

Also, it is evident that the TESTER terminology used in regular lifecycle models is totally different from AGILE.

Also with Agile, the role of developer and tester is all the same. Both play a role in SCRUM and they are treated as people with their own specialized skills who work together to deliver a quality product at the end of the sprint.

A tester’s role in Agile has evolved from the traditional methods. She/he is no longer expected to write test cases and keep executing them.

They have to take on the additional tasks of being:

1) BA – Business Analyst or Requirements Analyst – Testers are expected to provide inputs to user stories. A detailed understanding of the domain/requirement is expected and development team expects a brain-storming session by the tester.

2) Pair Programmer – Where the testers have to sit with the developers tests the story and discuss the impact of the change, if any. It is believed that the tester can add important inputs to IA (Impact Analysis).

3) Part of Test first approach – Sometimes testers are expected to wear the cap of a customer and start testing the stories even before it is developed

4) Automation Tester – Instead of repeating tasks, automating test cases help with speedy testing of repeating tasks. But, with Agile, automation teams are expected to test everything fast not just the repeated ones. So, there is much more pressure on the QA and inefficiency to the process that might force an Agile automation tester to work with unstable system

5) Exploratory Testers – Even though testers understanding the scope, working closely with developers and providing value-add to the product are all considered to be a boon in the modern IT world and without quick Automated Regression testing for the previous sprints Agile Testing never works.

Agile in itself has a chance for scope-creep in its methodology if testers fail to automate; the risks to the product’s quality are higher.

Agile testing would best work if:

  • Unit or component tests are done thoroughly
  • GUI smoke tests are done in every sprint
  • Respect the tests and testers feedback to add-on to requirements

Recommended read => Groundwork For A Successful Agile Journey

Agile Testing Quadrant:

Agile Testing Quadrant

Apparently, Agile Testing would work better if we adapt to the “Agile Testing Quadrant”.

Agile Testing – Boon or Bane?

Is it a boon or a bane – Projects implementing Agile from 2000 are yet to get the metrics baselines to substantiate the same.

While we talk about the agile tester’s roles, we should also clarify on the “SCOPE of a Tester” in Agile.

In Agile, a tester is more like a developer.

He is expected to perform unit/story based testing. This demands knowledge about development/code practices. Without this knowledge, if the tester is to make suggestions, it would not help.

Over a period of time, tester’s defects would not be considered. This is a challenge because most testers possess only domain knowledge. Currently, a tester can contribute on functional, regression or impact related testing activities. Expecting code/unit/story based testing activity is too much expectation and I am sure no tester would fit this perfectly.

Testers can play the role of a product owner and accept the stories developed. While acceptance testing the tester can enhance user stories like the product owner to make the product more likable in the market.

------------

Example: When a tester is testing a form, he can suggest/enhance features like “I would want to select the end date in a calendar field. It should not be auto-generated”.

Challenges in Agile:

Expectations should be set right when it comes to Agile Testing else there is a possibility that the testing effort goes for a toss. Some of the factors that help us understand the effectiveness for Agile for a project are:

Metrics – The phenomenal part of project management – Measure Everything That Results In Customer Satisfaction”.

Customers are more focused on the productivity of resources deployed and the effort spent by them in order to settle their bills. When it comes to costing, customers would a cost effective approach. Personally, I feel Agile Productivity is hard to measure.

Story Velocity – how many stories were planned and how much could be delivered.

Either it is poker based estimates or Fibonacci that calls for “equalization”.

  • Poker based estimates – Every user estimates the stories to be developed based on his expertise. There is an ‘Expertise” factor attached to it and that is always risky.
  • Fibonacci – This is a series of number in which each number would be the sum of two preceding numbers – When we say this it is evident that Maths can be fun but when it comes to productivity we cannot consider this to be fun as costing is involved.

Estimates cannot be considered accurate as in FP or Empirical methodology. It is all about AVERAGING there is where we will miss on “Effort spent vs stories completed” We cannot arrive at any concrete conclusion on productivity.

Also read => Automated Regression Testing Challenges in Agile Environment

To Conclude:

Agile is universally accepted and almost 2+ decade’s people have been using this terminology. But, I consider it to be a bane rather than a boon. I would rather  do iterative development and give small deliverables to my customers.

The real questions that bother me are:

  • I achieve Speed to market :) – But, do I achieve the needed productivity?
  • I deliver what the Customer agrees in bits and pieces – Do I achieve the intended purpose of the product?

The answer might mostly be YES with agile, but how do we measure it?

Also Read => How to Cultivate Agile In Your Organization – Issues and the Practical Solutions part 1 and part 2

To further back my point I want to present the following points:

PM’s in the industry feel that they have achieved iterative development and delivered what they have committed. The customer is happy. He is our boss no doubt. But shouldn’t the organization be benefited? We surely need HISTORY. We should be able to learn from our mistakes. We should be able to train the teams on what next and move the organization to next level. This is something that we tend to forget in AGILE.

Bottom line is, we are paying more attention to “I estimate I deliver” and not thinking about: “Can I do something beyond? Can I learn something beyond? How much can I do?”

Read more => 4 Steps Towards Developing the Agile Testing Mindset for Successful Transition to Agile Process

In our proven models of Software Development life cycle, we always had the ownership – I develop, I unit test and see that there is no defect leakage to the next phase. This is totally missing in AGILE. I develop; I have a tester to test my code? The developer accountability is missing.

Finally, a Tester’s life is pathetic in Agile. A tester associated with the developer to perform testing does only a black box testing as this is his area of expertise. But, the expectation is much more.

With all this in hand, it is the Organization and its Customers who have to decide on the adoption of this model and the entire UNDERSTANDING of AGILE TESTER to be clearly clarified for project’s success. Else, efforts put in by the testers would go in vain.

Too many questions unanswered…

Too many questions un-answered…

Also read => How to Eliminate Flaws of Waterfall and Agile Development Processes 

To list down on why Agile testing is a BANE for testers:

  1. A tester is not good in white-box or glass box testing – But the expectations sometimes is that they should be.
  2. The tester could contribute towards the product/project requirements. But we cannot contribute towards best practices on coding standards
  3. Pair programming – Two developers brainstorm have results – When a tester sees the code, it is always Greek and Latin – Where is the question of contributions?

To make this practice a BOON – It is still possible provided:

it is still possible provided

  1. Understanding the potential skills of a tester and defining his roles – This can take the project a very long way
  2. Treat tester only as a product owner – Let him first accept the stories developed. There is always a phase for Integration Testing where you will integrate all your stories and test – Let the developer play his/her role effectively by doing unit testing.
  3. Organizations have to be clear on agile adoption. This can work better for services mode when it is pure development. Iterative development should be stopped till prototyping phase. Through this, Organizations can control projects better and hold its HISTORY.

Iterative development

About the author: This is a guest post by VasanthiB. She is having 14 years of experience in QA and QC and currently working as a quality manager.

So, this has been my most candid opinion about Agile. What do you think? Please comment below.

Recommended reading

26 comments ↓

#1 NagiReddy S

Very nice article…

Clearly mentioned the points to benefit the Agile process based on Testers Capability.

#2 Tim Western

I love it, but at the same time wonder. Do they know what they mean when they say Agile Testing?

#3 Pallavi

Very nice article ………. People has to change their mindset on following blind things .

#4 Surya

Good one.As stated above if we follow agile testing,the product is delivered but there will not be any quality in that product .

#5 Prashant

Nice one!
But I didnt understand the financial models you’d mentioned at the starting.
The question if for the author have you ever used or been or tried AGILE ?

#6 VasanthiBaskar

Thanks for your comments NagiReddy, Pallavi and Surya

#7 VasanthiBaskar

Hi Tim,

As stated by you, PM’s or Clients tend to forget on the concept of Agile testing.

As iterated in my article just story based acceptance can be possible during story completion. Teams started using the terminology “Smoke-Functional”. Whenever we combine two types of testing, process gets diluted.

When we implement Story based testing, when there is an integration phase – defects are not accepted instead it is termed as Defect Leakage in the real-world. That’s the worry.

This is where I insist that Automation would help to avoid human slippages and this way we can also reduce our timelines/people/cost etc.,

#8 VasanthiBaskar

Hi Prashant,

Yes. I am using Agile and hence thought of this article.

Talking about costing as you must be aware we generally have three types of costing done with our Client

1. Fixed Scope with Variable Price – The scope is fixed. In this case, I can deliver based on my story velocity in a single sprint or through multiple sprints

2. Fixed Price with Variable Scope – Here the price is fixed. All we have to do is put the scope in product backlog, start planning based on business needs. This sometimes works better if the client is able to see his intended results with fixed price.

3. Fixed Price with Fixed Scope – Since the scope is defined and price is defined, as PM’s we try to squeeze resources and save money for the organization. This is something that is not accepted.

I had just mentioned project pricing to give an in-depth understanding of what works best in Agile.

#9 Sandeep

Nice Article,
Thanks for the information you have given.

Now a days so many people are asking about Agile PLM?
can you guide me what does that mean

#10 Durai

Well written.

#11 Sumit

I m working on agile scrum. Your article covered almost all the points that I came across every day per sprint. Just wanted to ask if scrum members are on personal leaves and national holidays also came in between Sprint, will it affect velocity? If we want to convince product Owner to reduce velocity, what is the ideal way? Shall we reduce velocity or not? As overcommitment may lead to sprint failure if resources are not able to deliver achievable story points. Please throw some light on this.

#12 VasanthiBaskar

Hi Durai,

Thanks.

#13 VasanthiBaskar

Hi Sandeep,

Agile PLM is nothing but product lifecycle management through Agile.

There is nothing new in-here. It is an optimized version solution that Oracle has built.

If you are a PM – you should know about Agile PLM and development to be done as expected. If you are a tester, you should know more about Performance, Capacity, Load and stress testing involved while releasing a product to market.

I hope I have cleared your doubts at a higher level.

If you want me to answer something in specific, please post back to me.

#14 Balaji

Hi Vasanthi,

Article looks really good and would be very useful
for companies trying to implement agile

#15 Mini Gupta

Thanks very nice article

#16 10G-force.com

We agree! In Agile tester is more like a developer!

#17 VasanthiBaskar

Hi Sumit,

Two things to consider:

1. When you consider your team’s velocity you obviously take into account on their leaves and national holidays. In this case the stories are also planned accordingly.

2. In Fibonacci, this estimation itself gives us enough time to close on the stories planned.

Let me know in particular on what types of estimation you are using. In both the cases the velocity is decided by us.

#18 VasanthiBaskar

Thanks to Balaji, Mini Gupta and 10G-Force for your comments

#19 sai

Hi,

Nice article. It throws light on drawbacks of Agile methodology on the overall quality of the product delivered. I personally feel (as a QA) that Agile methodology focusses on Efficiency rather than effectiveness.
Great call out on the cons. Appreciate it.

Hope organizations first understand the negative impacts/risks that Agile methodology brings in (implicitly) within it, before adopting it. Understand the priorities, risks from your respective Org or product perspective and then try exploring it. My advice to management folks is :

Please do NOT get carried away by the advantages alone and follow the rat race.
Try exploring Agile methodology for a low critical application/ product
or for a newly developed application (built from scratch).

#20 Gaurav Khrana

Nice article well balanced and give the other side of the fence about the Agile methodology and the good part is you have tried to provide the solution on how to change the bane to boon

#21 VasanthiBaskar

Thanks to Sai & Gaurav.

#22 Chris Muntzer

Excellent coverage on the advantages and pitfalls of testing in the Agile environment. I agree that it is difficult to measure results coming from Agile product development. However, it is my experience that the Agile environment breaks down silos between cross-functional team members. This enables the tester to be more involved early on, to share critical information with the developers and the rest of the team. Although the effect of this tightened collaboration is difficult to quantify, it can have a considerable impact on the success of the project.

#23 vasanthibaskar

Thanks Chris.

I agree that testing gets involved with developers at an early phase. But the fact here is functional testers do more of a User acceptance at story-point testing or at the end of sprint. Only when testers perform FUNCTIONAL/INTEGRATION testing, team would get the actual hold of project. Also, when I test at the end of every sprint I certify on the stories developed and when integrated I play a different role. Project success is possible when we automate our test scripts during AGILE so tat we don’t miss on anything.

#24 Tamara

This has a lot of great points about the ways of development using Agile methodology. I don’t know how much easier the process could be if using a product like Zephyr, enabling test tools that use customization and add-ons to integrate development and testing.

#25 anuradha

Nice Explanation . Good information included to understand agile methodology
thanks for this testing help postings.

#26 kishok

the above information is very complicated and it can be in simple english

Leave a Comment