Concluding Thoughts about Agile Implementation in Your Organization

This article has my concluding thoughts about Agile Implementation in an organization. We will continue the discussion from our last article, Cultivating Agile in your organization and conclude the series.

This article is the concluding part of our in-depth Agile implantation series. From our last article, we are learning how to cultivate Agile in your organization.

For all those who are in the process of switching to Agile, this series is a set of some very practical tips. This is based on the personal experiences of the author during Agile implementation.

Concluding Thoughts About Agile Implementation

Agile Delivery

Agile implementation does not work in isolation for software development. It has to be aligned with Agile delivery for your products. This is one of the primary reasons why I had advised to first convince management to properly adopt Agile.

Client deliverables and releases are usually determined by stakeholders outside the development team. If they are not synchronized with the development team and their delivery cycle is following some other methodology, Agile development will never seem to be working for them or living up to their expectations.

As you can see in Figure 1, the construction phase (generally called Agile development) is regarded as an Agile methodology. But that is by far a reductionist approach. It is only one of only three broad phases of the Agile delivery cycle. I would not go into the debate of whether it is the most critical phase or not but it doesn’t go on its own, without any dispute.

Figure 1

The inception phase comprises of product ideation, modelling, setting vision and high-level release planning. It involves product managers, release managers, marketing people, architects, program managers and other similar stakeholders. The modelling in Agile inception is quite different from the traditional processes like waterfall. It is not a concrete and unified model, but rather a set of hypotheses for testing and experimentation.

So, the expected outcome of Agile delivery is a minimum viable product with valuable client feedback and some learning to evolve the product. If the expectation of stakeholders from the release solution is only meeting the actual needs of the customer, then Agile is not the answer.

Agile has proven to be more successful than traditional process models because of Agile delivery. Since traditional models used to set a rather rigid and concrete target before delivering the solution, the solution was easily categorized as a failure not being able to meet the expectations. However, Agile treats client feedback as a learning opportunity and reiterates that we want to make the product better.

Therefore, failure is not as easy to achieve in Agile delivery as in plan based models. As long as you are progressing and improving, you have an ace up your sleeve.

Collaboration

Collaboration is a concept that comes from the Agile manifesto. In fact, two of the four postulates of the Agile manifesto are related to collaboration whether it is the collaboration with the customer or within the organizations or vendors. Although it makes collaboration half the interpretation of the Agile manifesto, I perceive it as an essence of Agile.

Collaboration can be achieved in a number of ways. It ranges from conventional practises like standup meetings, sprint planning meetings and dashboards to some other innovative methods.

Effective collaboration is a bigger target to achieve among geographically distributed teams. Offshore development teams are pretty common in the software industry. You can find a lot of research material on how to achieve collaboration among scattered agile teams.

Many tools and techniques have been recommended by the practitioners but in any case, you’ll have to go through this struggle if you have the distributed setup. We were fortunate enough that most of our key team members are working in the same premises.

The first thing I did after setting up the agile teams was make sure that the team members are co-located within the office. We have cubicles shared by 4 to 6 members without any physical barriers. It helps us prominently in keeping their interaction going throughout the day beyond formal meet ups. We have injected testers and product owners among developers to break these functional walls as suggested by the Agile philosophy. We even do it for products where team members don’t gather for daily standups these days.

Obviously, it comes with a side effect of some time spent gossiping while discussing work, but that’s not a substantial price for much-needed collaboration among team members. You may try different strategies for collaboration but remember that if you are agile, you have to be collaborative as much as you can.

Figure 2

Another important front for collaboration is with the stakeholders and the customers. We have to develop the software collaboratively with the customers. It’s not just about building an internal consensus on the solution and throwing it at the customer. But you have to be extremely receptive to the customers as well as the other stakeholders who are close to them.

I am personally in favour of making product owners the proxy for collaborating with the management, customers, product managers and other external stakeholders so that the development team can focus on their respective functions. But that’s the ideal scenario.

In practice, developers and testers have to engage with other stakeholders at times because when it’s a micro level discussion, they feel more at home discussing it with the creator directly. But in either situation, the product owner has to be the person moderating all the interactions within and outside of the teams. They have to be in command and sanction every move planned for the day, the sprint or the iteration.

Empowerment

Empowerment is the political motive behind Agile philosophy.

If we analyze the traditional plan based methods, we shall find out that the bureaucratic structure and culture were one of the major reasons for the failure of those methods. Bureaucracy repels the creativity and evolution required for software development. I am of the view that those methods are suitable only for the systems that need more imitation than creativity and the end result is perfectly known at the start. But that is seldom the case with software engineering.

In Agile, you share the goals of products, sub-products, the modules and the features. You even propose solutions. But this is more of a democratic development where you must respect the intellect, knowledge and the skills of each individual. They must participate in scoping, estimations, goal setting and the business solutions apart from their core job of technical solutions. The technical lead role is more of a supervisory role and on request.

I don’t recommend just pushing the features and deadlines to the developers. He might not be equally experienced or even as skillful as some of his peers but ultimately, he has to be held responsible for his share of work. You cannot hold anyone accountable without empowering him by providing him authority for his area and letting him contribute towards the overall team goal. Allow them to take some risks and do their experimentation without fearing about the short-term failures.

Figure 3

Usually, you will not find much literature on empowerment in the Agile discourse, but I discovered it as a hidden element of Agile as an answer to my boss query: “Where is the accountability in our process?” The answer I gave was “Accountability lies in empowerment”.

Since we have empowered each individual for a particular function, area, feature or a task, they are answerable for what they own. This leads to rather flat hierarchies with the onus of goal accomplishment on teams and individuals instead of particular roles. It is also noteworthy that people should be reminded every now and then about their ownership and authorities.

I have experienced some people not personally conceiving the power and influence that Agile provides them. This leads to the need for self-empowerment which is not in our scope for this article.

Figure 4

Compensating the technical debt

Quality management is the biggest headache for Agile practitioners to date. The only argument that goes against Agile theory is how it addresses quality. So, it’s not a problem only for the Agile practitioners but for the agile theorists also to advocate and convince the world that Agile addresses quality issues as better as any other methodology.

However, to avoid the theoretical debate, I will focus on the practical issues of quality management while implementing and following Agile. The foremost challenge for assuring quality in Agile is the ever-rising technical debt that has to be controlled.

Technical debt is modestly the defects backlog accumulating with every iteration. For experienced Agile practitioners, this might be an unadventurous definition but I have taken it to make it simple at this point. In reality, technical debt means much more than this and demands little more adventure to be dealt with.

For the sake of further comprehending it, let’s take an example of a typical SCRUM. At the start of every sprint, you set a sprint goal which contains some new features, enhancements, bug fixing or some support tasks etc. The build is released at the end of the sprint which is then evaluated against sprint goal. The sprint goal has minimum acceptance criteria and so do the product backlog items (PBI) targeted in the sprint.

After a demonstration of work done by the developers, the product owner accepts the PBI and moves on to the next sprint goal. But as soon as the tester receives this build, the formal testing is started and issues begin to pile up against the PBIs marked as “Done” in the ALM tool.

Agile advises to prioritize these issues according to the impact and effort required and fix them in the next sprint. But it typically does not inspire you enough to eliminate or minimize this backlog. So, usually, the graph rises with more issues reported than fixed with testers putting all their efforts into identifying the bugs while developers spend only a fraction of their time fixing those bugs.

Although none of the agile methods explicate the techniques to deal with this, fortunately they allow you to incorporate certain practices to keep it under control.

The top strategy for damage control of this ever-spreading tumor is to revise your definition of “Done”. If you make your acceptance criteria stricter by including the level of stability for every PBI, you can push your developers to inject a lesser number of bugs and build it more mindfully.

They will try to improve the built-in quality through unit tests and code reviews. It also controls product owners’ temptation to move forward without concluding their previous work. Once you do this, you will definitely see the graph slowing down significantly on its way up. The next thing to do is to automate the regression test suite as much as possible.

Automation not only provides precision in test case execution but also provides speedy results for extensive test coverage that is difficult to achieve during manual testing. This is because there is no dedicated phase for testing. Since we have a fairly mature automation framework with a committed automation team, we are able to go the extra mile.

The test scripts have been handed over to the development team for execution before delivering the build to QA for testing. But your automation scripts have to be immaculate to make this strategy work. It adds another layer of verification to ensure that there are no significant regression issues. Our testers have the leverage to focus on their bug verification and testing new work done by the developers.

Another way to deal with this issue is to dedicate some sprints/iterations for bug fixing. Some agile practitioners consider it an anti-pattern for Agile but I am a follower of James Waletzky’s belief in this regard. He clarifies this myth in his article on MSDN i.e. “Motley says: ‘Bug fix sprints are a Scrum anti-pattern.”

I personally recommend to spend 1-2 sprints/iterations after a certain period of time to clear out this debt before you move further, especially when you are closer to the market releases. Figure 5 shows how to deal with the technical debt

Figure 5

Conclusion

Dear readers, as I am approaching the maximum length of this article, I admit that I am guilty of not discussing a number of aspects including strategies, tactics, tools, technologies, artifacts and practices.

In fact, there is so much to share that I will need a series of books rather than a few articles! But then, you have a lot of books available to learn about the theory, benefits, risks and standard flows of different Agile methods. The goal for this series was to share the typical practical issues and solutions on your way to successful Agile implementation.

The USP of this series is its freedom to communicate and advise anything related to Agile that works. Writing an academic paper after a formal research, one is bound by a certain research methodology and the narrowed down research problem.

Here, I have shared my beliefs and experiences of several years and numerous projects without worrying about formal methods of verification of my claims. Once you experiment with my recommendations, you are more than welcome to either endorse or challenge my thoughts based on your experiences.

My concluding thoughts about Agile implementation comprise of two pieces of advice.

This is a reminder that Agile was created to simplify the lives of software developers, testers, and organizations.

If learning an Agile method and implementing it are demanding that you become like Einstein, it signals that you need to be flexible and shift your creativity-discipline equilibrium more towards the creative side of Agile development.

Your transition from a plan-based approach to Agile should result in a reduction in management overhead and lessen the burden of formalities from your creative development team.

We should be mindful of eliminating the right column of the Agile manifesto i.e. processes, tools, documentation, contract negotiations and the plan. Agile was never intended to replace these elements as most beginners perceive. Rather, it originated to avoid the over-emphasis on these factors. So, it is just a misconception that they can be eradicated.

I know there are many micro-level misconceptions that have not been discussed on purpose in this piece. I did not want to confuse the beginners by opening up the debate on controversial topics. Misconceptions will arise along the way, which will need some overhauling, but it doesn’t seem to be the right time to discuss them.

I would love to hear from my readers how much this series has helped you. Since Agile is flexible, vast and multi-faceted, I expect to receive some refuting claims in response. I would also appreciate the comments more than the appreciation and endorsements of my thoughts.

I want to categorize this artifact as “Thought-provoking” instead of “a guide or Bible”.  Post your thoughts in the comments section below.  We would love to hear from you.