Concluding Thoughts about Agile Implementation in Your Organization

This is the last part in our in-depth Agile implantation series. From the last article, we are learning how to cultivate Agile in your organization.

This is the concluding part of this series. Those who are in process to switch to Agile, this series is a set of some very practical tips learned from the own experience of the author while implementing Agile by overcoming all the obstacles in the way.

Let’s continue from our last article – Cultivating Agile in your organization, before concluding the series.

Agile Delivery

The agile implementation does not work in isolation for the software development. It must be aligned with the Agile delivery for your products. This is one of the primary reasons why I advised to convince management first to adopt Agile with heart and soul.

Client deliverables and the releases are usually decided by the stakeholders outside the development team. If they are not synchronized with the development team and their delivery cycle is following some other methodology, Agile development will never seem to be working for them or living up to their expectations.

As you can see in Figure 1, the construction phase (generally called as Agile development) is regarded as an Agile methodology. But that is a reductionist approach by far. It is only one of the three broad phases of the Agile delivery cycle. I would not go into the debate of whether it is the most critical phase or not but it doesn’t go on its own, without any dispute.

Figure 1

The inception phase comprises of product ideation, modelling, setting the vision and high-level release planning. It involves product managers, release managers, marketing people, architects, program managers and other similar stakeholders. The modelling in Agile inception is quite different from the traditional processes like waterfall. It is not a concrete and unified model, rather a set of hypotheses for testing and experimentation.

So, the expected outcome of Agile delivery is a minimum viable product with a valuable client feedback and some learning to evolve the product. If the expectation of stakeholders from the release solution is only meeting the actual needs of the customer, then Agile is not the answer.

Agile delivery is the reason why Agile has proven to be more successful than traditional process models. Since traditional models used to set a rather rigid and concrete target before delivering the solution, the solution was easily categorized as a failure not being able to meet the expectations. On contrary, Agile treats the client feedback as a learning opportunity and reiterates to make the product better.

Therefore, failure is not as easy to achieve in Agile delivery as in plan based models. As long as you are progressing and improving, you have an ace up your sleeve.

Collaboration

Collaboration is a jargon that comes straight from the Agile manifesto. In fact, two of the four postulates of the Agile manifesto is related to collaboration whether it is the collaboration with the customer or within the organizations or vendors. Although it makes collaboration half interpretation of the Agile manifesto I perceive it as an essence of Agile.

Collaboration can be achieved in a number of ways. It ranges from the conventional practices like standup meetings, sprint planning meetings and dash boards to some innovative methods.

Effective collaboration is a bigger target to achieve among the geographically distributed teams. Off shore development teams is a pretty common phenomenon in the software industry. You can find a lot of research material on how to achieve collaboration among scattered agile teams.

Many tools and techniques have been recommended by the practitioners but in any case, you’ll have to go through this struggle if you have the distributed setup. We were fortunate enough that most of our key team members are working in the same premises.

The first thing I did after setting up the agile teams was making sure that the team members are co-located within the office. We have the cubicles shared by 4 to 6 members without any physical barriers. It helps us prominently in keeping their interaction going throughout the day beyond formal meet ups. We have injected the testers and the product owners among developers to break these functional walls as suggested by Agile philosophy. And we do it even for the product where team members don’t gather for daily standups these days.

Obviously, it comes with a side effect of some time spent in gossiping while discussing work but that’s not a substantial price for much-needed collaboration among team members. You may try different strategies for collaboration but remember that if you are agile, you have to be collaborative as much as you can.

Figure 2

Another important front for collaboration is with the stakeholders and the customers. We have to develop the software collaboratively with the customers. It’s not just about building an internal consensus on the solution and throwing it to the customer. But you have to be extremely receptive with the customers and other stakeholders who are closer to them.

I am personally in favor of making product owner as the proxy for collaborating with the management, customers, product managers and other external stakeholders so that development team can focus on their respective functions. But that’s the ideal scenario.

In practice, developers and the testers have to engage with other stakeholders at times because when it’s a micro level discussion, they feel more at home discussing it with the creator directly. But in either situation, product owner has to be the person moderating all the interactions within and outside of the teams. They have to be in command and sanction every move planned for the day, the sprint or the iteration.

Empowerment

Empowerment is the political motive behind Agile philosophy.

If we analyze the traditional plan based methods, we shall find out that the bureaucratic structure and culture were one of the major reasons for the failure of those methods. Bureaucracy repels the creativity and evolution required for software development. I am of the view that those methods are suitable only for the systems that need more imitation than creativity and the end result is perfectly known at the start. But that is seldom the case with software engineering.

In Agile, you share the goals of products, sub-products, the modules and the features. You propose the solutions too. But this is more of a democratic development where you must respect the intellect, knowledge and the skills of each individual. They must participate in scoping, estimations, goal setting and the business solutions apart from their core job of technical solutions. The technical lead role is more of a supervisory and on request.

It is not recommended to just push the features and the deadlines to the developers. He might not be equally experienced or even as skillful as some of his peers but ultimately, he has to be held responsible for his share of work. You cannot hold anyone accountable without empowering him by providing him authority for his area and letting him contribute towards the overall team goal. Allow them to take some risks and do their experimentation without fearing about the short-term failures.

Figure 3

Usually, you will not find much literature on empowerment in the Agile discourse but I discovered it as a hidden element of Agile as an answer to my boss query: “Where is the accountability in our process?” The only answer I had was “Accountability lies in empowerment”.

Since we have empowered each individual for a particular function, area, feature or a task, they are answerable for what they own. This leads to rather flat hierarchies with the onus of goals accomplishment on the teams and the individuals instead of particular roles. It is also noteworthy that people should be reminded every now and then about their ownership and authorities.

I have experienced some people not personally conceiving the power and influence that Agile provides them. This leads to the need for self-empowerment which is not in our scope to this article.

Figure 4

Compensating the technical debt

Quality management is the biggest headache for the Agile practitioners to date. The only argument that goes against the Agile theory is how it addresses quality. So, it’s not a problem only for the Agile practitioners but for the agile theorists also to advocate and convince the world that Agile addresses quality issues as better as any other methodology.

However, avoiding the theoretical debate, I will focus on the practical issues of quality management while implementing and following Agile. The foremost challenge for assuring quality in Agile is the ever-rising technical debt that needs to be controlled.

Technical debt is modestly the defects backlog accumulating with every iteration. For experienced Agile practitioners, this might be an unadventurous definition but I have taken it for the simplicity at this point. In real, technical debt means much more than this and demands little more adventure to be dealt with.

For the sake of further comprehending it, let’s take an example of a typical SCRUM. At the start of every sprint, you set a sprint goal which contains some new features, enhancements, bug fixing or some support tasks etc. The build is released at the end of the sprint which is evaluated against sprint goal. The sprint goal has a minimum acceptance criteria and so as the product backlog items (PBI) targeted in the sprint.

After a demonstration of work done by the developers, product owner accepts the PBI and moves on to the next sprint goal. But as soon as the tester receives this build, the formal testing is started and issues begin to pile up against the PBIs marked as “Done” in the ALM tool.

Agile advises to prioritize these issues according to their impact and effort required and fix it in the next sprint. But it typically does not inspire you enough to eliminate or minimize this backlog. So, usually, the graph rises with more issues reported than fixed with testers putting all their efforts to identify the bugs while developers spending only a fraction of their time in fixing those bugs.

Although none of the agile methods explicate the techniques to deal with this but fortunately they allow you to incorporate certain practices to keep it under control.

The top strategy for damage control of this ever-spreading tumor is to revise your definition of “Done”. If you make your acceptance criteria stricter by including the level of stability for every PBI, you can push your developers to inject a lesser number of bugs and build it more mindfully.

They will try to improve the built-in quality through unit tests and code reviews. It also controls product owners’ temptation to move forward without concluding the previous work. Once you do this, you will definitely see the graph slowing down significantly on its way up. The next thing to do is to automate the regression test suite as much as possible.

Automation does not only provides the precision in the test case execution but also provides speedy results for an extensive test coverage that is difficult to achieve during manual testing in this process because there is no dedicated phase for testing as in waterfallish methods.Since we have a fairly matured automation framework with a committed automation team, we have gone an extra mile.

The test scripts have been handed over to the development team for execution before delivering the build to QA for testing. But your automation scripts have to be immaculate to make this strategy work. It adds another layer of verification to make sure that there are no significant regression issues. Our testers have the leverage of focusing on their bugs verification and testing new work done by the developers.

Another way to deal with this issue is to dedicate some sprints/iterations for bug fixing. Some agile practitioners consider it as an anti-pattern for Agile but I am the follower of James Waletzky’s belief in this regard. He clarifies this myth in his article on MSDN i.e. “Motley says: ‘Bug fix sprints are a Scrum anti-pattern’”.

I personally recommend to spend 1-2 sprints/iterations after a certain period of time to clear out this debt before you move further, especially when you are closer to the market releases. Figure 5 shows how to deal with the technical debt

Figure 5

Conclusion: Putting it all together

Fellows! Just as I realize that I am approaching the maximum length of this article, I am feeling guilty of not communicating so many aspects including strategies, tactics, tools, technologies, artifacts and the practices.

In fact, there is so much to share that it needs a series of books rather than of articles. But then, you have tons of books available to learn the theory, benefits, risks and standard flows of different Agile methods. My goal for this serial was to share the typically practical issues and the solutions on your way to successful Agile implementation.

The USP of this serial is its freedom to communicate and advise anything related to Agile that works. Writing an academic paper after a formal research, one is bound by a certain research methodology and the narrowed down research problem.

Here, I have shared my beliefs and experiences of several years and numerous projects without worrying about formal methods of verification of my claims. Once you experiment with my recommendations, you are more than welcome to either endorse or challenge my thoughts based on your experiences.

My concluding thoughts about Agile implementation comprise of two advice.

Primary of that is a reminder that Agile was born to simplify the lives of software developers, testers, and the organizations.

If learning an Agile method and implementing it are demanding you to be Einstein, it signals that you need to be flexible and shift your creativity-discipline equilibrium more towards the creative side of Agile development.

Your transition from plan based approach to Agile should result in the reduction of management overhead and lessen the burden of formalities from your creative development team.

My secondary conclusion is another lesson learned from my long struggle along Agile. We should be mindful of eliminating the right column of Agile manifesto i.e. processes, tools, documentation, contract negotiation and the plan. Agile never intended to replace these elements as most of the beginners perceive. Rather, it originated to avoid the over-emphasis on these factors. So, it is just a misconception that they can be eradicated.

I know there are many micro-level misconceptions that have not been discussed on purpose in this piece. I did not want to confuse the beginners by opening up the debate on controversial topics. Misconceptions will arise on your way which will need some overhauling but it doesn’t look to be the right time to discuss them.

I would love to hear from my readers how much this writing helped you. Since Agile is flexible, vast and multi-facet, I expect to receive some refuting claims in response. And I would appreciate those comments more than the appreciation and endorsements of my thoughts.

I want to categorize this artifact as “Thought-provoking” instead of “The guide” or “The Bible”.