Agile Methodology Risks and their Mitigation – A practical approach based on our hands-on end to end experience.
In the previous article, I attempted to authenticate whether you are convinced about using Agile or not. Remember that I didn’t attempt to impose rather presented the possible reasons that could convince the organizations to adopt Agile.
But no one knows the organization better than its key decision makers. If they don’t feel persuaded by those reasons, there’s no point following a trend or pursuing it half-heartedly.
It is your belief in the Agile manifesto that can make it possible for you to successfully adopt it as an organization. To me, the belief should be so firm that I categorize it sometimes as a parallel religion that you have to get into.
Now comes another phase.
Assuming that you are induced and have a firm belief in Agile, it is a time to explore its feasibility within your organizational context.
Your faith in Agile usefulness and its mass success cannot guarantee your success alone. It has to be compatible with your organization, project, teams and the management framework.
As I mentioned in my previous post, there are no known reasons for the absolute incompatibility of Agile with any organizational parameters. But the purpose of this feasibility study is just to validate the extent and particular form of Agile that is comparatively better suited for a project or an organization than others.
It should also include the comparative assessment of Agile with plan based methodologies for the success of an individual project.
For reference, just have a look at one of the pieces of stats on why Agile projects fail:
(Note: Click on the image for enlarged view)
Below are the few questions that will help to answer your fundamental question of whether to invest in Agile adoption or not.
Obviously, there is no explicit financial cost of moving to Agile but there is always an implicit investment involved in any radical change in the organization.
The expected difference in project value, timeline, and the quality will determine this decision.
What You Will Learn:
As we can see, Agile is 80% delivery while only 20% quota of efforts is remaining for assessment, planning, and deployment. So, it is mandatory to have your top people, authorized and responsible for delivery, on board.
They should not only be in agreement with Agile delivery rather should be excited enough to sponsor the transition.
If they are not committed towards Agile and endorsing it just for the experimentation purpose, they would most probably opt for traditional processes after few failures rather than converting them into success through retrospectives.
One of the major reasons of our successful Agile implementation was our president’s inspiration from the Scrum used in Microsoft. As Microsoft partner, he had been in close interaction with key Microsoft people who influenced his opinion about the viability of Agile more than any survey could do.
We had been discussing implementing Agile but it never started before our president himself called me and had a final whistle to kick off the mission. I presented few obstacles, constraints and the risks associated with him. And mysteriously, all of the obstacles started disappearing one by one through his executive orders. From structural changes to resource ownership to infrastructure issues to the seating arrangement, he was never reluctant.
So, if you are trying to convince your top management for moving to Agile and he wants to evaluate your claims, your chances of getting succeeded are slim.
Instead, if he is excited and passionate to make you successful by using all this means, you are likely to overcome all the hindrances if you have ‘never give up’ approach because continuous improvement and learning is the only way to find the best Agile methods for your project and there is no off-the-shelf solution available.
The top management and you just have to be in the Agile mindset all the time. Whoever in your top management is in charge of your product development, I would suggest making him the patron-in-chief of this mission.
In Figure 1, you can see a significant proportion of the survey respondents believe that their Agile adoption was not successful because of lack of management support.
Organizational culture is an under-discussed but highly relevant factor in adopting agile according to my experience. If you are not willing or capable enough to modify your power structure, then Agile adoption is barely possible in its true spirit.
The most widely used classification of organizational culture is mentioned below.
There are four types of organizational cultures varying mainly in their distribution of powers within an organization. Most of the software companies, following conventional methodologies, have “Role” culture which is also termed as a Bureaucratic culture.
This type of culture is attributed with high emphasis on following the organizational hierarchy and top-down distribution of power and information. Roles are strictly followed that causes red-tapism. Agile attempts to shift the Role, or any other prevailing culture in the organization, to Task culture as it is the only culture compatible with an Agile manifesto.
The transition from other cultures to the Task culture is a transformational change in any organization. It has to face natural resistance from the direct beneficiaries of the alternative power structures. In case of Role culture, those are mostly middle managers.
In our organization, as I had always been advocating moving to agile, I noticed that it was a trio of middle managers who felt insecure by devolving powers to the self-organizing teams, Agile/scrum master, and the product owners.
Even if the functional managers are offered the Scrum master or product owner roles, they don’t feel in command as much as they are in the development manager, QA manager, and the Business Analysis manager roles. Their ownership of resources and authority to manipulate things is compromised apparently.
I, as a QA manager, also had that initial feeling. So, it’s easier for me to confess it while if you ask other functional managers, they are not likely to admit the fact. It was my personal ambitions and orientation towards Agile that made me take up the additional role of a SCRUM master and implement Agile successfully.
If you are a startup company, this is not a major issue for you. But if you are transforming yourself, then you need to assure the functional managers that there are indirect benefits of the devolution of powers and they can perform their functions better by customizing their practices according to the individual team needs. But it’s much easier said than done.
One of the myths about Agile adoption is that there is a Yes/No answer to this question. In reality, you can neither rule out Agile nor adopt it in its entirety. Rather, you have to decide the extent to which you can implement it for different projects and also the particular flavor of Agile.
After convincing top management to move towards Agile, I was pretty excited to embed it in all the projects right away. Then there was some divine assistance that asked me to experiment with a smaller startup project first. Once, we had a success story, we intended to replicate the same practices across all projects.
Things didn’t work as we expected. Soon I realized that the feasibility of any form and extent of Agile should not be on a just organizational level but also on the project level. Even within an organization, it’s not practically possible to materialize the Agile principles as a standard across the board.
Although there are many variables of a project that affects Agile adoption but broadly speaking, it is project size and nature that have to be emphasized.
If the requirements and timelines are well-defined initially and are not likely to change much, then moving to Agile might not create a significant difference considering the risk associated with a transformational change in an organization.
Agile has been effective for those software projects in which requirements evolve over the time, which is true in most of the projects but not for all. Secondly, for large-scale projects, you have to go for large-scale Agile with higher complexity and a higher level of risks associated with it.
The project size also helps you decide the particular Agile method to adopt for a project, as shown in Figure 4, taken from an academic evaluation of different Agile methods.
(click image for enlarged view)
Since we have discussed the role of top management and middle management, it is the time to highlight the importance of two other major stakeholders’ consent in moving to Agile i.e. customers and the software development, team.
Although customers are not directly executing any development process, they are not irrelevant in this decision at all. However, he needs to be in agreement with a couple of implications of using Agile. First, is not following the contract as Bible and the other is an Agile delivery approach.
Unless your customer is not too old-school or fundamentalist, he’ll hopefully not mind using Agile since he is the main person who was comparatively overlooked in other software development methodologies due to which Agile stepped in. He can not only witness the software developing but is also participating in the development as opposed to the plan based approach where he can validate the product according to the written plan after it has been developed. But make sure that in this collaborative development, he remains as active and focused as the development team.
It varies from customer to customer. So, this has to be validated for every project. As you can see in figure 5, Agile has continuous ROI as opposed to the Waterfall where positive ROI received in the final stage.
Agile teams matter more than the development and testing teams in plan based processes. We have the concept of self-organizing teams who have to follow a high-level plan and requirements. Instead of being micro-managed, they have to get their act together to meet the targets.
For this, they have to buy-in the Agile philosophy that gives them more ownership and responsibility than they conventionally expect. We found some of the teams sufficiently excited to adopt Agile while some had a cold response.
The later ones were mostly groomed and trained in plan based culture. There were some natural issues of adaptation to the new process that we had to address. There was a resistance to change that had to be overcome in order to implement Agile successfully and make the teams own the process themselves.
A natural inertia of existing process is fine but the teams should theoretically be convinced and committed to adopt Agile so that everyday challenges can be resolved relatively easily.
The most problematic thing about Agile is its compatibility with larger and distributed teams. Since the need for collaboration is so vital to the success of Agile, it becomes impossible at times to maintain it to the required level unless your project is intrinsically small enough to afford smaller team or can be broken down into autonomous smaller units.
There have been many surveys and studies on the relationship between the success of Agile and the teams’ size or location. As you can see in figure 6, Agile delivers the most value when it is applied to co-located teams. But when the teams are geographically distributed, it has little or no edge over other processes.
That’s why I made the members of all the teams co-located with their co-workers even within the same building premises because I needed them to interact continuously rather than only during daily standups and sprint planning meetings.
Another change we did was to divide the larger team into 4 sub-units of variable but smaller sizes. Ideally, team size should not exceed 10 including all the developers and testers. Earlier on, it was simpler as the interdependence of the units was not high. But during the integration phase, we faced few issues of UX consistency as well as the teams’ different working approaches.
So, make sure that either your teams are smaller or separable. And it will be possible for you to manage the integration issues later on.
The motive of this article was not to discourage you or frighten you of Agile adoption at all. Rather, it is just to emphasize the impact of adopting Agile. It’s not just a tool, a process or methodology. It just transforms the way you think and you act while developing your products and delivering them.
So, you need to be certain about all the risks and their mitigation. You also need to be specific about the Agile practices that suit you and to what extent you should adopt each of them.
There’s an ongoing list of issues we have faced that I can communicate to you. But rather than making it my personal or organizational biography, I have tried to focus on what is valuable to you as a potential Scrum master.
These are the must-to-answer questions before you jump into the deep sea of Agile and then there’s no looking back from there.
I just don’t want you to stop and think in the middle of the road “Is Agile really feasible for me?” This might be inevitable in some of the cases but this is the best time to get this question answered.
Here’s a brief checklist you can make to ensure the feasibility of Agile in your project:
If the answers to these questions are “Yes”, you are good to go.
In the next article, we’ll describe the do’s and don’ts of Agile based on our experience and industry observation.
Author: This series of Agile development and testing articles are written by STH author Umair Khalid. He is having 10+ years of experience as a QA manager and Scrum master.
If you are thinking or in process of adopting your development and testing processes to Agile methodology and have any queries/concerns, feel free to let us know in comments below.