This article explains what are CMM levels and how to achieve these CMM levels for QA processes – explained with best examples.
For any process, whether it is a QA process, development process or any non-technical process, there are levels of its maturity. By levels of maturity, we mean that the level of formality and processes improvement, like ad-hoc processes – to formally defined steps – to managed result metrics – to optimization of the processes.
CMM (Capability Maturity Model) is a process-based model which is used to assess the maturity of an organization for different domains. The concept of CMM was introduced by Software Engineering Institute (SEI) in the USA.
Although this model is applied to software development process, eventually it is used for other processes as well like QA and testing.
It has 5 different levels of maturity from 1 to 5. As we go towards level 5 from 1, variability and inconsistency get reduced. Below are the details of 5 levels. Here we will go through the 5 CMM levels with respect to QA process and what all output/result is expected for each level to mature a QA/testing process and reach up to level 5.
What You Will Learn:
- Level 1 (Initial) – Ad-Hoc: Unplanned, unsystematic, and inconsistent
- Level 2 (Repeatable) – Control: Initiate defining processes at a high level:
- Level 3 (Defined) – Core Competency: Come up with a generalized process for wider audience and domains:
- Level 4 (Managed) – Predictable: Measure the processes
- Level 5 (Optimizing) – Innovative: Continuous Improvement
- Additional Info:
- Recommended Reading
Level 1 (Initial) – Ad-Hoc: Unplanned, unsystematic, and inconsistent
As the word ‘Ad-Hoc’ states: unplanned, unprepared, at this level, the significance is not given to planning, following processes, guidelines, and standards. There is no standardized & consistent way of doing any task. The only thing which is important at this level is meeting the timelines, irrespective of the quality of the end product and deliverables.
As there are no pre-defined standards and processes, the same task is done in different ways by different people.
And this becomes even more unsystematic and inconsistent if the same task is done differently next time as there are no documents available on the process which can let the process to be replicated.
So, at this level, the process is poorly controlled, unpredictable and reactive.
QA – The example would be that in an organization although QA is one of the phases in a product life cycle, there are not any standards & no defined process, no templates for QA deliverables – test plan, test strategy, test scenarios, and test cases are not standardized. Even if these things are defined & documented then all team members have their own way of doing the tasks and the process is not consistent at all. So, basically there is no control over QA and it’s a chaotic phase.
Level 2 (Repeatable) – Control: Initiate defining processes at a high level:
In this phase, we get the solution to the problem regarding the unavailability of QA processes, methodology & standards which we saw at Level 1. We have processes, methodology & standards in place. The standards and processes are not only finalized but also are well documented so that those can be re-used by any of the similar tasks which have been done previously. That is why this level is known as ‘repeatable’ – as we can repeat the steps for doing the same kind of work.
So, the focus is on basic project management at this level.
QA – Define overall QA process and methodology for different types of testing like functional, data, performance etc. Define the role & responsibilities of a QA engineer & test lead in project’s life cycle and prepare templates for deliverables in each phase. Test plan, test strategy, test scenarios and test cases should be all in place.
Not only define and prepare but also share the documentation within the team.
Level 3 (Defined) – Core Competency: Come up with a generalized process for wider audience and domains:
At level 3, people are motivated to follow the standards and processes defined at level 2. For this, first of all, the processes need to be conveyed to all the involved people. It needs to be identified that what all skills are needed to use the processes & standards effectively & efficiently and also if there is any training is required for that. Then, motivate and support the resources to follow those standards and processes. Here, the people having more experience need to share their knowledge with others.
The focus is on documentation, process standardization, and integration. By this time, the organization has developed its own standard process of software testing.
QA – Conduct webinars and training sessions to let people get acquainted with the newly defined QA process and standards and motivate them to make use of those during their day to day project work.
Level 4 (Managed) – Predictable: Measure the processes
At this level processes defined at level 3 are measured quantitatively. This is done to control the effort required on any task. Based on this quantitative analysis, processes can be adjusted if needed, and that too without degrading the quality of the end product. An analysis is done by dividing the complete process into smaller sub-processes and then quantitative techniques are applied on these sub-processes. As per the result, sub-processes are adjusted if needed. This level is called predictable because based on prior experience, we can predict the process quantitatively and make use of this prediction for the upcoming processes.
The key process areas of CMM level 4 are quantitative project management and organizational process performance.
In short, the process is measured & controlled at this level.
QA – Performing regular audits would be a good idea here. This can include checking if teams are actually following the processes defined, using the standard templates, adhere to methodology or not.
If you are into automation testing, then doing periodic code reviews of the automation test scripts would be an apt example here.
Level 5 (Optimizing) – Innovative: Continuous Improvement
At this level, innovative ways are identified to further improve the pre-defined processes and standards. This is a continuous process. For this, our own processes are watched and re-engineered continuously by adding new tools & technologies, by continuous research & studies and by keeping ourselves updated with new information in the market. This can also be achieved by benchmarking other organizations and learn from them and try to improve our process by adding new innovations to it.
So, the focus is on continuous process improvement in this level. The key process areas are organizational performance management & quantitative project management.
QA – Keep on improving the methodology, processes defined based on prior audit results.
Based on some studies it has been concluded that the organizations at level 1 may spend $1000 for any particular task then for the same task organization at level 5 needs to spend $10.
Recently in my organization, it was identified that we are doing the regression testing manually which takes manual repetition of the same kind of efforts and consumes a lot of hours which can be saved and put into some other productive works. We then did a Proof of Concept to automate the regression testing process with the help of an automation testing tool. The POC went fine and finally, we were successful in doing the regression testing through automation test scripts. This saved a lot of effort & time and contributed to overall process improvement.
After going through all 5 levels mentioned above, it looks like reaching up to level 3 is difficult. Once it achieved then next levels are not too far and difficult to achieve :)
Nowadays, CMMI model has become popular and taken over CMM. CMMI (Capability maturity model integration) is nothing but the successor of CMM. It is an integrated approach that deals with separate models of CMM and overcomes the drawback of traditional CMM. It has also got the 5 levels similar to CMM.
Below is a very useful link where you can learn the difference between CMM & CMMI and compare these two:
In the software testing market, some of the CMMI level 5 names which are worth to mention here are Capgemini India Pvt. Ltd – “FSGBU India – Development and Testing Projects”, Capita – “IT Professional Services – Testing Projects” and Infosys Public Services – “Software Development, Maintenance and Testing”.
You can get the CMMI rating of the various organizations here.
Further Reading: What is SEI? CMM? ISO? IEEE? ANSI? Will it help?
This is a guest article by Meenal Balajiwale.
Meenal is working as a Team lead in an MNC. She is specialized in overall QA process for performing functional, data, performance and security testing. Worked on Waterfall and Agile models. Have worked on BI testing, web testing, data quality as well.
Please feel free to post your queries in below comments.