CMMI Appraisals – Tester’s Journey as an Appraisal Team Member (ATM)

CMMI Appraisal

Personal Note: It’s a great opportunity for any QA/Tester to be a CMMI Appraisal Team Member (ATM). ATM gets an extensive exposure of a complete CMMI model. If your organization is going for appraisal and if you get an opportunity to be an ATM, grab it. It’s an achievement that you will like to add on to your resume.

To Readers: Appraisal is a complex and time taking process. However, I have tried to describe it as simple as possible. This article will give you a brief idea about CMMI Appraisal. It’s not about CMMI model but it will explain what happens during appraisal! There may be understanding problems initially (If you are not familiar with CMMI terminology) but, until you arrive at an example below, Please keep reading until then.

Also, read => CMM levels for QA Processes

What is CMMI Appraisal?

CMMI Apraisals

An appraisal is an activity that helps us to identify the strengths and weaknesses of our organization’s processes and verify how closely those processes relate to CMMI best practices.

Flash Back: When any org decides to go for a maturity level appraisal (Commonly known as CMMI Appraisal) they contact an authorized organization from Software Engineering Institute (SEI) Partner network to do the appraisal. The Authorized agency will appoint a consultant to help/guide the org to understand CMMI, creating documents and organization policies etc. This is a very long process. It starts 1 year or 6-7 months (Depending upon the maturity level of the organizational process) before the final appraisal. After preparation, LA (Lead Appraiser) will be invited to do the appraisal.

Preparation at organization level:

Before CMMI implementation, the organization has to:

  • Identify staff to attend Intro to CMMI course
  • Create SEPT or SEPG (Software Engineering Process Team/Group) (Responsibility of SEPT/SEPG explained at the end of this article.)
  • Create organization policies.
  • Employ CMMI consultants

ATM Journey: After Intro to CMMI course you will have Appraisal Team Member (ATM) training. LA provides training of “SCAMPI Appraisal Method Training” to ATM. After training, an exam would be held, which you need to pass. A reference document called Method Definition Document (MDD) will be provided to all the ATMs, this document describes the SCAMPI. (Topics of Intro to CMMI Training course is covered at the end of this article)

Now what is “SCAMPI”, who is “LA” and what ATM does

SCAMPI: Standard CMMI Appraisal Method for Process Improvement.

Lead Appraiser (LA): Software Engineering Institute (SEI) is an Authorized SCAMPI Lead Appraiser. (Commonly known as LA. The LA announces the final findings report and signs the Maturity Level certificate. The word certificate isn’t generally used; SEI discourages the word “certification” and instead it refers to a company as having “achieved CMMI Level 5”).

What ATM does?
ATMs support the Lead Appraisers in collecting and validating evidence for Document Review, Interview of FAR (Functional Area Representative) members and preparation of other important artifacts. (More about ATM responsibilities is mentioned at the end of this article.)

Little more about CMMI: Capability Maturity Model Integration is a process improvement model developed by the Software Engineering Institute (SEI), Carnegie Mellon University.

CMMI is comprised of a set of “Process Areas.” Each Process Area is intended to be adapted to the culture and behaviours of your own company.

There are 22 Process Areas in CMMI.                                                        

CMMI Process Areas

[Source: wikipedia]

Within the Process Areas in the CMMI, there are multiple “Specific Goals (SGs)” and Specific Practices (SPs).” These practices define the expected behaviours of projects and organizations.

Process area components

Image: Process Area components

SG: A specific goal describes the unique characteristics that must be present to satisfy the process area.
GG: Generic goals are called “generic” because the same goal statement applies to multiple process areas.

Every ATM will be responsible for a Process Area allocated by LA to him/her. Suppose there are 22 ATMs, every ATM will have 1 process area of all the projects selected for Appraisal. Generally, LA forms a small group of ATMs. If suppose 20 ATMs are present, then the LA will create 10 groups (each group with two ATMs) and each group will be responsible for a few PA of all the projects.

Example:
Let me give an example to make you understand Specific Goal and Specific Practices and what is ATM’s role in the appraisal process.

Take an example of Process Area, Validation. (keeping in mind most of the readers here are Testers/QA’s)

Validation (VAL) is an Engineering process area at Maturity Level 3.

Purpose of this Process Area: The purpose of Validation is to demonstrate that a product or the product component fulfils, it’s intended use when placed in its intended environment.

There is two SG (Specific Goals) in this Process Area.

  1. SG 1 Prepare for Validation
  2. SG 2 Validate Product or Product Components

Let’s see Specific Practices by Goal. Specific Practices in SG1 mentioned below.

SG 1 Prepare for Validation

  • SP 1.1 Select products and the product components to be validated and the validation methods to be used.
  • SP 1.2 Establish and maintain the environment needed to support the validation.
  • SP 1.3 Establish and maintain procedures and criteria for validation.

It is not possible to elaborate all Specific Practices here; I will consider SP1.1 to give you an idea of the appraisal process.


  • SP 1.1 “Select products and product components to be validated and validation methods to be used”: STP (Software Test Plan) is the most suitable artifact for this SP. All Features/Component/Capability needs to be tested and the method is captured in this document.

ATM responsibilities

  • First ATM will verify if the STP is approved.
  • Approval mail or any supporting artifact for approval is checked for. E.g. affirmation from approver in MoM. If it’s in MoM, approver presence in that meeting will be checked in attendance sheet on that particular date for this meeting.
  • Review plan present in the PMP Annexure? Was the review done? Who reviews? Review date! Whether the review points are captured and closed in the review log? Closure date?
  • If the review is conducted, there should be a review-checklist.
  • Is STP baseline included? All review points were closed before the baseline! What is baseline date?
  • Whether validation method is mentioned in the STP or not?
  • What is the reference document for preparing STP? Requirement capture document (RCD) or Software Requirement Specification (SRS) or any other reference doc; if suppose RCD is the reference document, ATM checks Bi-Directional Traceability Matrix (BTM) for proper mapping of RCD to STP. All requirements should be covered properly in the STP for validation.
  • Guideline for preparing STP present? Does the organization have a standard template for STP? What is the version number of STP? (Guideline and template creation comes under different PA; I have included this point just for reference.)

Verifying Date plays an important role! Any document will be a baseline after review, so baseline date will come after review date. Similarly, the date in a checklist, review date, review point closure date will be compared; date on the respective documents should appear in a correct sequence.

I hope, now you must be having some clarity on ATM responsibility. Cross-checking all documents prepared during Software Development Life Cycle, dependency on each other followed by cross verification covers the core responsibility of an ATM. Other than this, the ATM participates in the interview of FAR members. I shall cover this topic in the future articles.

Now few more examples:

These are the generic examples to give more insight on appraisal process.

Let’s see a scenario: – While going through a weekly status meeting MoM (Minutes of a meeting) of a particular date ATM ensures on verifying two points.

  1. Risk discussed in the team meeting.
  2. One new member of the team and PM ensuring the subsequent training.

1) Risk: Risks are covered in the PA called Risk Management (RSKM).
ATM will check if this risk is mentioned in the Risk Log? If present in the Risk Log, there will be Risk Mitigation Plan or Contingency Plan? If Mitigation/ Contingency Plan are present, there may be some action item against the Plan. Now if Action item is present, would Monitoring be required? Monitoring comes under Project Monitoring and Control (PMC). ATM must have an idea about what to check and where to check.

This example shows how process areas are connected to each other and how many artifacts ATM will go through to make sure all practices are followed properly.

2) Training
All training related activity come under a Process Area called Organizational Training (OT).

Project Manager sends a request to HR department regarding training. HR will find an Internal/External trainer for the resource and conduct training. This is the general scenario, what else happens in OT?

Let’s see how OT works, suppose this financial year organization has allocated XX amount of money for training. Now HR will identify training needs, will collect the data from managers, resources and will prioritize it, along with this, HR will also identify organization’s future needs. For instance, if the organization decides to start working on communication projects along with software, either HR will hire new Resource/Managers/TL or will identify within the organization to work on this new domain.

ATM Verification Areas

  • Identification of Training needs by HR
  • Identification of Trainee
  • Identification of Trainer
  • Intimation to the manager and the trainee to block their calendar and subsequent affirmation in advance.
  • Date of request to the Admin and IT team for Infrastructure (Training Room, Laptop, Software, food, vehicle etc.)
  • Is the Attendance sheet maintained?
  • Was the after training feedback taken from trainee?
  • HR updated Org Skill matrix after training?
  • If some training is pending from a previous year, a plan is to be made to execute the same in a current year and the reason why HR did not conduct the training last year. Etc.

For sure this is not the complete list of activity for Organizational Training; other activities are also there, these are just quotes from my memory.

  • Responsibility of SEPT/SEPG (Software Engineering Process Team/Group)
    • Study organization processes
    • Write process standards
    • Develop SDLC stages and details
    • Create meeting procedures
    • Create estimation procedures
    • Create planning procedures
    • Create Quality Assurance procedures
    • Create Configuration Management Procedures
  • Intro to CMMI Training course covers following topics:-  
    •    Introduction to the Model
    •    Model-based process improvement
    •    Overview of CMMI components
    •    Institutionalization of Processes across the Organization
    •    Process areas of CMMI Model
    •    Structure for the continuous and staged representations
  • ATM responsibility
    • Each ATM is responsible for understanding the CMMI model, understanding the organizational context, and understanding and applying the SCAMPI methodology to evaluate implementation.
    • Each ATM is responsible for participating in data review, participating in interviews (e.g. identifying interview questions, taking notes, asking questions, tagging the answers to the model practices), participating in all group discussions and consensus and participating in creating the final findings report.
    • Each Appraisal Team Member will review and evaluate objective evidence of practice implementation and participate with the other Appraisal Team Members (including the Lead) in coming to a consensus on how adequately practices are implemented at the organizational level and whether or not goals, practices, and maturity levels are satisfied.

Appraisal Validity: The findings and results from a SCAMPI A Appraisal are valid for a period of not more than three years from delivery of the appraisal findings. That means after every three years, organization have to go for appraisal.

Conclusion

Now see the difference in a Tester’s activity from Pre CMMI to Post CMMI.

  • Pre CMMI: Dev team sends a mail “All defects closed, Defect sheet attached” and requests for testing.
  • Post CMMI:
    • Tester checks for code baseline followed by upload in a repository.
    • CR raised for code change and accepted by SEPT/SEPG?
    • Availability of Updated peer test report?
    • Correct Version number offered for testing?
    • The above are the prerequisites for testing.

About the author: This is a guest post by Kumar Amit Anand. The author is having more than 8 years of software testing experience in Finance and Defense domain. Having SCAMPI appraisal method training certificate, he has also participated in SEI L4 and L5 final appraisals. The author also helps his team members to understand CMMI and documentation.

I hope this was an interesting read and I appreciate if you could spare few minutes and post your valuable comments. This would help me come up with more interesting topics in the future.




Recommended reading

34 comments ↓

#1 Harshal

This is an amazing abstract description of CMMI,I been following this site from last few months and I consider this post as a milestone achievement.

Thanks a Ton for posting .

Very Important for professionals working in CMMI level 5 MNCs.

#2 Saritha

as an ATM I can understand the importance of this post. it’s perfectly put. kudos to all!

#3 Suvarna

Very good Information and Inputs. Thank you very much for the effort taken

#4 Rajeev Bhowiya

How you were able to simplify CMMI ! its nice article, i have experience of appraisal.. The way you presented is awesome… there is no article like this available on net… Congrats to SWtesting help and author !

#5 Sangeeth

An article that is direct and apt without much deviations.The content is simple and easy to understand for any beginner.
Waiting for more articles by the author in future.

#6 Ankur

do company need to repeat the whole process again after 3 years?

#7 Gentle Arsonist

Great article. Despite being in brief, this article touches all the important aspect of the CMMI. Also article gets the concept understood through examples. I would not hesitate in saying that this article has all the quality of being starting reference for newbies involved in CMMI related processes.

Kudos to author for bringing such a short and nice article about CMMI.

#8 SUNITA KATRE

Thanks for valuable information on CMMI Process area. The content written over here is systematic and easy to understand. It is helpful to understand CMMI practice for beginner working with CMMI level 5 companies.Waiting for more information…..

#9 Seshabhargavan

Its a valuable and well organized article on CMMi appraisal activities for software testing and related areas like training.
It enables a testing team to understand the sequence of activities performed and ensure adequate evidences are gathered before the start of the appraisal.

#10 Shuza Haider

Thanks for good information about CMMI Process area. The content written over here is easy to understand. It is helpful to understand CMMI practice for beginner working with CMMI companies.Waiting for more information…..// Shuza

#11 Chaitanya

Simple & effective post

Kudos!!

#12 kumar Amit Anand

@Harshal – “You’ve made my day.” Getting opening comment like this is blessing.
@Saritha – Nothing better than getting approval from an ATM. Thanks a lot.

#13 kumar Amit Anand

@Suvarna- Thanks a lot Suvarna.
@Rajeev Bhowiya- Making it simple was not easy :), Thanks for your time and comment.
@Sangeeth- Thank you Sangeeth.

#14 kumar Amit Anand

@Ankur- Well if you don’t act on final finding and don’t apply any improvement suggestion, Yes you have to repeat all this, other wise NO.

@Gentle Arsonist- Glad you liked it. Thank you.

@SUNITA KATRE and @Shuza Haider – Happy to know its good for beginner and easy to understand for all. Thank you for your time.

@Chaitanya-Thanks Chaitanya

#15 Pratap Raorane

Nice article which captures all the important aspects of ATM/CMMI in shortest possible word count.
Examples given are indeed helpful in better understanding the concept/process so as to work effectively as an ATP.

The article brought out the talent of the author of excellent and effective technical writing skills.

Expecting more articles/case studies based on ur experience/study on Testing… especially Protocol Testing, Firmware Testing and different Tools used by SW testers.

KUDOS…!! Keep it Up.

#16 Shiv shankar

Great Article.. Must Read..

The Article very well explain every flow of CMMi Appraisal,
which really motivates every tester to be a part of this process and definitely be a inspiration to the testers who is part of the CMMi L5 Company.

Thanks K Amit Anand for sharing such a useful experience.

Keep it Up :)

#17 Ajay

Good details of CMMI in simple language.

Thanks

#18 Shivakumar KR

Good explaination of CMMI, Where evryone can easily understood from scratch & deployed. thanks to author

#19 Pavithra

Very useful explanation about the CMMI…

#20 Ajay Babu

Simple, easy to understand, clear in explaining the concepts. Good article

#21 Bandaru Sreenivas

Good Article

#6 Ankur — yes after 3 years company need to go for assessment and need to repeat the whole process

#22 Liju R

Very Good article.

Thanks

#23 Anil Kumar Gupta

I being new in this domain understand model (CMMI) as it is explained in a very simple way.Hats off to Mr Kumar amit Anand and hope to have something new in future.
Thanks.

#24 suraj

I never read such a nice article.it clear my all the questions.
Thank you.

#25 Vinny

I am being nominated for ATM..and this article is definitely a kick start for me..A very detailed yet simple article covering all aspects of CMMI for a new entrant like me to know..

Thanks for this post :)

#26 ambika

Very Good Article.
Well written covering all CMMI process areas and with good examples that anyone can easily understand.

Thanks for the author
ambika

#27 Suraj Patil

Well superbly presented and covered all most topics from CMMI. I was part of CMMI from my Org it was very good to gain quality level knowledge. Hope will get more such post from you to increase level of thinking in all quality work based environment. Thanks a lot for such info.

Regards,
Suraj Patil

#28 Ujwala Reddy Sirineni

Thanks for valuable information on CMMI.Good article. It is helpful to understand CMMI practice for beginner working with CMMI level 5 companies.I was part of CMMI from my Org it was very good to gain quality level knowledge.

Regards,
Ujwala Sirineni

#29 E Kalaiselvan

People who are in beginner stage, this article would help to understand the CMMi in easier way

Nice & Very Simple, Good article

I am So Proud to be as your colleague.

#30 S Chandra Naik

I got more clarity on what is ATM role SCAMPI,how he verifies the artifacts.I request you to give some idea how the interview will be developers and testers,what question they asks

#31 Kumar Amit Anand

Hi Chandra,
Interview is nothing different, your statement should support the artifacts you have submitted.
If you are a tester, you should explain STLC, which includes, creation of STP,STC,STCR, Review logs etc.
Actually the verify that the documents produced are not fake which means every person who is appearing for interview should be aware of their process areas.
If you are unable to explain due to any reason ( i.e. u forgot, or you are nervous or due to any other reason) they will ask same from your TL if he also unable to answer they will ask same to PM.
Nothing much to worry, just keep your thoghts clear about the process. All the best.

#32 Aniruddh Borderwala

I am willing to become (Appraisal Team member), does your organization provide any training for CMMI models, if yes i have this query regarding (Appraisal Team Member), If my organization is going for appraisal and if i get an opportunity to be an ATM i will surly grab it. But what if my organization is not going for appraisal but still i want to become ATM (Appraisal Team Member) is it possible ? As i am willing to become (Appraisal Team Member). Please guide me in achieving the same.

#33 Kumar Amit Anand

Hi Aniruddh,
Good to hear you are eager to learn something new. Keep it up. I will try to answer your queries.

1.) No, our organisation doesn’t provide any training for CMMI Model. Only authorized agency can provide CMMI training. There are few in india like “QAI Global” and others.
2.) You cant be an ATM as individual. You have to be part of organization which is going for CMMI appraisal or

A.) Join any organisation which is CMMI L-4 or 5 as
SQA ( Software Quality Assurance) team member.
There you will learn about process and will come to
know how CMMI is implemented @ organisation
level. After every three years CMMI appraisal cycle
will be there at that time you will witness the
appraisal done by LA.

B.)Or you can try to get hired by any “authorized
organization from Software Engineering Institute
(SEI) Partner network which does the appraisal”.
Then they will provide you all the training and you
can work as consultant. Consultants help
organisation to achieve CMMI.

3.) I see few people are interested to understand CMMi for career prospects or to apply the learning in their organisation.

@ Vijay :- If possible can we start CMMI training here though your site ? Here they can learn
1.) CMMI Basic Concepts
2.) Key concepts about the model structure
3.) Training will include complete SDLC for better
understanding of concepts and implementation
4.) No qualification needed, freshers and
working professional both can join.

#34 venkatraman

Great and understandable information

Leave a Comment