Again let me remind you Jeet Kune Do is just a name used, a boat to get one across, and once across it is to be discarded and not to be carried on one’s back

- Bruce Lee, while describing Jeet Kune Do (JKD)

Over the last couple of months, I have been reading a large number of blogs, discussions, arguments, between purists on either side of the “Process fence” as well as middle-of-the-road liberals on what development model is best suited.

There are thousands of successful projects delivered using Agile, CMMI, SPICE, RUP and the hybrids in between, so clearly there is no single silver bullet. That begs the question, is there a single best way to develop software? Increasingly, I feel the answer is No. Even if you decide to go one way due to the needs of the project, it is not necessary to abide by a rigid set of rules prescribed in general by experts in that model.

How then do you decide how to develop software? The answer may lie in adapting JKD’s “The Way of the intercepting fist” philosophy, which, I believe, is closely aligned to Lean “thinking.”

Some thoughts:

• Start with understanding the nature of the project, the clients, the project teams and management needs
• Let this understanding drive the selection of the overall framework. For instance, if the project is to develop against an evolving standard, one of the Agile methods may be suitable. But if the team is not mature in terms of development practices, one of the iterative-but-process-oriented methods like the 2I or RUP may be better
• For engineering practices, use concepts and tools proven within the organization
• For project management, the established way within the organization or PMI’s methodology would be a good start
• Don’t be afraid in discarding practices if they are not useful, but be sure to substitute them with more useful ones. The easiest example I could think of is integration. A big-bang integration is not the norm these days, even in most process-oriented shops, but you can institute multiple daily builds instead of weekly or monthly ones. Another example is using Failure Modes Analysis (FMEA) to drive design decisions in iterations to prevent too much refactoring

There must be a term in psychology to describe this behavior: the moment we associate ourselves with something, we start believing that we must adhere to it 10o%, else the skies will fall on our heads!

As project managers, we must be careful not to fall into this trap, but carry a “toolbox” of practices which can be used in different situations. I am told that successful managers have this toolbox subconsciously, but are unable to spell it out exactly.

What about you? Do you think it is advisable to go with a single system of tried-and-trusted practices or have an assorted toolbox?

Related posts:

1. Practitioner-led Process Improvement:For “sticky” processes
2. PMO Series: Change Management

The 3 most common ways you can fail at Process improvement, that I have personally seen are:

1. Blind Adherence to Frameworks

Process frameworks such as CMMI or SPICE are intended as guiding principles to help

you develop processes that, in turn, help you achieve your business goals. Trying to incorporate each and every aspect of a framework without regards to its applicability in your organization can result in bloated processes that no one can or will follow.

Usually, process improvement begins with Gap Analysis of existing processes against the chosen Process Framework. This is where you can take a wrong turn.

Understand the intent behind the Framework when performing the Gap Analysis. One tried-and-trusted practice is to extract the objectives from the process (in CMMI, these are neatly defined in the Specific Goals) and then use them to see if the goals are met by your current processes.

2. Committees

Yes, Practitioner-led process improvement is the way to go. But that doesn’t mean that you try to get “representation from every section” of people into your process definition team! After a certain threshold, an increase in the number of people is only going to hold you back.

From my experience, the magic range is 3-5, but that may vary if you are in a business process improvement scenario. One symptom of a dysfunctional large team is endless bickering over every word and its intent when drafting the process – in one case, there was a 2-hour discussion on using “measurements” vs “Metrics”!

3. Endless Alternates

This can be a standalone issue or an offshoot of the first two. I have seen teams debate over different scenarios which they might face, when the probability of these scenarios occurring is really small! Usually, there are a few critical paths where the teams might need direction and then there is a catch-all section (which is usually an escalation path for more guidance).

This situation arises when

• People don’t trust themselves and others to do the right thing and/or
• Managers are not very forgiving when their teams take decisions using judgement and then mistakes happen. Only a learning environment is a growing environment and without it, you cannot help people become mature. Without mature people, you cannot have a high-maturity organization!

Yes, there are more ways in which you may fail, but I’ll follow my own advice and stop from listing out all other scenarios . That, however, does not prevent you from adding your own experiences in the comments. They may be a bigger cause of failure!

Do share in your experiences in process improvement and of course, subscribe to the blog.

Related posts:

1. Practitioner-led Process Improvement:For “sticky” processes
2. Process – How to develop one that is not stupid
3. Defining a Process? Use the Six Honest Serving Men

[In industry jargon, Practitioners are the people who perform the tasks indicated by the process - software development teams, for example.]

Why Practitioners have to participate in process definition? Some common objections encountered are:

• They are not experts in process development
• It is not in their job description or they have other work to do
• If they do it, why do we need process specialists?

While these are valid to some extent, lack of ownership of the very people who have to use the process is the single biggest reason for failure of processes. This isolationist, ivory-tower approach results in processes that are out of touch with reality, do not take into account established practices and a general feeling of “process policing” among the development and project management community.

Most people that I encounter, including die-hard Agile champions, agree that some agreement on how things will be done is necessary when such activities involve many people. A process is such an agreement. When we trust people to develop mission-critical software for us, it is foolish to think that they cannot define an effective way of doing things!

Are you convinced yet? If yes, let us move on and see how we can implement this effective means of defining processes. The title points below are meaningful enough without me trying to elaborate on them.

• Assemble the right team
• Identifying process requirements
• Identifying current practices
• Defining the process
• Piloting
• Implementing

Assembling the right team is probably the most important part of this whole exercise. You need to bring in people of all kinds. You need process champions, critics as well as technical experts.

What about you? You are there to assist them in wording the process, doing the documentation work, creating simple flows and probably to see that process requirements are defined right.

One other thing I have found helpful is to incorporate as many current practices, documents and tools as possible. To do this, you as the process specialist have to talk to people, do the research and generally make it easy for the team to define the process.

A good Practitioner-led process improvement initiative reduces the inertia and encourages others to follow what has been defined by their fellow clan members.

In fact, many guidelines from the SEI show that the use of practitioner-led process improvement journeys lead to sustained improvements in appraisal ratings as well as in achieving project maturity.

Share with me your stories, criticisms and your experiences in the comments below.

Related posts:

1. 3 Ways Your Process Improvement Initiatives Can Go Wrong
2. Process – How to develop one that is not stupid
3. The Model does not matter: Projects and JKD

Why: As I mentioned in a previous post, without stating clearly why we are doing something, it is pretty difficult to convince people to even read something, forget about following it. The Why is often stated in grand terms, and since people are not stupid, they understand it is just for the sake of having it! Have a simple description of the outcome and why following it will help them (in their daily life and not something like, “It will make the organization compliant with the GRAND THEORY OF NOTHING”. Who cares?)

What: What defines the activities that need to be completed to achieve a certain goal. The inputs for this can be based on existing team practices or from best practices in other teams or (God forbid!) from frameworks.

When: When here does not relate to time, but the sequence in which the above activities should be performed.

Who: Code does not get written just because we have defined “Write code”. Someone needs to write it and someone else needs to test it. Mr. Who helps us identify the people to perform the activities in the desired sequence.

Where: This is the easy one. You ask me to enter the bug – fine. Where? You get the idea (please don’t create forms for every small bit of information!  Funny one here – http://www.bureauofcommunication.com/compose/romanticintent)

How: This is the most difficult one. If you list out how to do an activity in great detail, your process will be cumbersome and if you don’t give any details, it will not be useful. Err on the less side, since you can always add detail. This is easy to say, but difficult to implement and unfortunately, the answer for how much is, “It depends, at least for me. By the way, do you have any good principles for this? Share with me.

One of the easiest ways to spell out a process is the System flow chart or activity diagram coupled with annotations for inputs/outputs of the process. This makes the process simple, visual and clear.

If you are wondering if this can be used in Agile methods (not using Agile process, in case you are offended. Ha.), the answer is yes. A process is just a structured method of representing what needs to be done and does not mean it needs to be “heavy”; it can be as light as you need. If you come from a process mature organization and want to find out if you can use Agile methods or not, read this interview and then buy the book (disclaimer: I am not affiliated in any way with the author and have not yet read the book completely).

Go on, become a process specialist. May the Force be with you.

Related posts:

1. Process – How to develop one that is not stupid
2. 3 Ways Your Process Improvement Initiatives Can Go Wrong

Lagging or Validation Metrics

These are “after the fact” metrics to show how we performed and if everything is going smoothly. These “validate” the current processes and may show deviation from the expected value. Typically, there is nothing we can do about the completed activities, but can take corrective actions for the next set of activities.

This is generally ok for repeated activities, such as bug-fixing, but not for activities that are unique in the lifecycle. For example, if we have metrics for measuring quality of design, having a report on the defects after the design phase (design defect density, phase containment defects etc) is not terribly useful (even in iterative projects).

Leading or Improvement metrics

These metrics give us early information on the quality or progress of activities that are currently being performed. Considering the same Design phase example, if we can get inputs on the quality of design as it is currently being done, there is scope to make corrections to the process. In software development, these tend to be more of engineering metrics, such as cohesion, coupling etc.

When defining a metrics program, we should be careful in having a balance between these two types of metrics. If Lagging metrics dominate, there is not much you can do if something goes wrong, except as Lessons learned (which I doubt are learned!). If leading metrics dominate, you don’t know how you performed against overall targets or compare against previous projects.

You must be wondering – what has satellite images of a storm got to do with metrics? An analogy that is possibly distant, but the intention is this – Your metrics must be able to show all 3 aspects – what has happened, where do we stand, what is going to happen in the immediate future. If you have defined the right metrics (its easy to define a lot of wrong metrics!), you will then be able to decide how to act.

If you are thinking on where to capture information on the type of metric, the Metrics catalog is a good place to put it in.

Related posts:

1. Metrics Definition – Gaining Agreement from your Stakeholders – Part 1
2. Metrics Definition – Gaining Agreement from your Stakeholders – Part 2
3. 4 different ways to use your Metrics

What is about the word “Process” that makes people run fro cover? I see a lot of people who turn to Agile, not because they realize its worth, but because they think it frees them to do what they want, how they want. Agile is discipline, folks and you can do it only if you have the maturity to handle that discipline on your own.

Here,  I am defending a process that is simple, provides clear roles and responsibilities, is changed appropriately when needed and most importantly, the one that is applied well. I am not defending a rule book written in circa 1800.

I hear you. You are saying that process as I defined is not the one that you see everyday in countless organizations. Right. But I ask you, is that the fault of the process? Did it slowly creep into your organizations? No, we wrote it. We are the ones implementing it. We are the ones hiding behind it.

A process is nothing but a set of steps we write to accomplish a particular task. We write it down so that others may follow the path easily and when we have thousands of people doing the same task, we don’t want everyone to do it differently, unless circumstances require.

These exceptional circumstances occur more frequently in the SW dev world than in other places like manufacturing (that is why concepts in manufacturing don’t lend themselves well to SW, but that is for another day).

So what do we need? We should define processes that result in desired outcomes. We need to create awareness that a process needs to be adopted and adapted as the situation changes to make it effective. Let us look at some things that we can do to define a process that is not stupid.

1. Define the Why

There is a better chance that people will follow the process if it is clear why and the why is “reasonable”

For example, for software configuration management, the “why” is to ensure that code is checked-in frequently (to avoid crashes or overwrites), follow naming conventions (to easily determine what the file does/belongs by looking at the file name), follow the structure (to modularize code) etc.

2. Minimize Forms

If people have to fill out forms for every step of the process, be sure that will never be followed. While some amount of paperwork is necessary, it should not detract from actually performing the task. The biggest mistake I see people make is trying to get a lot of information filled out, even when that information is not required or may not be used. There are two reasons for this:

• Incorrect interpretation of process frameworks like CMMI or ISO
• When something goes wrong, find out where and why. The failure points are minimal, but to have some documentation for those failure points

3. Practitioner-led

Every activity, whether defined by a process or not, has to be done by someone. Why not ask those people to define the process? It is well-recognized that when a team of practitioners own the process, the adoption is much higher. Do we want someone sitting in an ivory tower to tell me what to do? Nah.

4. Automate

Look for ways to automate as much of the process as possible. If you are capturing information, don’t provide forms for people to capture data. Use tools or write scripts to capture and present that information.

5. Teach people to change it

Make it clear that the process is not sacred. When people start to feel it is not working or not comfortable any longer, change it. Fiercely implement a culture where people can voice their concerns about something that is not working for them.

6. Never talk about the stick

We all know about the “Carrot and Stick”. In this case, however, never ever use the stick. Are we cattle to be driven ahead? Lead us and we’ll follow. What this means is don’t penalize when someone does not follow it. It could be that the process is not usable under stressful situations or the leadership does not care about it. If the manager asks the team to forget about the process just for this urgent request, it shows that the manager does not think the process is useful!

If we follow some of the common-sense principles, any process can be made simple and usable. Otherwise, we will end up defining a complex “Agile” process!

Do you still think Process is stupid? What would you do to help people follow a method that can help them be productive, yet have fun?

Related posts:

1. Defining a Process? Use the Six Honest Serving Men
2. 3 Ways Your Process Improvement Initiatives Can Go Wrong
3. Practitioner-led Process Improvement:For “sticky” processes

The SEI has published a seminal report (although its around 150 pages only), comparing the use of statistical methods and models with high-maturity levels from 2008 and 2009 surveys. The work, as expected, has a lot of details, including validation of the results using statistical analysis!

Key Takeaways:

1. Process Performance Models are used extensively in the areas of defect prediction, cost/schedule performance, estimation accuracy while other areas are relatively low. Interesting: Models for Customer satisfaction are less frequent

2. Many organizations use optimization techniques when building/using process performance models. Monte Carlo simulation and use of probabilistic modeling have grown. Interesting: Other techniques have reduced in popularity, while “don’t know” responses have increased!

3. Level of stakeholder involvement in measurement and analysis is along expected lines with measurement specialists having a high level of involvement. Interesting: It is not clear if all organizations have dedicated measurement specialists or process engineers take on the role as needed. Customer involvement is, predictably, less at organizational level

4. Organizations seem to have invested in training specialists in modeling techniques followed by process engineers. Interesting: It is not clear who the “users” of the models are – in a Software product/service organization, I expect users to be project managers and engineers

5. 75% of managers understand the results of the models well. Interesting: The % itself is interesting, since many of the managers I have met do not understand well how the models are built!

6. Just about 66% of those who build statistical models understand the intent behind it from the CMMI perspective. Interesting: Somehow, this does not resonate well with me. The only explanation I can think of is that the model builders are statisticians who are guided by the Process Engineers in identifying factors, building models and interpreting the results

7. Documenting the models and results well is a significant differentiator for high-maturity organizations. Interesting: No surprise there!

8. Not enough expertise is the only challenge that remained constant between 2008 and 2009. Other reasons have decreased! Interesting: In one year, have our problems decreased? I think in 2008, they were exaggerated!

9. 65% of managers want to use PPMs for knowing when their projects are out of track. Interesting: This is good, because having something just to gain a “high-maturity” tag is not, uh, “high-maturity” [Although, "PPMs are the way in the organization" comes a close second!]

10. There are 5 “healthy” ingredients for a good process performance model that is consistent across many research reports. When all ingredients are present, the value of the PPM to the organization is “substantial”. Interesting: The CMMI does not provide any directions on using such reports as guidance!

My observations:

1. There are many responses that mention the lack of clarity in what is expected from high-maturity practices.

2. Problems like lack of accurate historical data, wide variation in the type and nature of projects, resources etc continue to plague industry

3. There are no peer-reviewed, published reports on factors to be considered in process performance models. Even common ones like defect prediction do not have standard regression equations, where values/co-efficients can be adjusted based on organizational performance

4. Process Performance Models do not have enough documentation to describe the input data that was used to produce them. This causes resistance in using them well

5. Impact of people variation is not usually considered as a factor, but which often skews actual performance

6. Experts in statistical techniques tend to forget that finding the cause of variation is notoriously difficult in software development, which is what managers are more interested in! Stating variance values often brings up the question – “what is causing the variation?”, for which the answer is “Thats what you have to find out”. Silence.

7. High-maturity organizations often have the management commitment to stay the course even in financial difficulties – they believe that having high maturity practices is a necessary element of beating the competition and hence coming out of the financial crisis. Without this belief, High maturity goals remain another management fad

Read the report a few times to digest it. What conclusions did you draw? What aspects do you observe in your organizations? Did I miss something in my observations?

Related posts:

1. Avoiding a common trap during statistical analysis
2. Do your Metrics report performance or help improve performance?
3. 3 Ways Your Process Improvement Initiatives Can Go Wrong

The mild arrows show how you can mature the metric to the next level of maturity. For example, you can use a metric such as project schedule adherence to control the current projects, but when you can capture historical information and apply to relevant future projects, you gain the ability to predict their schedule adherence.

Now that you have some idea of how metrics can be used, take a hard look at your metrics (your metrics catalog, if you have one, can have this field!) and tell us what % of your metrics fall into each category in the comments.

Related posts:

1. Metrics Definition – Gaining Agreement from your Stakeholders – Part 1
2. Do your Metrics report performance or help improve performance?
3. Metrics Definition – Gaining Agreement from your Stakeholders – Part 2

[Oh and if you are looking at an approach for defining Real Metrics, look no further than the Goal-Question-Metrics approach. You don't need to look further than How Do You Know Your Metrics Are Worth It by Derek Huether]

Standard Templates

I can’t stress this enough. Without standard templates for capturing, reporting and presenting, you will be firefighting every reporting period. Period.

Capturing raw data: Have a standard template in a spreadsheet or a tool to capture data as activities are being performed. For example, if you are going to collect information on bugs, use a bug-tracking tool for the entire Defect resolution process. This will get you the information much faster.

Capturing aggregate data: Once you have defined the granularity at which your data is going to be represented, have a standard template for aggregating the raw data into this. For example, if you are planning to capture helpdesk calls and have some categories for the types of help desk calls (hardware, OS, Standard software installation, network, email etc), aggregate the data for each type into this template for the current reporting period. Be sure to keep this information in some sort of a Document control system – this is the heart of your reporting system!

Presenting data: Create a standard presentation template with dummy data and create charts using the dummy data. Play around with different chart types until you are comfortable with the presentation. Most people don’t think about this much and are left wondering why the chart does not show the one point which the audience should notice. There is a wealth of information on the internet and elsewhere on the presentation of data (think Edward Burke), so I’ll not go over it now.

Tip 1: A good resource to determine which chart type suits your data best is at http://chartchooser.juiceanalytics.com

Tip 2: Have a standard template where people can enter data, but cannot edit the charts themselves. People do try to gloss over bad information by using “tricks” such as changing the axis’ values etc. If you have an honest organization, feel free to ignore this!)

When you are finished with creating the standard presentation with dummy data, make sure you drive it by the key stakeholders – those who are going to produce this information and those who will consume this information. Most of the arguments during presentations are around the representation of the information – how it gives incorrect/incomplete picture, the axis values not starting with zero, colors/fonts not being consistent etc. So, ensure you have buy-in on this part so that people focus on the content subsequently.

Bonus Tip: Automate your templates as much as possible, if you are not using a tool. It makes everyone’s life that much easier. Initially, you will spend a lot of time fixing problems with the automation, but believe me, it saves a lot of pain later.

I have tried to crystallize my learning around getting your metrics accepted and implemented. What has worked for you? Do you have any specific templates/resources that has helped you in getting the numbers out? Share with me below in the comments.

Related posts:

1. Metrics Definition – Gaining Agreement from your Stakeholders – Part 1
2. Do your Metrics report performance or help improve performance?
3. 4 different ways to use your Metrics

1. Defining a Metrics catalog
2. Creating standard data collection, reporting and presentation templates

In this part, let us take a look creating a Metrics Catalog to hold the metrics definitions and communicate in an unambiguous way on what the metrics mean and how they will be reported.

Part 2 will look at creating standard templates for the execution, once the metrics are agreed. However, it is often best to gain agreement on the templates along with the definition, since any major change in the templates can cause changes in the mechanics of the definitions.

A Metrics Catalog

A metrics catalog is simply a table with information on the collection and evaluation of Metrics. It can be a spreadsheet with columns for the different parts of the Metric. A set of columns given below can be used as a starting list

Name: Define the metric name. Be careful to make the names consistent (calling one "defect density" and the as "other % reduction in defects across testing cycles " doesn’t help!)

Inputs: What are the raw inputs that you will be using for this metric

Tip: A Metric is always a relationship between two entities")

Formula: How will be the metric be computed? Describe the relationship in mathematical terms.

Objective: Describe the intent of this metric – how will you interpret the values of this metric

Tip: Use general descriptors for the interpretation – don’t say, for example, "if the value is <99%, it means we are not doing good." Rather, say "the values for this metric will help us determine how our customers perceive our services")

Data Source: Identify where the inputs will come from and if possible, who is responsible to collect this information. This is one of the fields where the more detailed the information is, the easier it is for everybody later.

Unit of Measure: Describe the units for the values of the Metric. Is it % or defects/Lines of Code or just a number. Be very careful with this as this will impact how you will report the metrics in a visual form

Target Values: Describe the acceptable range of values for the metric

Tip: Be cautious with this, if you don’t have historical data. Leave it blank for the first few periods and then fill it in with the best performance of the actual values. Once you have sufficient samples, you can devise a proper target value).

Tip 2: Be extra cautious with “industry benchmarks”. Unless they are really similar, don’t thrust them on your organization or you will encounter lot of resistance to the metrics initiative)

Frequency of collection: Describe how often will you collect the inputs, compute and report the metric

Tip: As much as possible, try to keep the frequency constant for all the metrics in the Catalog. Think Collection=Reporting -just because data is available weekly, it doesn’t mean you need to collect and report it weekly.)

Area: Describe which area of the product/service lifecycle does this metric belong to.

Type of metric: List if it is a leading metric or lagging one

Related posts:

1. Metrics Definition – Gaining Agreement from your Stakeholders – Part 2
2. Do your Metrics report performance or help improve performance?
3. 4 different ways to use your Metrics