GovernIT

The most common problem in a Metrics program is defining the "why" for each of the metrics. The second most common problem is getting agreement from all stakeholders on the "what-when-where-which-how" parts of the definition. In this 2-part series, we will look at what you can do to make this easier for your stakeholders to understand what to expect from your metrics approach.

1. Defining a Metrics catalog
2. Creating standard data collection, reporting and presentation templates

In this part, let us take a look creating a Metrics Catalog to hold the metrics definitions and communicate in an unambiguous way on what the metrics mean and how they will be reported.

Part 2 will look at creating standard templates for the execution, once the metrics are agreed. However, it is often best to gain agreement on the templates along with the definition, since any major change in the templates can cause changes in the mechanics of the definitions.

A Metrics Catalog

A metrics catalog is simply a table with information on the collection and evaluation of Metrics. It can be a spreadsheet with columns for the different parts of the Metric. A set of columns given below can be used as a starting list

Name: Define the metric name. Be careful to make the names consistent (calling one "defect density" and the as "other % reduction in defects across testing cycles " doesn’t help!)

Inputs: What are the raw inputs that you will be using for this metric

Tip: A Metric is always a relationship between two entities")

Formula: How will be the metric be computed? Describe the relationship in mathematical terms.

Objective: Describe the intent of this metric – how will you interpret the values of this metric

Tip: Use general descriptors for the interpretation – don’t say, for example, "if the value is <99%, it means we are not doing good." Rather, say "the values for this metric will help us determine how our customers perceive our services")

Data Source: Identify where the inputs will come from and if possible, who is responsible to collect this information. This is one of the fields where the more detailed the information is, the easier it is for everybody later.

Unit of Measure: Describe the units for the values of the Metric. Is it % or defects/Lines of Code or just a number. Be very careful with this as this will impact how you will report the metrics in a visual form

Target Values: Describe the acceptable range of values for the metric

Tip: Be cautious with this, if you don’t have historical data. Leave it blank for the first few periods and then fill it in with the best performance of the actual values. Once you have sufficient samples, you can devise a proper target value).

Tip 2: Be extra cautious with “industry benchmarks”. Unless they are really similar, don’t thrust them on your organization or you will encounter lot of resistance to the metrics initiative)

Frequency of collection: Describe how often will you collect the inputs, compute and report the metric

Tip: As much as possible, try to keep the frequency constant for all the metrics in the Catalog. Think Collection=Reporting -just because data is available weekly, it doesn’t mean you need to collect and report it weekly.)

Area: Describe which area of the product/service lifecycle does this metric belong to.

Type of metric: List if it is a leading metric or lagging one

   Send article as PDF   

I was reading a great article from Harwinder of Deep Fried Brain on 20 Common Project Risk Management Terms Explained. The other two terms that are quite important in the risk Management domain are Risk Probability and Risk Impact.

These are usually quantitative and provide guidance on correctly prioritizing risks (and therefore allocating the planned contingency budgets). One other term that I usually recommend when talking about Risk is “Risk Clarity Date” – a term I use to describe the possible date when more information about the risk becomes known to re-evaluate it.

Typically, projects look at the Risk log every x weeks to go through all the risks and figure out if something needs to be changed – this is not very realistic, since many risks are either time-based or event-based.

For example, if there is a Risk that a software component being developed will not be on time, but this is not going to change unless more information on the actual progress is available.

How do you do risk management? At Project levels only or do you have a Enterprise-wide Risk Management process? Share with us.

   Send article as PDF   

9CQMG4TUUK3Q

We started off the PMO series with a basic introduction about the PMO – terminologies, the different types of PMO and some of its typical functions.

Let’s talk about one very important part of a PMO function – Change Management. Change is the only constant in life – cliched? Of course, but true nevertheless. It is also one of the biggest causes of “project death” – those projects which go on indefinitely, but always overdue and a cost sink (read an extreme example of how change in scope resulted in a 12-year project that was also a massive failure!).

In a large project/program, change management becomes very important to ensure that something remains stable or atleast manageable.

Change Management has become the norm in the industry today and there are dedicated “Change Managers” too sometimes, but there is enough change mismanagement too. One of the biggest reasons for this mismanagement is because it is used synonymously with managing Requirements Change.

Managing change does not only mean managing changes to scope (“scope creep”, as it is called, but that is a creepy term). Architecture/Design decisions, standards and tools also must be controlled to prevent chaos. This is where most change management processes fail.

Let us look at some change management mechanisms and then we will revisit how change management can be applied.

Change Control Board (CCB):

One of the most common responses/techniques, but often under utilized. The CCB need not be a single, all-powerful entity, but there can be more distributed ones at different levels. For example, for large architectural changes, there can be a high-level CCB, but smaller design decisions can be changed by a lower level CCB. It is usually good to organize such mini-CCBs by the amount of control they have rather than by phase – this will create cross-functional teams at all levels, rather than more silos by function

Change Request Creation and Tracking processes:

Having a formal Change processes itself is a barrier to most spur-of-the-moment change decisions. At the minimum, change request processes should describe how a change request is created, who reviews it, criteria for escalation, stakeholders to be involved and change closure. It also needs to tie in Configuration Control for effectiveness.

Incorporating change (and its consequences) into planning:

Usually this is a fatality in most change processes. Changes to non-scope areas of the project are considered to be immune to schedule or cost effects, which is rather unlikely. Sometimes, the development team is asked to absorb the effect as the price for not understanding or doing it right the first time. Managers in charge of change control must resist this thought process or risk losing much more at a later stage in the project.

Stricter controls as Project progresses

At the start, change is more likely, since everyone is feeling around in the dark, establishing sign-posts and installing lights, figuratively, but as you progress in the project, it is important to ensure that every change request is asked “why” several times. Any change later in the lifecycle, especially with respect to decisions, is likely to affect work products already produced and accepted. A common victim of this syndrome in an application development project is the User Interface, which is thought to be like a skin – easily replaced, but is it? In services, Change is more tightly connect to configuration than with Application development, but the principle still holds true.

Having looked at some mechanisms for managing change, let us go back on how and where to apply change management. Change Management in an application development scenario can be used at:

• Scope management
• Technology stacks
• Architecture
• Design
• Standards to be followed, such as branding, user interface etc
• Third-party components
• Development environments

In a services environment, change needs to be managed for

• Hardware
• System Software (OS, standard application software etc)
• Communication equipment
• Services and their endpoints
• Processes and
• knowledge databases

Note: In IT Service Management circles, the CCB is termed as CAB, shortened for Change Advisory Board (though why it just “advises” stumps me).

That’s alright, I know this stuff, but where does the PMO fit in, you ask? PMO must be the oversight for managing change. The PMO establishes the procedures for change control and provides necessary direction to the Program on the levels of CCB (scope of change control, escalation criteria etc). It is also the final arbiter for changes to Project scope, schedule or cost.

In fact, for rescuing troubled projects, one of the first things a PMO should do is to take a hard look at the project for change leaks and based on the amount of leakage, institute an appropriate level of change control. I say “take a hard look” since it is almost guaranteed that a typical derailed project has issues managing change.

What are you experiences in managing change in your projects or services? Is there something else? Think and let me know about it.

   Send article as PDF   

Over the years, the management of IT projects has evolved considerably. Project Managers managed and delivered projects, regardless of size. However, with increasing complexity of IT projects, it was becoming difficult to deliver projects on time, within budget and with acceptable quality levels.

Project Management began to be studied as a separate discipline and project management activities and skills were codified as Frameworks. Some well-known Project Management frameworks currently in use are the Project Management Institute‘s PM Body of Knowledge (PMBOK) and PRINCE2.

With IT receiving more attention (much negative too), standards, planning and tracking became important to organizations. Setting up standard project management practices, oversight of plans, tracking and reporting became a priority. Thus was born the Project Management Office (PMO).

Programs got their own Program Management Office and now there is the EPMO – the Enterprise Program Management Office, a corporate level PMO that sets standards for Programs across the organization.

The role of a PMO has grown so important that it is listed as one of the roles in the RACI matrix in COBIT 4.1, the commonly used framework for IT Governance.

But the basic activities/responsibilities of the PMO remain consistent in principle, if not in scale. Some of the typical PMO activities include:

• Program and Project Planning
• Tracking
• Risk Management
• Financial Management and Reporting
• Change Management
• Staffing andTraining
• Quality Management

Based on the organization, the functions of a PMO may include other activities not listed above.  It is becoming increasingly clear that using a PMO can improve the project delivery performance, provided we know where and how to use it.

In an entry at CIO.com, Jim Vaughan discusses this issue in “Developing Your Project Management Office (PMO)”

We will look at some of the activities of the PMO in more detail in the next few posts. If you have some links of blogs, articles or whitepapers that document any of these aspects or you have some links to some templates, please share them with the community. You will receive an attribution with your name and website.

Before you leave, here is a good link for you. Download TenStep’s white paper (pdf) on building a PMO

Merry Christmas

   Send article as PDF   

The Standish Group has released the famous CHAOS report(http://www.standishgroup.com/newsroom/chaos_2009.php), which talks about success and failure rates of software projects all over the world. This report is obviously based on a small sample of the total software projects executed all over the world, it is significant since it is almost the only report of its kind.

Compared to the earlier report, the number of projects that are successful has gone down (to 32%), while the number of cancelled projects has gone up. From my perspective, this is a good thing and a bad thing.

The bad thing being the drop in success rates, while the good things is that there seems to be increased monitoring of projects and cancelling them when it becomes clear that they will not deliver expected value.

However, respected researchers like Robert Glass(www.robertlglass.com) have questioned the research methodology and criteria used to collect and analyze the data.

I think that due to such questions, the Standish Group has quietly removed the Summary report (costing $99) and replaced it with a “CHAOS Manifesto” (costs $1000) that (hopefully!) provides in-depth information on the results of the research and the research methodology itself.

   Send article as PDF