GovernIT

How do you use your metric? Every metric can provide you with a different information/decision making capability, depending on how you use it. In fact, the way you use it can give an insight into the maturity of your measurement practices. The graphic below gives the 4 most common ways in which a metric can be used.

The mild arrows show how you can mature the metric to the next level of maturity. For example, you can use a metric such as project schedule adherence to control the current projects, but when you can capture historical information and apply to relevant future projects, you gain the ability to predict their schedule adherence.

Now that you have some idea of how metrics can be used, take a hard look at your metrics (your metrics catalog, if you have one, can have this field!) and tell us what % of your metrics fall into each category in the comments.

   Send article as PDF   

In the last post, we saw one of two things we should do for communicating and gaining agreement on your Metrics Definition. This week, we will look at something that even experienced people leap over – gaining agreement on the details of collection and presentation.

[Oh and if you are looking at an approach for defining Real Metrics, look no further than the Goal-Question-Metrics approach. You don't need to look further than How Do You Know Your Metrics Are Worth It by Derek Huether]

Standard Templates

I can’t stress this enough. Without standard templates for capturing, reporting and presenting, you will be firefighting every reporting period. Period.

Capturing raw data: Have a standard template in a spreadsheet or a tool to capture data as activities are being performed. For example, if you are going to collect information on bugs, use a bug-tracking tool for the entire Defect resolution process. This will get you the information much faster.

Capturing aggregate data: Once you have defined the granularity at which your data is going to be represented, have a standard template for aggregating the raw data into this. For example, if you are planning to capture helpdesk calls and have some categories for the types of help desk calls (hardware, OS, Standard software installation, network, email etc), aggregate the data for each type into this template for the current reporting period. Be sure to keep this information in some sort of a Document control system – this is the heart of your reporting system!

Presenting data: Create a standard presentation template with dummy data and create charts using the dummy data. Play around with different chart types until you are comfortable with the presentation. Most people don’t think about this much and are left wondering why the chart does not show the one point which the audience should notice. There is a wealth of information on the internet and elsewhere on the presentation of data (think Edward Burke), so I’ll not go over it now.

Tip 1: A good resource to determine which chart type suits your data best is at http://chartchooser.juiceanalytics.com

Tip 2: Have a standard template where people can enter data, but cannot edit the charts themselves. People do try to gloss over bad information by using “tricks” such as changing the axis’ values etc. If you have an honest organization, feel free to ignore this!)

When you are finished with creating the standard presentation with dummy data, make sure you drive it by the key stakeholders – those who are going to produce this information and those who will consume this information. Most of the arguments during presentations are around the representation of the information – how it gives incorrect/incomplete picture, the axis values not starting with zero, colors/fonts not being consistent etc. So, ensure you have buy-in on this part so that people focus on the content subsequently.

Bonus Tip: Automate your templates as much as possible, if you are not using a tool. It makes everyone’s life that much easier. Initially, you will spend a lot of time fixing problems with the automation, but believe me, it saves a lot of pain later.

I have tried to crystallize my learning around getting your metrics accepted and implemented. What has worked for you? Do you have any specific templates/resources that has helped you in getting the numbers out? Share with me below in the comments.

   Send article as PDF   

I was reading a great article from Harwinder of Deep Fried Brain on 20 Common Project Risk Management Terms Explained. The other two terms that are quite important in the risk Management domain are Risk Probability and Risk Impact.

These are usually quantitative and provide guidance on correctly prioritizing risks (and therefore allocating the planned contingency budgets). One other term that I usually recommend when talking about Risk is “Risk Clarity Date” – a term I use to describe the possible date when more information about the risk becomes known to re-evaluate it.

Typically, projects look at the Risk log every x weeks to go through all the risks and figure out if something needs to be changed – this is not very realistic, since many risks are either time-based or event-based.

For example, if there is a Risk that a software component being developed will not be on time, but this is not going to change unless more information on the actual progress is available.

How do you do risk management? At Project levels only or do you have a Enterprise-wide Risk Management process? Share with us.

   Send article as PDF   

The first part of this series provided an overview of the PMO, types of PMOs and typical functions.

The second part looked at the role of PMO in setting up and monitoring Change Management processes and activities.

The third part discussed the Quality Management responsibilities of a PMO and provided a table of contents to a Quality Management Plan

This post shares some information and experience on how the PMO can review projects and what to focus on in such reviews.

One of the most important functions of the PMO is to periodically review projects, to be able to answer the  following questions:

1. Where is the project wrt where it should be?
2. Will the project deliver on its objectives – timelines, quality etc?

We have all worked on projects, where the status is green for weeks and even months and suddenly moves to  “Red” one fine day.

The best early warning system is effective and in-depth reviews by the PMO for each project in its portfolio. The frequency of such reviews depends on:

Size of the project

If the project is large and complex, one review meeting with all stakeholders is not effective. There is usually too much discussion on some items, especially those that are over the tolerance levels, while routine ones are not given much time. Instead, multiple reviews with separate teams will provide the necessary focus and insight into that area.

Separate reviews also help you to validate information being provided by one team with others. with a single meeting, contradictory statements are not voiced due to fear or a desire to avoid conflict.

If the project is small or medium sized (<30 – 40 people and less number of cross-domain teams), a single review can be effective as all stakeholders can present information quickly.

A typical review should not be more than 3 hours, as information overload sets in and people become mentally tired.

Criticality to business

Review depth also depends on how important the project is to the business. For example, a public-facing market solution will need to be monitored much closely than a project for generating MIS reports.

Current status

• If the project is progressing smoothly, with interediate deliverables on time and within quality limits, you may want to schedule a monthly meeting with offline status reports weekly.
• If the project is just about surviving, weekly reviews are necessary to tightly control the ship.
• Iif the project is behind on timelines or there are escalations from customers (can be internal such as marketing, end-users etc), day-wise monitoring may be required.

This does not mean having long meetings everyday, but you may request for daily status reports to be circulated to the governance team, with meetings held twice in the week.

What should you review

At the minimum, the review should focus on

• Verify status of tasks with respect to the Plan
• Reviewing Key accomplishments during the reporting period
• Understanding key deliverables and activities during the next period and the progress on them till now to determine if they will still be met. A good way to do this would be to ask for Estimated time to complete in-progress activities and verify against the plan
• Check for Dependencies for the upcoming activities to see if there are any impacts due to external and internal dependencies (such as staff from another team, software or hardware availability etc)
• Status of top issues and any new issues added
• Status of top risks and any updates to the Risk profile
• Change requests created/modified during the period
• Quality indicators such as defect trends, incident escalations etc

How to review effectively

• Instead of having a template which can restrict information, ask the project to develop something incorporating the above. The main point of this is to ensure they don’t feel constrained to report in a manner they feel uncomfortable with
• The report can be simple to start with, but must be able to provide enough information for the PMO to decide on the true status of the project.
• Status is usually shown in Traffic-light symbols, but this generally is not accurate or atleast consistent. Insist on objective criteria to determine what is yellow and what is red.
• Watch for tasks that rapidly change in progress completion, especially ones that slide downwards.
• When people use vague qualifiers like “I think it should be done in a couple of days” or “I believe we are on track”, look at start and end dates of the activity to gain an idea of the effort consumed. Ask for time to complete to gain a true understanding of the remaining work
• A major factor in missed deadlines is underestimating the time it takes to solve operational issues. A solid issue management mechanism will help PMO understand the blocking issues that could impact the delivery
• How is product doing with respect to Quality? Are defects being captured accurately? Schedule and review external audits that verify this one process specifically, since defects may not always be reported under the belief that they are minor
• Take the time to review customer feedback, if any and see how it dovetails into the performance of the project
• Periodically reviewing risks is one of the most important tasks of the PMO. Risk profile must be kept updated when more information is received on a subject that is impacted by a risk

Important Note:

The critical part, I cannot overemphasize this, is that the project must feel that the PMO will do anything to help the project solve issues and move forward. This may mean releasing additional funds or add experts for short durations to solve problems. If the project team feels that the PMO is only reviewing/policing, it will find ways to hide information.

You can find an example of a status report template (and some other good ones) at Derek Huether’s blog Critical Path.

Conclusion

A project review is a good opportunity for the PMO to demonstrate leadership to the projects. Transparent communication, accountability, decision-making and support are necessary elements to conduct a good project review.

What’s your take? What have I missed completely? Do you have something more to add?

   Send article as PDF   

The first part of this series provided an overview of the PMO, types of PMOs and typical functions.

The second part looked at the role of PMO in setting up and monitoring Change Management processes and activities.

This post looks at the Quality Management/Assurance responsibilities of the PMO.

Quality Management is a less-emphasized function of the PMO. In large IT organizations, primary Quality guidance is provided by a centralized Quality function and actual implementation guidance by the PMO. For smaller IT organizations, the PMO.

However, it is important that the PMO incorporate the Quality Management aspects into its guidance and governance systems, since process-orientation can bring in discipline and streamline all activities in the Programs/projects.

The key responsibilities of a PMO for Quality Management include:

Setting up quality standards if none exists or tailoring organizational standards
Provide guidance on defining acceptance criteria to measure successful completion of the project
Provide guidance on setting up Program and project specific metrics for monitoring, tracking progress and quality
Schedule, conduct and review Program and project audits to ensure they are following the guidance provided by the PMO.
These aspects can be detailed out in a Quality Management Plan. A well-structured QM Plan can help the Program/Project adhere to the accepted practices in their projects. In addition, the PMO may also provide
Quality management support to projects through a dedicated team of people.

A typical QM Plan will have the following Table of contents (sections):

• Reference to organizational processes (if available)
• List and reference to any adaptations to the organizational processes, templates and checklists
• List and reference to program/project specific processes, templates and checklists
• List and reference to all standards/guidelines (including technical, industry-specific regulations, domain etc)
• Release Reviews performed by the Quality function before any customer/production release
• Program/Project specific metrics and tolerances
• Work product reviews that will be performed by people in the Quality function
• Tools and techniques used for Quality activities
• Defect prevention, causal analysis activities and techniques
• Reports and Dashboards
• Frequency and timing of project reviews and audits by the Quality function

If you have implemented the Quality Management function as part of the PMO, we would love to hear your experiences and challenges.

   Send article as PDF