GovernIT

PMO Series: How to Review Projects

The first part of this series provided an overview of the PMO, types of PMOs and typical functions.

The second part looked at the role of PMO in setting up and monitoring Change Management processes and activities.

The third part discussed the Quality Management responsibilities of a PMO and provided a table of contents to a Quality Management Plan

This post shares some information and experience on how the PMO can review projects and what to focus on in such reviews.

One of the most important functions of the PMO is to periodically review projects, to be able to answer the  following questions:

1. Where is the project wrt where it should be?
2. Will the project deliver on its objectives – timelines, quality etc?

We have all worked on projects, where the status is green for weeks and even months and suddenly moves to  “Red” one fine day.

The best early warning system is effective and in-depth reviews by the PMO for each project in its portfolio. The frequency of such reviews depends on:

Size of the project

If the project is large and complex, one review meeting with all stakeholders is not effective. There is usually too much discussion on some items, especially those that are over the tolerance levels, while routine ones are not given much time. Instead, multiple reviews with separate teams will provide the necessary focus and insight into that area.

Separate reviews also help you to validate information being provided by one team with others. with a single meeting, contradictory statements are not voiced due to fear or a desire to avoid conflict.

If the project is small or medium sized (<30 – 40 people and less number of cross-domain teams), a single review can be effective as all stakeholders can present information quickly.

A typical review should not be more than 3 hours, as information overload sets in and people become mentally tired.

Criticality to business

Review depth also depends on how important the project is to the business. For example, a public-facing market solution will need to be monitored much closely than a project for generating MIS reports.

Current status

• If the project is progressing smoothly, with interediate deliverables on time and within quality limits, you may want to schedule a monthly meeting with offline status reports weekly.
• If the project is just about surviving, weekly reviews are necessary to tightly control the ship.
• Iif the project is behind on timelines or there are escalations from customers (can be internal such as marketing, end-users etc), day-wise monitoring may be required.

This does not mean having long meetings everyday, but you may request for daily status reports to be circulated to the governance team, with meetings held twice in the week.

What should you review

At the minimum, the review should focus on

• Verify status of tasks with respect to the Plan
• Reviewing Key accomplishments during the reporting period
• Understanding key deliverables and activities during the next period and the progress on them till now to determine if they will still be met. A good way to do this would be to ask for Estimated time to complete in-progress activities and verify against the plan
• Check for Dependencies for the upcoming activities to see if there are any impacts due to external and internal dependencies (such as staff from another team, software or hardware availability etc)
• Status of top issues and any new issues added
• Status of top risks and any updates to the Risk profile
• Change requests created/modified during the period
• Quality indicators such as defect trends, incident escalations etc

How to review effectively

• Instead of having a template which can restrict information, ask the project to develop something incorporating the above. The main point of this is to ensure they don’t feel constrained to report in a manner they feel uncomfortable with
• The report can be simple to start with, but must be able to provide enough information for the PMO to decide on the true status of the project.
• Status is usually shown in Traffic-light symbols, but this generally is not accurate or atleast consistent. Insist on objective criteria to determine what is yellow and what is red.
• Watch for tasks that rapidly change in progress completion, especially ones that slide downwards.
• When people use vague qualifiers like “I think it should be done in a couple of days” or “I believe we are on track”, look at start and end dates of the activity to gain an idea of the effort consumed. Ask for time to complete to gain a true understanding of the remaining work
• A major factor in missed deadlines is underestimating the time it takes to solve operational issues. A solid issue management mechanism will help PMO understand the blocking issues that could impact the delivery
• How is product doing with respect to Quality? Are defects being captured accurately? Schedule and review external audits that verify this one process specifically, since defects may not always be reported under the belief that they are minor
• Take the time to review customer feedback, if any and see how it dovetails into the performance of the project
• Periodically reviewing risks is one of the most important tasks of the PMO. Risk profile must be kept updated when more information is received on a subject that is impacted by a risk

Important Note:

The critical part, I cannot overemphasize this, is that the project must feel that the PMO will do anything to help the project solve issues and move forward. This may mean releasing additional funds or add experts for short durations to solve problems. If the project team feels that the PMO is only reviewing/policing, it will find ways to hide information.

You can find an example of a status report template (and some other good ones) at Derek Huether’s blog Critical Path.

Conclusion

A project review is a good opportunity for the PMO to demonstrate leadership to the projects. Transparent communication, accountability, decision-making and support are necessary elements to conduct a good project review.

What’s your take? What have I missed completely? Do you have something more to add?

Tags: How-to, Implementation, IT, PMO, Program Governance, Program Management, Reviews

PMO Series: Quality Management

The first part of this series provided an overview of the PMO, types of PMOs and typical functions.

The second part looked at the role of PMO in setting up and monitoring Change Management processes and activities.

This post looks at the Quality Management/Assurance responsibilities of the PMO.

Quality Management is a less-emphasized function of the PMO. In large IT organizations, primary Quality guidance is provided by a centralized Quality function and actual implementation guidance by the PMO. For smaller IT organizations, the PMO.

However, it is important that the PMO incorporate the Quality Management aspects into its guidance and governance systems, since process-orientation can bring in discipline and streamline all activities in the Programs/projects.

The key responsibilities of a PMO for Quality Management include:

Setting up quality standards if none exists or tailoring organizational standards
Provide guidance on defining acceptance criteria to measure successful completion of the project
Provide guidance on setting up Program and project specific metrics for monitoring, tracking progress and quality
Schedule, conduct and review Program and project audits to ensure they are following the guidance provided by the PMO.
These aspects can be detailed out in a Quality Management Plan. A well-structured QM Plan can help the Program/Project adhere to the accepted practices in their projects. In addition, the PMO may also provide
Quality management support to projects through a dedicated team of people.

A typical QM Plan will have the following Table of contents (sections):

• Reference to organizational processes (if available)
• List and reference to any adaptations to the organizational processes, templates and checklists
• List and reference to program/project specific processes, templates and checklists
• List and reference to all standards/guidelines (including technical, industry-specific regulations, domain etc)
• Release Reviews performed by the Quality function before any customer/production release
• Program/Project specific metrics and tolerances
• Work product reviews that will be performed by people in the Quality function
• Tools and techniques used for Quality activities
• Defect prevention, causal analysis activities and techniques
• Reports and Dashboards
• Frequency and timing of project reviews and audits by the Quality function

If you have implemented the Quality Management function as part of the PMO, we would love to hear your experiences and challenges.

Tags: EPMO, PMO, Program Governance, Quality Assurance, Quality Management

Forrester research on what CIOs want in 2010

A long title, but that is because I couldn’t find a shorter and apt one.

In this blog by forrester research (posted on CIO.com, by the way), CIOs have identified a few things they would like to see in their organizations in 2010.

I’ll summarize the basic points here, while you can read the rest of the article(rather small, actually) here – CIOs spoke, We listened. The basic idea is to improve Business-IT relationship, which as we know is being tried since IT was considered an industry.

How do we get closer to the Business? Lets try some of these:

1. Future trends in IT:  How the next generation of technologies can change the way people interact in and with our business using technology. The Cloud (that sounds like a horror movie), social media, knowledge management etc are the areas which can rock the boat – the boat being Enterprise Architecture in this case.

2. Talent Management – Every industry is grappling with this question – How to find the best people and keep them. Of course, it is heartening to note that IT is slowly realizing that IT is still about people and not about technology.

3. Where do we stand on the scale – Like it or not, humans have reference frames which they use to judge things around them. Corporations are no different. Every senior manager wants to know how he is doing vis-a-vis the competition. This could be in terms of people, services, products etc.

4. IT Governance – Phew! I was afraid this would not be listed. Nothing need be said here, except to note that IT Governance is still considered IT’s headache. Let me reiterate, IT Governance is the Business’ Governance of IT. IT Management is what CIOs do.

5. Communicate value – Everyone needs to market their contributions and value-adds or risk being outsourced. IT still has to do better job in communicating business value, which will come only when IT is one way of doing business, which will come when business realizes the true value and potential of IT and… You get the point.

Tags: Business-IT Alignment, IT Governance, Strategic IT

3 Fatal mistakes in implementing IT Governance

Out of all the major mistakes you can make when embarking on an IT Governance program, 3 stand out for their impact – complete failure of the program!

1. IT driving IT Governance

Business has to own and drive IT Governance. IT Governance is the business of governance of IT and hence business has to ensure that IT is aligned to the short and long term needs of the business. A simplified example of doing it wrong is when IT is asked to cut costs and and it cuts in places where long term strategic advantages are affected.

2. Thinking IT Governance is solved by a Process Framework or model

No single framework can work in establishing true IT Governance – as most models are focused on solving problems in one area of IT. COBIT is the most suitable framework for an overall Governance structure, but it only talks about the “what” – for the “how”, you will have to look at specific models for detailed guidance. ITIL for service management and CMMI for application development and maintenance are excellent choices (and industry standards!).

There are other areas, such as managing the portfolio of IT projects and their value to the business. There are no formal standards for such things and you might have to bring in reputed consultants to help you draft one for your organization.

3. Thinking that deploying a tool will give you IT Governance

There are a large number of organizations where improper deployments have become white elephants. Often, management confuses reporting with a working system – leading to deploying tools that can churn out good-looking reports and dashboards, but not really actionable information.

Tools can automate your processes and bring in consistency and effectiveness, but only after you have thought through the controls, defined processes and trained people.  

In short, full IT Governance is only for those who can stay the distance, investing time, effort and money. There are no quick-fix solutions here.

Tags: challenges, Implementation, IT Governance, lessons, mistakes

Checklists in IT – Boon or Bane

One of the most controversial elements among IT staff is the use of checklists to verify accurate completion of activities.

Most implementations of IT frameworks, particularly in application development and IT services, strongly advocate the use of checklists as the first record of verification and/or validation. The reasoning behind it is quite simple – verify if critical activities/items have been completed and the resultant output meets the requirements.

At the outset, people create a checklist to verify a long list of things to be checked, with the opinion that all of them are important! Over a period of time, “Process improvement” adds more items to the checklists, resulting in a checklist that takes more time to fill than the activity which it verifies. Result – no one uses it in spirit, defeating its very purpose.

A look at other industries shows a different trend though. Manufacturing, construction and even pilots routinely use checklists to ensure nothing of relevance is left out. Let us a take a detailed look at the Airline industry.

Pilots use a number of checklists, before, during and after takeoff  – sometimes using as many as 17 different checklists for each flight:

• Pre-flight checklists on safety, external inspections, cockpit inspections, before engine start, starting the engines, before taxiing, during taxiing
• Take Off checklists – before take-off, line-up, during take-off, after take-off, Climbing, Cruising
• Landing checklists – Descent, before landing, going around, after landing, shutdown, before leaving aircraft.

In addition, there are other checklists for abnormal conditions, such as emergency landings, loss of cabin pressure etc.

Two things differentiate checklists in other industries and checklists in IT – one, the target for applying is potentially infinite in IT and the items are not necessarily sequential.

Checklists in IT usually check for standards adherence and may have some questions to check for common mistakes. The problem with this is that standards have to be checked through the output and may occur hundreds of times, which may not be humanly possible.

The best way is to automate the standards verification process, through tools such as static analyzers, scripts to check authorized installations of software across the enterprise etc. Checklists must be used only for checking logical errors, common mistakes in applying principles and other areas, where automation may not be optimal.

Unfortunately, most IT Frameworks mandate organization-wide definition of standard checklists that become outdated quickly. Effectiveness can be improved exponentially by requiring that checklists be used by IT, but the contents of the checklists can be defined by the teams themselves. This is a COBIT-style approach, where control points are defined, but it is left to the teams themselves to define how they will pass the control point.

The Personal Software Process by Watts Humphrey (SEI CMU), for example, recommends that software developers create their own checklists based on their work style. For more information on PSP, visit the SEI website.

Tags: Checklists, Processes

Observations on IT Metrics

I recently read an article on IT measurement titled, “IT Metrics: Feast or Famine.” It is an interesting piece, emphasizing that metrics should exist for a reason and it makes sense to periodically check if that reason is itself still valid!

As a long time designer of measurement systems, I have a few observations on IT Metrics, which I share with you below.  Some of them are humorously written, but true nevertheless:

• A useful set of metrics cannot be defined in one go. It takes a minimum of n² iterations to arrive at a stable set of metrics, n being the number of stakeholders who will report the metrics
• Metrics that measure the positive side of a transaction are more easily accepted. For example, instead of the number of test failures, the number of tests passed stands a better chance of being accepted
• The reporting format is usually the most debated part of a framework. Since this is what everyone will finally see, there will be much angst on how the information is represented. In many cases, the success or failure of a measurement system is measured by the reporting framework
• A basket of metrics, with freedom for managers to select different metrics goes down better than a small set of core metrics that are inflexible
• Senior management always want comparison – between projects, projects in other departments and against industry standards (specifically in the same type of business). Without this “referential”, no metrics program will be accepted, even though everyone tells you to start small
• Someone will invariably ask for traffic light colors in your reporting, even though you stress a thousand times that no metric should be looked at in isolation
• Someone will invariably ask for a “Dashboard”, no matter the contents