A metric mystery (what, how & when) every project manager needs to solve to be able to effectively use project metrics. Project metrics, you either swear by their effectiveness or consider them manipulative. Though project metrics and statistics in general are subject to manipulation, but does not take away their effectiveness. In fact they are one of the most powerful tools available with a project manager traditional or agile. However measuring and reporting project metric(s) can be a tricky business, specially since it is well known that metrics drive and influence behaviour. (The Hawthorne – a form of reactivity whereby subjects improve or modify an aspect of their behavior being experimentally measured simply in response to the fact that they are being studied,not in response to any particular experimental manipulation. - Wikipedia)
So what makes them such an indispensable tool? While the reasons may be many here are a few key ones
- Project metrics drive key decision making without which most of the decision making process would be just based on guess-work.
- Brings in the objectivity to the status reporting as you can now measure and count the progress made. It gives you that all important baseline to rely upon.
- A key tool for problem solving and carrying out root cause analysis specially in cases where it is not easy to isolate cause from symptoms.
- Enables the project manager in implementing a balanced and effective project control system and effective risk management system.
- Last but not the least, project metrics enable forecasting.
Now getting back to cracking the mystery – The What, How & When
What To Measure
While there can be be numerous metrics that can be computed based onf the phase of the project to the type of the project, most of them will likely fall under the following fundamental categories.
- Scope – One of the basic components of the project management triangle yet difficult to measure (beyond the project scope document). Difficult but not impossible, a few matrics that can be used to measure scope can include
- Change Requests – Metrics around the number of CRs entered during the project, disposed (accepted Vs rejected) can provide valuable insights around the project scoping exercise.
- A detailed work breakdown structure – Another way to measure scope is using a WBS to break down scope into more tangible format which can then be mapped with other project phases.
- Schedule – Perhaps the most visible area once the project enters the execution stage, there are quite a metrics that have been documented to baseline,measure and report on schedule. The most common ones include
- Earned value method used to get key performance indicators on schedule like schedule variance and the schedule performance index. There are many great references on how to calculate these so I will not go into those details here, or maybe save them for a later post.
- The project schedule mapped with the WBS is another way to baseline and measure schedule.
- Closely related to the project schedule is the critical path. A critical path assessment can be used to generate metrics around schedule as well
- Cost – Similar to schedule the metrics around the budget are equally critical. A couple of common ones include
- Earned Value as used to measure schedule can be used to generate metrics around the project cost using indicators like cost variance an cost performance ratios
- ROI assessment can be used to calculate the return of the investment made on the particular initiative based on the saving incurred or future earnings.
- Break even analysis , similar to the ROI assessment, the break even analysis can be used to calculate precisely when you would be able to break even on the costs incurred on the project.
How To Measure
With the importance and the categories of project metrics clear, lets move on how best we can measure and generate these project metrics
- Balanced Approach – The approach to metric(s) collection process shuld be a balanced one. It should be taken care to avoid any sudden data variations.They should be capable of covering all important aspects of the area for with the metric(s) have been generated.
- Review Industry Benchmarks – Though project metrics are centric to the specific project, it is often useful to review industry benchmarks where applicable.
- Gain acceptance – It is vital that acceptance around how metrics are collected and reported is attained from all stakeholders to avoid any potential confusion or conflicts. This is also important to ensure support and ownership on the metrics to be able to drive results. If the metrics are going to be used for on-going monitoring it must be agreed upon by everyone.
- Transparent and Actionable – The collection and measurement process itself should be transparent to the audience. This further enables to get a team buy-in on the metric(s).At the same time project metric(s) should be actionable (quantifiable of-course)
- Revision and Updates - Things change during the course of your project, so your metrics may need to be revised, re-baselined and updated as needed. Remember a stale baseline can be counterproductive.
When To Measure
With project metrics, timing is critical. Though metric collection,analysis and reporting is an ongoing exercise , the type of metric(s) and its usablity may vary. Broadly we can put these into the following three
- At the beginning of the project - These metrics will be based on historical data or raw data. Most of them will be planned estimates.During this phase the collection process itself and the on-going monitoring and reporting process needs to be socialized and agreed upon as discussed in the above section.This is where most of the baselines are also set.
- During the execution phase – This is where the rubber meets the road. The planned and the baselines are measured against the actual. Any revisions or updates are done here. We cannot afford to wait until the end of the project to do these.Metrics generated here will be used for significant decision making and project control.
- At completion of the project – Metric(s) generation does not end with project completion, but goes on into the closure phase. How well did the project do? Of course, you may have a successfully product or an outcome, but how do you measure the success of the project at holistic level? Metric generation and analysis should be part of your lessons learnt exercise.
How do you use metrics within your projects? As usual would be glad for your feedback.