Schism
Something is amiss in Project Management Land and it’s not the usual “Traditional PM vs. Agile PM” squabble. It’s a lot more fundamental as it goes to the core of every project:
The measure of success.
This is different from the question of goal attainment, a topic which in my opinion has been examined from a lot of different angles and can be measured to a satisfactory degree, from the very basic following the SMART / INVEST criteria to fully-fledged project scorecards. No – this is not about goal attainment, it’s about finding out if we’re are targeting the correct goal in the first place.
“A major cause of the disconnect between completion and success of strategic projects is that most PMOs still rely on the same old metrics as a proxy for success:
Operational measures such as on-time and on-budget, and stakeholder satisfaction.”
– Matt McWha, CEB IT Quarterly Q3 2014
“We’re all familiar with the phrase “what gets measured gets done,” but what if we’re measuring the wrong things?”
– Andrew Horne, The IT Scorecard Disconnect, How Metrics Can Derail Your Strategy
Could it be that over the years we have become utterly efficient at checking for goal attainment and lost contact to WHY we’re actually pursuing these goals?
The Curse of Tradition
Looking at traditional project what’s getting measured? We usually check if:
- We were within or below budget
- Met the project dates & milestones
- Met the quality requirements
Tick the checkbox next to each of these requirements and the project is golden and we can move on to the next task. If only it were that simple.
Traditional KPIs usually miss one central aspect: Short and long term contribution of the project to business success.
Not true you say? Well tell me – when does a project end? Does it end when the delivery is complete? When the customer has paid?
What about long-term effects such as market share? Reputation? Or even more down-to-earth things as ongoing maintenance / support costs?
All of these effects contribute to a business’ success but very few projects are monitored beyond the hand-off to the customer – and even if they are it’s usually with the wrong metrics or the wrong intentions.
100 PMs were asked about project success…
70% cannot adequately quantify project success
38% report overly positive results to secure future funding
80% analyze / review completed projects inadequately (*)
(*) Due to focus on classic KPIs such as delivery date, cost, quality metrics – not on business results
– John Ward. „Delivering value from IS and IT investments“, Cranfield School of Management, 2006
Long-Term Effects
All of these issues create the problem of a business closing off projects one-by-one but ultimately stumbling towards doom.
What we need is not blindly following a plan but an examination of long-term effects on the business. This requires new KPIs such as:
- Project Fit with Customer’s Needs at Time of Delivery
Delivery is not about what the customer wanted twelve months ago but is needed now.
On average 33% of all requirements change within 12 months. Was the project able to keep up? - Effect on Business Reputation
Did the business reputation bloom or suffer during the project?
What are the long-term implications on the brand? - Development of Turnover Time and Cost for Change Requests
How fast was the project team able to integrate change requests?
What was the cost associated with each change?
What is the trend of the turnover time and cost?
Did the project team get more efficient over time? - Team Knowledge Accumulated and Shared
What new skills have been brought into the team?
What is the level of existing skills?
Has knowledge been transferred between individual team members?
What new ideas and developments have started as a side-effect to the project?
What is the long-term implication on the business’ ability to deliver? - Ratio of Participation in Innovative Development
What’s the ratio of projects targeting an innovative market compared to those targeting a saturated market?
Is the business just treading the ground it already knows or is it pursuing new endeavors? - Level of Project Sponsor Engagement
How did the executive sponsor’s engagement develop during and after the project?
Where they engaged? Did it seem that they recognize the project’s importance or were they mostly just walking piggy-banks?
Do the projects align with the corporate vision? (Usually goes hand-in-hand with high sponsor engagement) - Market & Mind Share Gained
What was the effect on the market share of the business? Will it be able to look back on an expanded market share?
Perhaps by winning over return customers and creating positive word-of-mouth/references?
Did the new product capture new market share? How did the market share develop over three, six or twelve months? - Long-Term Project ROI
Was the project actually worth the investment?
This is not just adding up costs and comparing with project income – it’s also about measuring the impact on future business.
Did the project drive the customer away or did it actually increase the prospects of new contracts?
Who said that the project will only pay out once?