Search This Blog

Wednesday, April 30, 2014

Key Performance Indicator must start with business need

In a recent workshop that I attended on Agile Engineering Practices, as we began discussion on differentiating the roles of individuals for acceptance test driven development (ATDD) and test driven development (TDD), question came up on what key performance indicators (KPI) are to be used. I offered my view point that the approach to deciding what key performance indicator to measure is an incorrect strategy for any organization or business unit.

Let us define what a KPI is. My interpretation of KPI is that it is an objective measure of the efficiency with which a business process is executed to deliver incremental value effectively to a business need. By this definition, I see four contributing factors in two ordered pairs, which are as follows:

  1. Business Need & Effectiveness
  2. Business Process & Efficiency

The business need differs across the organizations and within the business unit in an organization. For example, consider schedule variance (SV). This SV is a project management measure to see how the planned value (BCWP) of a project compares to the actual value for that project (BCWS).

While this SV is a very good measure of an individual progress, in regulated industries that manage portfolio or programs with multiple dependent subprojects where the control on regulatory boards are external constraints, the SV may not be an accurate measure of the performing organization’s project efficiency unless such delays are removed from consideration. So, if the goal is to measure the project health discounting such external influences, then, such measures may not accurately address the business need.

Now, when the right KPI is chosen based on what the organization’s or business unit’s goal is, we can shift the attention to the business process that is in place to address how efficiently it is serving up. For instance, if we use number of forecasted project launches to actual project launches when proper workflow, documentation, and versioning controls exist, then, the focus can become more objective to measure why tasks slipped, how projects were prioritized, etc.

As a result, whether it is a traditional software development setting or iterative software development, the focus should be on what is that we are trying to measure and why and then have the right processes and tools in place to collect data for analysis. Collecting exhaustive data accurately and apply an incorrect KPI will only lead to inaccurate assessment. Don’t you think so?

I would like to start a list of KPIs for a Project, Program, & Portfolio Management setting that would be a great list. Please let me know what you think should be added to this context.
  1. Schedule Variance and/or Schedule Performance Index
  2. Cost Variance and/or Cost Performance Index
  3. Number/Percentage of Projects Forecasted to Launched
  4. Number/Percentage of milestones missed
  5. Number of FTE hours / project in a delivery based schedule (no resource leveling)
  6. % of projects coming behind schedule
  7. Number of defects logged
  8. % of failure rate against test cases
  9. % of test cases/requirements coverage
  10. Number of Escaped Defects
  11. Burn rate
  12. % projects on budget
  13. % of Challenging Projects/Program or Portfolio
  14. % of Change Orders $ to original SOW $
  15. Internal Rate of Return
  16. Net Present Value
  17. Payback Period
  18. Net Profit Margin
  19. Capacity Utilization Rate (e.g.: Profitability per Project Management Resource)
  20. Account Growth Revenue
  21. Cash Conversion Rate (Revenue Recognition Frequency)
  22. % of Complaints over Product/Program
  23. Customer Satisfaction Index
  24. Training Efficiency

No comments:

Post a Comment