Evaluate Project Success

Traditional methods of determining project success take into account only whether the project met its budget and schedule. A more meaningful assessment evaluates the impact of the project on the organization.

Defining project success can be a difficult task in the chemical process industries (CPI). Some companies look past the traditional parameters — cost, schedule, and operability — and determine project success by evaluating predetermined key performance indicators (KPIs). This approach generates more meaningful, fair, and balanced post-project performance assessments.

Many parameters and indicators have been used to determine project success. Some companies, however, do not have a consistent way to determine the parameters that should be applied to assess how a project met its goals and objectives and how it impacted the organization in both tangible and intangible ways.

This article compares two approaches to evaluate project success and recommends a way to implement a performance measurement tool to quantify project success.

Fairness matters

Project success is in the eye of the beholder and depends on the individual’s function within the organization. For example, the business sector evaluator is looking at corporate growth and sustainability, while the engineering sector evaluator is considering the project’s engineering objectives. An imbalance in the assessment team could skew the assessment, which can create an unfair measurement of project success. Because of these different and sometimes conflicting viewpoints, it is management’s responsibility to utilize a methodology that will measure project success fairly.

Rating project success is important because it can impact the future of the organization’s capital deployment. Often, a project’s execution can serve as a model for future projects. However, project execution approaches may vary by project size, project location, and the availability and quality of personnel. In other words, one-size-fits-all is not always the best approach.

Rating project success fairly is also important because in many organizations, employees are rewarded or punished for their performance based on perceived project success. This ignores individual contributions and can overlook a strong performance on an unsuccessful project, even if the cause of perceived project failure was beyond the individual’s control (e.g., poor execution planning, unrealistic budget and schedule, inappropriate front-end loading, etc.). When that happens, individuals are less likely to take calculated risks that may benefit future projects.

If a project is deemed a failure, some team members can suffer negative impacts to their reputations, confidence levels, and career growth and development. This can prompt highly talented people to leave the organization or, in some cases, cause them to never get another opportunity to display and develop their capabilities. In all of these cases, the organization suffers because of the erosion or underutilization of its talent.

Organizations that have a sustainable long-term capital-planning program often maintain a group of seasoned professionals to develop and manage projects. If employees involved in the execution process believe that they will be punished for their involvement in a bad project, they may choose another career path that they believe to be more rewarding and less risky to their careers (e.g., operations or general management). This is certainly not in the best interests of the company and the individual.

If an upcoming project is similar to a past failed project, management may be hesitant to proceed with it, although the first project may have failed due to unique circumstances. If this happens, the company may be passing up perfectly good opportunities to increase revenue, market share, and operability.

The traditional approach to measuring project success

The parameters that are typically used to measure project success are project cost, project schedule, and unit operability.

Project cost is the measurement of the actual project costs vs. the authorized project budget (including change orders reflecting changes in scope). An overrun of the project budget negatively impacts financial measurements such as payback, discounted cash flow (DCF), and internal rate of return (IRR), so there is validity in using this measurement. However, it has several drawbacks.

One drawback is that project authorizations may be arbitrarily established or reduced because of profitability concerns, corporate budget allotments, unreasonably low initial project estimates, lack of understanding of contingency, etc. In these cases, the project team is constrained by an unrealistic financial target that can lead to less-than-optimal decisions concerning project execution (for example, selecting the low bidder, who may provide inefficient operating equipment).

If success is based on project cost, the project team might attempt to control capital costs by sacrificing operability and maintainability to meet an unrealistic budget. For example, a project team might choose to accept a bid for pumps because of their low capital cost, but ignore their high operating costs. Or, the team might select a material of construction to save capital costs when it would be best to select a more expensive material of construction that would last longer.

The flip side of this is an overestimated project budget. Although everyone feels good about underrunning the budget, funds tied up in this project could have been appropriated to other projects with potentially higher paybacks. In addition, an excessive authorization budget clouds the picture of both individual and team performance.

Small projects are often managed as a portfolio. Excess costs for one project are transferred to a project that is underrunning its budget. This relieves the project manager on the overrunning project of accountability.

Project schedule is the measurement of the actual project duration vs. the authorized project schedule. In schedule-driven projects, management has committed to adhere to a specific schedule to start a revenue stream on time, enter the market with a new product, become the lowest-cost producer of the product, etc.

If the project is cost-driven and not schedule-driven, spending additional funds to meet the schedule makes no sense. Examples of this are planned or spot overtime, the use of multiple shifts, and premium payments to a supplier for earlier delivery. These additional catch-up costs could far exceed the additional field overhead costs to extend the schedule.

The impact of some actions taken to correct a slipping schedule can change the project’s critical path or, worse yet, create multiple critical paths. The management of multiple critical paths is extremely difficult because it eliminates the projects team’s freedom to make decisions.

Unit operability can be defined in a few different ways, but for the purpose of this article, let’s define it as how well the project met its operability goals (e.g., startup duration, startup costs, time to reach production requirements, product yields and quality, and operating and maintenance costs)...