Software Project Management Meets Six Sigma

2. Identifying Predecessor-Successor Relationships and the Critical Path
As tasks are evaluated for their levels of dependency on other tasks, they can be organized into a chain or network that illustrates one or more feasible plans for progressing through the whole set of tasks. Gantt Charts and Critical Path Diagrams illustrate these kinds of progressive task layouts. Figure 2 shows a simple case with seven tasks (A through F), each with a "Most Likely" duration shown in the task box.

Figure 2: Critical Path with Parallel Tasks

From the point of view of duration estimation, the question is, of course, "How long will it take to get everything done?" The '"Critical Path," highlighted in Figure 2 is, in simple terms, the longest of all potential routes through the task network. Stated differently, the "Critical Path" is long enough to ensure that all other project tasks could be completed in parallel (assuming that required resources are available). For that reason, the critical path tasks represent a good sample for completion time estimation. Changes in the durations of critical path tasks are directly reflected in the overall project completion time. The dynamics connected with the critical path should track pretty well with the top-level dynamics of the overall project.

3. Forecasting Completion Time
With the critical path identifying the subset of all tasks to consider, practitioners typically select one of three approaches to build the completion time forecast:

• Simple Approach
  Add up the "Most-Likely" estimates for each task. While it may seem obvious that this approach fails to account for the Best and Worst estimates for each task, some software and some users still use it. The "Most Likely" estimates for the 49 tasks in our working case total 268 days. We'll contrast that result with the others as we work them through.
  
  
• Rolling Up Task Expectation Values and Variances
  This method attempts to account for the influence of all the values in each task estimate window while assessing the variation 'stackup' that accumulates over the entire set of tasks. The calculations at the task level are typically done as follows:
  
  
  
  In Equation 1 (sometimes called the "PERT equation" since it may be used in PERT Charts), the expectation weights the "Most Likely" estimate by a factor of 4, while factoring in the "Best" and "Worst" estimates at face value. This skews the expectation in a way that responds to some degree to the shape of the estimation window.
  
  The variance estimate (Equation 2) assumes that the range of the task estimate window represents approximately 99% of the cases, or +/- 3 standard deviations. One sixth of that span can be used to approximate the standard deviation. Squaring that value gives an estimated variance.
  
  Table 2 illustrates these computations for several of the 49 tasks in our working case.
  
  

  Table 2: Sample Task Expectation and Variance Estimates

  
  
  

  Figure 3: Completion Time Estimate (days)

  Summing the Expected Values for each task gives an expected overall project completion time. In this case, the sum for the 49 tasks on the critical path is 271 days. Using the knowledge that the variances for a series of independent events are additive, the overall completion time variance is simply the sum of the individual task variances (about 46 in this case). Figure 3 illustrates the completion time estimate as a distribution with a spread that spans from about 254 days to about 290 days.
  
  
• Monte Carlo
  Monte Carlo refers to an approach that uses the distribution of each "input variable" (in this case, each task-level estimate) as the basis for a simulation and draws a series of random samples from those distributions to build a set of hypothetical result values. Without any other underlying math, the simple process of input variable sampling produces a distribution of output variables that can be used as an indicator of what one might expect to see.
  
  Setting "Assumption" Variables
  

  Figure 4: Distributions to Describe Input Variables

  Tools like the commercial software package Crystal Ball from Decisioneering make it easy to set up and assess Monte Carlo simulations. A few examples from the case study will illustrate. First, the distribution of each input (called "Assumption") variable is defined, using a representative distribution and appropriate parameters. Figures 4 and 5 show the way that setup is accomplished using Crystal Ball.
  
  For the type of variation we expect to see in these task estimate windows, the most typical distribution choices are "Triangular" and "Beta." We'll use Triangular as the best simple illustration.
  
  

  Figure 5: Triangular Distribution For One Task Estimate

  Figure 5 shows the triangular distribution for the first task in our case: Best and Most Likely = 10 days, Worst = 15 days. (Note that these are defined as "Min", "Likeliest" and "Max" in Figure 5)
  
  In a Monte Carlo setup, all other estimates are defined with the corresponding shape that the window values dictate.
  
  Setting Forecast Variables
  Another step in the setup is to define one or more forecast variables, which describe the way the predefined input variables will generate the forecast. In this case, the forecast is simply the sum of the 49 (Triangular-distributed) critical path task-level "Most Likely" values.
  
  Running the simulation generates a number of graphs and statistical reports. Figure 6 shows the basic frequency chart for predicted duration.
  
  

  Figure 6: Monte Carlo Frequency Distribution Output

  
  
  Assessing the Probabilities Associated with Particular Completion Times
  Both the Expectation Values and Monte Carlo methods allow us to calculate the likelihood that the project will be completed before or after some time point or range. With Crystal Ball, that is accomplished by moving sliders within the frequency distribution window, as illustrated in Figure 7.
  
  

  Figure 7: Assessing Completion Time Probabilities

  
  
  Compare the Monte Carlo appraisal with the "Simple" method in regards to the chances of meeting a 270 day deadline. The simple approach added "Most Likely" estimates to yield a project completion forecast of 268 days - suggesting we could make the 270 day deadline. Monte Carlo delivers the more sobering message that our chances may be slim indeed - less than 30%.
  
  The Expectation and Variance method can be used to make similar predictions. With this method, an approximation to a Normal distribution is often assumed, allowing the use of Z tables to calculate probabilities.

Comparing the Three Bottom-Up Estimation Methods
We see that the simple "add-em-up" method was most optimistic (268 days) but had no defined frequency distribution connected with the forecast. The Expectation and Variance approach was more conservative (271 days) and supplied an estimate of variation (Std. Deviation = 6.8 days) as well as a target value for project completion. The Monte Carlo method was most conservative, with a project duration estimate of about 275 days and an estimated standard deviation of about 8.9 days.

These differences are not surprising, since there are quite a number of tasks with skewed estimation windows; i.e., the "Most Likely" estimate is the same as either the Best or Worst. (In our study, more of the tasks favored the "Worst" estimate.) The computation used to build the Expectation for each task weights the extreme values only by a factor of 1 (Equation 1), thereby limiting their influence on the outcome. During Monte Carlo simulation, enough events representing task times near the edges of those estimate windows showed up in the mix to pull the forecast distribution a little further in the "Worst" direction.

In practice, if most tasks are balanced (e.g., isosceles triangles), the Expectation and Variance and the Monte Carlo approaches will tend to agree. As more task estimates become increasingly skewed (e.g., right triangles), the two methods will diverge; the Monte Carlo method will pay more attention to the extremes. Bear this in mind as you explore the use of these tools. Of course, it is often possible, and advisable, to run two or more of these approaches, comparing results and interpreting the estimates on their merits for the specific case at hand. As we will show in Part 2 of this article, no one estimate can be meaningful enough to trust in isolation. A group of estimates from different perspectives can, however, advise and inform us against the backdrop of all our other experience. Together they help illuminate blind spots and steer us clear of some of the common traps.

Looking Ahead to Part 2
We will explore top-down project estimation, looking at the way commercial or custom software estimating models work and considering how such models can be calibrated and tailored for specific development environments. This discussion will bring into play some of those "other X's" connected with the project team and the management environment as they may influence the size and nature of our decided enemy in this article set - the software project delivery "Expectations Gap."

Part 2 > Top Down Project Effort, Duration, and Defect Prediction

About The Author
David L. Hallowell is a Managing Partner of Six Sigma Advantage, Inc. and has 20+ years experience as an engineer, manager and Master Black Belt. As Digital’s representative to Motorola’s Six Sigma Research Institute he worked on the original courseware for Black Belts and the application of Six Sigma to software. Mr. Hallowell has supported Six Sigma deployments and trained many Black Belts and trainers worldwide.

With a special focus on Design for Six Sigma, he has led development teams to breakthrough improvements in the concept development and design of a number of commercial products. Mr. Hallowell has patents and publications in the area of microelectronics packaging and high speed interconnect. He has authored courses in Software DFSS, Design of Experiments, C++ , and Computational Intelligence Tools. He is a co-founder of Six Sigma Advantage Inc., where he co-authored the Black Belt, Green Belt, and Foundation curriculum. He can be reached via email at dhallowell@6siga.com.