By Gary GackThis article illustrates the Six Sigma DMAIC process using an organization that develops software packages as an example. The Six Sigma DMAIC approach to process improvement provides a powerful mechanism for improving the software process. Typical benefits will exceed costs within 6 to 12 months from initiation of a Six Sigma program for software development, and the on-going return will be very substantial -- often a 15-25% reduction in software development costs in year two, with continuing reductions thereafter.
Project Selection
While the selection process precedes a project's Define phase, identifying an initial general goal, there is a chicken and egg relationship with Define as that goal is better understood and refined. We have an initial idea of our goal, but we may need to do some of the Define work before we know if the scope is reasonable. Project selection brings out another important consideration not directly addressed by Define - it establishes the link between candidate projects and corporate strategy (in one sense, these are the top level Ys).
| Figure 1: Linking To The Strategic Plan |
|  |
|
|
The Define Phase 
The first phase of a Six Sigma process improvement project, known as "Define," has four steps:
- Identify the Customer(s) and the "Critical to Quality" (CTQ) requirement that will be the focus of the project. The CTQ may also be referred to as the project "Y" [as in Y=f(x)].
- Create the Project Charter.
- Develop a high level Process Map.
- Define Phase Tollgate Review.
As a general rule, the scope of a Six Sigma DMAIC project is limited so that the project can be completed within four to six months - this guideline avoids projects that try to "boil the ocean." Experience clearly establishes that overly large projects have a high failure rate. Hence, it is often necessary to decompose the initial project objective into a series of lower level objectives that become individual projects. This may be referred to as "decomposing the project Y."
The following diagram illustrates one way we might decomposed a "big Y" into a series of smaller, more manageable objectives that contribute to realization of the larger goal - in this instance, improving software related "Capability" (to deliver projects on time, with predictable effort, and with an acceptable number of released defects). This decomposition raises the issue of project prioritization and selection - we have limited resources that can be devoted to improvements, so which shall we do first?
| Figure 2: Narrowing Goals And Scope: Preliminary Project Selection |
|  |
|
|
Let us suppose that you are in an organization the produces software packages for sale. If this is "where you live" you probably are most concerned with the software definition, design and construction path. If that is the case, the second level Ys might include time to market, total development cost per size, and delivered quality in terms of defects. We recognize that there are potentially many factors that influence these outcomes, so we need to decompose further to get to a Six Sigma project of manageable scope.
Let us then suppose that recent experience has led you to believe that your ability to meet schedules is poor - which in turn suggests a third level Y. At the third level the CTQ objective might be something like the percent of project commitments delivered on time (or perhaps within some plus or minus window). Now we're focused on something likely to be actionable in a reasonable time, and we have an initial idea how to measure success. As we investigate further we may decide to decompose to a fourth level (or more), so let's take this decomposition process one more step. Let's assume we are a decentralized organization with various Divisions in different parts of the country - in that case we may elect to further narrow the scope of our project to a particular Division. Typically we will look at data we have, such as the percentage of late projects for each Division, as a way to select the Division to tackle first. Assuming we are successful with the first Division we can replicate the process to other Divisions later.
| Figure 3: Late Projects By Division |
|  |
|
|
Once through this first step in the thought process we move to step 2 - creating the project charter. A charter will typically include:
- A business case - what is the expected payoff if you are able to improve the CTQ? Preferably this to be expressed in dollars whenever possible - some organizations are totally insistent on this point, some accept 'soft' benefits.
- A Problem Statement - a succinct statement of the business problem that you are attempting to solve. Using the software company scenario above this might be something like "Missed schedules are impacting customer satisfaction and are causing us to miss our revenue targets." An organization concerned with deployment of purchased packages might describe the problem as "Conversions that miss planned dates are causing unexpected budget increases that impact our profitability."
- Goal statement - here we want to state the objective in terms of the metric associated with the project Y. This could be stated as "Improve the on-time project percentage from the baseline value of 62% to 90% within the next year." (Notice this goal could be equally appropriate for an organization that buys and deploys software packages.)
- Project Team - for example, the project leader would likely be a Green Belt. The project sponsor (Champion) is the VP of Marketing, and team members will include the Director of the Project Office, three software Project Managers, and the Director of Quality Assurance.
- Scope - e.g., the project will concern itself with standard product development and deployment projects only, not custom software developed under contract.
- Financial Opportunity - "Marketing surveys indicate that up to 10% of potential service contract revenue is lost due to late software deliveries. Subject to validation, the opportunity in our most recent fiscal year amounts to at least $ 800,000 per year."
The Measure Phase 
The Measure phase can be viewed as consisting of eight steps:
- Confirm / refine the project Y (CTQ) - we will investigate more fully how this will be measured, evaluate the validity of available measures, and confirm our initial hypothesis as to the magnitude of the problem/opportunity.
- Define performance goals or standards - here we establish our target for improvement in the Y, setting a goal that is aggressive but attainable.
- Identify segmentation factors for data collection plan - here we identify factors that naturally segment our project Y (by project, product, organization, etc.) and Xs that are likely to influence the Y, and define how, when and where we will measure them. Generally speaking, factors that influence outcomes are of two types - things we can control and other factors ("noise") that we can't control. We are trying to understand "what's going on?"
- Assess and calibrate the measurement system(s) to be used. Are they reliable and consistent? How accurate are the data?
- Data collection - we gather the needed data.
- Describe and display variation in current performance - what are the distributions/ranges of X and Y values currently?
- Containment plan - if the current process is in critical condition, what band-aids (quick fixes) could be put in place to reduce the bleeding until we devise a permanent fix?
- Measure Phase Tollgate Review.
Let's work through each of these steps, continuing our focus on improving our project planning process.Confirm The Project Y
We indicated in the Define phase that our goal for the Y was to improve our on-time project performance from the current baseline of 62% to 90% during the next year. To confirm or refine this Y we will need to probe a little more deeply into where this data came from, and what definitions it was based on. For example, what is the definition of "Completion"? Does that mean the date the customer accepted the system? Or does it mean the date that the software team declared it was 'finished'? There is often a big difference between these dates - the one that really counts is the customer's date.
Define Performance Goals Or Standards
After we collect some data that will allow us to confirm that the initial estimate of the on time rate (62%) is correct. we will re-examine the feasibility of our goal. Our initial targets calls for an improvement of about 50% which seems reasonable as a stretch target for a Six Sigma project.
Identify Segmentation Factors For The Data Collection Plan
A review of professional project planning practices reveals certain attributes of effective planning processes. These attributes give some clues how we may wish to segment (or characterize) our projects for analysis. We might, for example, decide to investigate four controllable factors that appear to be associated with effective planning (recognizing that there may be other factors we haven't yet identified):
- Short task durations
- Defined predecessor/successor relationships among tasks
- "Leveled" resources (making sure we had not planned 80 hour weeks for the team)
- Defined deliverables or end states for each task
There are also likely to be certain "noise" factors that we do not control - things that are simply aspects of the environment. Depending on the size of our organization and the number of projects we have completed in the last year or two, we may be able to segment or stratify the data in ways that will give additional insight into what is going on. We may, for instance, segregate the data according to the development or deployment group that did the work, or we may stratify according to the type of software project (e.g., business applications versus "firmware" that is embedded in a hardware component). It will often be the case that such segments will have different success rates.Evaluate (Analyze) The Measurement System(s) To Be Used
In this instance our "measurement system" is probably largely our project management software system - something like Microsoft project, for example. So, we will want to see if we can locate the plans that were developed and used for the recent set of projects that were used to determine the baseline performance of the Y. Assuming we find them (not always the case) we will want to interview the people responsible for updating and using these plans.
Do the plans reflect the work that was actually done? Were tasks added or deleted as the project progressed? Was the "baseline" plan saved so that we can accurately determine the promised completion date for comparison to the actual? How were project requirements changes handled? If the customer added significant new requirements during the project, was the baseline (target) date appropriately adjusted to reflect the change in scope? What rationale was used to make such adjustments? Does the customer agree that adjustments were reasonable?
The answers to questions such as these may lead us to make various adjustments to the data to make it more consistent and valid before we begin to analyze it. Sometimes we must face the fact that the data is so bad it is not usable without serious risk of drawing the wrong conclusions. The Six Sigma message is simply "understand the quality of the data before drawing any conclusions."
An additional issue we deal with here is how to convert attribute information into something quantitative. There are a great many ways that might be done - here we offer one approach suitable to this situation. The attributes mentioned above are potential Xs that influence the schedule performance outcome - we believe that if we do a good job satisfying these attributes we will be more likely to deliver the project on time. Our hypothesis then is that high ratings on the Xs will be positively correlated with higher Y ratings.
One way we might approach determining if this hypothesis is correct would be to set up a scoring scheme by which we rate the Xs for each of our historical projects in order to see if higher attribute scores are indeed correlated with better schedule performance. Here is one example of how we might do that (many other reasonable approaches are possible):
| Table 1: Possible Scoring System |
|  |
|
|
Data Collection
Using this scoring scheme, our next step is to collect the X and Y data for each of the projects in our baseline sample. That might produce a set of data something like that shown below.
| Table 2: X And Y Data Collection |
|  |
|
|
Describe And Display Variation In Current Performance
Once we have this data we will want to display it graphically in order to see what, if any, relationship there may appear to be between our Y (schedule performance, defined as % of plan, and our summary X, total score). We might produce similar graphs of the individual elements of the score.
| Figure 4: Schedule Versus Score Trend |
|  |
|
|
This would gives us a result like the adjoining graph which shows us at a glance that there does appear to be a relationship between our X and Y, as suggested by the trend line - as the score increases (more of the attributes of a professional plan are present) we see that the our Y (% of plan) improves - projects with professional plans seem more likely to be on time. But we also notice that there are some projects (those inside the circle) that don't seem to fit the general pattern - these suggest that some other factor(s) we have not yet considered may be impacting the outcome. We don't have enough evidence yet to be sure there is a cause and effect relationship, but clearly the connection merits a closer look - that's what we will do in the Analyze phase.Part 2 - The Analyze, Improve And Control Phases
About The Author
Gary Gack is a co-founder of Six Sigma Advantage, a firm dedicated to training and coaching in the application of Six Sigma to software development and information technology. He is co-author of Six Sigma Foundation, Green Belt, and Black Belt training programs tailored to software and IT audiences.
Gary has over 40 years experience in the software and IT industry with extensive large-scale project and program management, including teams with over 200 developers. He has owned and managed several software/consulting businesses, and has extensive experience with software process assessments using the SEI/CMM, ISO 15504, and various proprietary methods. Gary is the author of numerous articles dealing with IT project management, IT process improvement, cost accounting and metrics, and software quality assurance. He has consulted with leading companies in the United States, Canada, and Europe.
For more information contact Gary Gack at (401) 782-8323, email to ggack@6siga.com, or visit www.6siga.com.
