Improved IT Project Forecasting Through Six Sigma

	Bookmark This Page Email This Page Format for Printing Cite This Article Submit an Article Read More Articles
	Related Tools & Articles Six Sigma Applies to IT, as Evidenced by Success Implementing Six Sigma Methodology in a Small IT Firm Wrestling with IT-Six Sigma Integration Survey Results
	Discussion Forum "How do we apply the basic methdology of Six Sigma (DMAIC) in IT industry?..." Contribute to this Discussion

B

Download Products Deployment Project Evaluator iSixSigma Magazine: Digital Edition Electric Load Forecasting Project Example

y Lakshmidhar Dhulipala

The various costs of an information technology (IT) project is forecast at the beginning of the fiscal year and usually re-forecast at the end of each month based on the actual spending pattern. Variation in the IT project forecasting process can result in either underspent or overspent budgets. As is true in most processes, variation is not good. A Six Sigma approach applied to IT project forecasting process minimizes forecast accuracy variance resulting in dollar savings for the business.

The importance of correct forecasting is easy to understand. In the case of underspending, new technology projects are not undertaken because funding did not appear to be available at the beginning of the fiscal year. In the case of overspending, the technology leadership team has to scramble to arrange for additional funds and the organization may have to shortchange other projects. Improving the capability of the IT project forecasting process so that the total project cost incurred is within specified limits of forecasts at post-Analyze sizing will improve technology's overall revenue and recovery position.

IT Project Forecasting Process

The technology organization of a major financial institution based in the United States, in which project managers estimated costs in the project start up phase and re-estimated them at the end of each phase of the project life cycle, provides a good subject for a case study. Alignment between the business partners and project managers happened at the end of each month based on actual spending and the re-forecasts. The project managers updated the forecasts in the project management tool and baselined forecast accuracy variance at the end of each month using the re-forecast values.

Project Forecasting Process

The IT project forecasting process improvement project was executed by a company technology team and consultant. They evaluated the IT project forecasting process performance and improved the same by leveraging Six Sigma expertise. Six Sigma DMAIC (Define, Measure, Analyze, Improve, Control) rigor was used in executing the project. The company team was responsible for implementing the Control plan/forecasting process improvement framework and verifying the success measures.

CTQs Are an Eye Opener

Voice of customer (VOC) was collected by conducting interviews and focus groups with the stakeholders from a key portfolio of the company's technology organization. Level 2 and Level 3 as-is process maps were prepared for IT project forecasting process. Internal customer critical-to-quality characteristic (CTQ) was derived from the VOC data, and the project CTQ was determined using a CTQ drill-down tree. Customer and project CTQs were defined as:

Customer CTQ = Percentage post Analyze sizing accuracy = (Percentage variance of post-Analyze sizing and total project cost / the post-Analyze sizing)

Project CTQ = Percentage monthly average forecast accuracy = (Difference between monthly forecast and monthly actual spend in the post-Analyze phase / the monthly forecast)

With the current practice, project managers reported lower forecast accuracy variance to senior leadership team since the baseline was done using re-forecast values at the end of each month, which were accurate as compared to the forecasts at the post-Analyze sizing. The suggested CTQ baselines the variance using forecasts at the post-Analyze sizing uncovering the hidden variance. The new practice brought focus on the ability of the project managers to perform better forecasts at the post-Analyze sizing.

Analysis of Underspend and Overspend

Data analysis indicated that 57 percent of the projects were underspend cases while 28 percent were overspend cases when compared with post-Analyze sizing. Process capability of monthly forecast accuracy with a specification limit of +/- 5 percent was poor and the observed CPK was negative. Process capability of forecast accuracy with respect to post-Analyze sizing with +/- 5 percent specification limit also was negative. Median of forecast accuracy for projects with a dollar value less than $50,000 was smaller than that of projects with a dollar value greater than $50,000, indicating better accuracy in forecasting process costs of small-sized projects. The level of variability, however, was the same for both low- as well as high-dollar value projects.

The project costs included hardware, resource, third party software and miscellaneous costs. Detailed brainstorming sessions were conducted by the project team to determine root causes for inaccurate estimates, underspend and overspend situations for each project cost component. Trend analysis of the cost variance categories indicated underspend cause categories as 1) less effort realized, 2)unregistered billing for external receivables, 3) administrative issues, and 4) inaccurate forecasts registered. Similarly the overspend cause categories identified were 1) additional effort realized, 2) unplanned billing registered, 3) administrative issues, and 4) inaccurate forecasts registered.

Industry benchmark data indicated that 60 percent%† of the IT projects failed in terms of cost and schedule compliance, according to the book Information Technology Project Management by Kathy Schwalbe. Only one out of five IT projects were likely to be compliant on cost, scope, quality and schedule parameters. Also data indicated that larger the project size, the more likely was the failure on cost and schedule compliance. Based on industry benchmark reports, estimation standards, current performance baseline and agreement of the project team, the performance objectives were determined.

Monte Carlo Simulation and FMEA

Based on the current trend, the likenesses of different causes were analyzed. A probability distribution function for each cause category was determined based on the historical frequency data. Forecast accuracy results were then captured by running a Monte Carlo simulation for each of the cause categories. Factors related to less effort realized, unregistered billing and administrative issues were found to be vital Xs for the underspend variance situation. Simulation could not be performed for overspend variance due to lack of confidence in the probability distribution function determined for these cause categories. It was concluded that factors related to overspend situation were not vital since there were a fewer number of overspend cases reported.

Since there could be other unreported causes, a system and component a failure mode and effects analysis (FMEA) was performed to completely identify and prioritize causes of the project cost variance. Assessment of as-is risk was done using FMEA. Key causes with a risk priority number greater than 120 were identified. All the identified vital Xs impacting forecast accuracy variance were classified using a control impact matrix.

Action Plans and a Framework

Detailed future-state process maps were prepared. The high- and medium-impact action items were categorized based on the effort, then detailed action plans were developed for each stakeholder group including the business partners, the technology leadership team, project managers, tools group, hardware vendors and third party software vendors.

A forecasting process improvement framework with focus on requirements management, resource management, schedule management and administrative issues was suggested to eliminate sources of variation and achieve forecast accuracy variation within the +/- 5 percent specification limit.

About the Author: Lakshmidhar Dhulipala is a Master Black Belt and has more than nine years of experience in Six Sigma and CMMI deployments. He has worked primarily in IT, financial services and manufacturing domains executing/mentoring Six Sigma projects, conducting improvement opportunity assessments, developing Six Sigma training material, performing gap analysis, defining improved IT processes, and mentoring project managers on new IT process frameworks. Mr. Dhulipala is an IT process and service management consultant at Tata Consultancy Services. He can be reached at lakshmidhar.dhulipala@tcs.com.

BEST SELLING PRODUCTS (iSixSigma Publications) Certified Lean Six Sigma Green Belt Assessment Exam This assessment exam is useful for students interested in assessing their knowledge of Lean Six Sigma on the Green Belt ... Six Sigma Black Belt (DMAIC) Training Slides The 2008 Six Sigma Black Belt course is comprised of: 1,176 PowerPoint slides, Instructor notes, Slide explanations, 37 ... Six Sigma DMAIC Training Slides The complete 2008 Lean Six Sigma DMAIC course prepares participants to perform the role of a LSS Black Belt; covering wh... Process Management Training Slides The 2008 Process Management course is designed in two phases comprised of:352 Powerpoint slidesInstructor notesSlide exp... Six Sigma Green Belt Training Slides The 2008 Six Sigma Green Belt course is comprised of: 1047 slidesInstructor notesSlide explanations35 data sets20 suppo... Certified Lean Six Sigma Black Belt Assessment Exam Interested in assessing your knowledge of Lean Six Sigma? Preparing for certifications? Testing your students and traine... 5S Training Course One of the key fundamental tools of process improvement is 5S. 5S is a methodology for organizing and minimizing item... Sell Your Products