TIGTA Finds Problems With Implementation of Tax Exempt Determination System Release 1

Implementation of TEDS Release 1 was expected to provide significant benefits in the receipt, handling, and processing of Forms 5307 to the EP function and other TE/GE Division programs that need access to Form 5307 information. The enhancements were supposed to assist TE/GE Division management with a large influx of determination applications received as a result of a remedial amendment period.¹

Impact on the Taxpayer

TEDS Release 1 was designed to reduce the processing time for Form 5307 determination applications and improve service to customers. However, the automation of closing processes for these applications and the automation of processes for issuing determination letters were not operating as envisioned. As a result, the new System did not significantly improve the processing of Forms 5307. This is important because the time to process all types of determination applications increased from 208 calendar days to 324 calendar days between Fiscal Years 2004 and 2005, the period of time in which the new System was operational.

Synopsis

The delivery of TEDS Release 1 on March 16, 2004, did not significantly improve the TE/GE Division's processing of Form 5307 determination applications or assist in providing electronic EP determination information to other TE/GE Division programs, as originally envisioned.

Because TEDS Release 1 was implemented a couple of months after the remedial amendment period and the EP function had already started to receive a large number of applications, the EP function would not have realized the full benefits of the System even if it had worked effectively. As a result, the delay in the delivery of the System severely limited the impact TEDS Release 1 had on the processing of Form 5307 determination applications. Also, while electronic Form 5307 information was available to TEDS users, its impact was limited because the use of electronic images had not been fully incorporated into TE/GE Division operating procedures.

In addition to the failure of TEDS Release 1 to deliver many of the projected benefits, investments associated with the development of TEDS Release 1 were not appropriately tracked, which prevented TE/GE Division senior management from receiving the information needed to effectively evaluate their investment in the TEDS. TEDS Release 1 was developed from December 2001² to March 2004. The TE/GE Division used a business case model for estimating the costs and benefits to justify the need for the project and to provide a basis for estimating the cost associated with the development of the TEDS. The actual cost to develop, implement, and maintain TEDS Release 1 (through February 17, 2006) was over $16.9 million, which was approximately $2.3 million higher than that estimated in August 2003. However, the August 7, 2003, TEDS Business Case was not appropriately updated to reflect the severely curtailed capabilities of the System, the 16 percent increase in cost, and the 5-month delay in delivery of the System. Thus, TE/GE Division management did not use the Business Case to fully evaluate investment decisions. This is especially critical because TEDS Release 2 is under development by the TE/GE Division, where the Director, Exempt Organizations function, is responsible for overseeing its implementation.

Recommendations

We recommended the Director, Business Systems Planning, ensure tasks needed to achieve the system functionality for future releases of the TEDS are clearly established to provide greater assurance that expected benefits will be realized. We recommended the Director, Exempt Organizations, implement processes to ensure the TEDS Business Case is revised when changes to the project's scope occur, increases in the project's target completion date are extended by more than 10 percent, or project costs are increased by more than 10 percent, so the most current information is available to make informed investment decisions; monitor the progress of the System's development; and evaluate whether business benefits are realized. We also recommended the Director, Exempt Organizations, fully evaluate investment decisions by adopting a business case model that includes processes for tracking actual costs of processing the determination applications to compare with the estimated costs.

Response

TE/GE Division management agreed with the findings, recommendations, and measurable benefits on tax administration included in the report. Specifically, TE/GE Division management has taken corrective actions to ensure the criteria for automating case closure processes are as complete as possible for the available data, to increase the flexibility of the criteria governing automated case closure and automated letter generation systems in TEDS Release 2, and to ensure processes are in place to identify all requirements. TE/GE Division management has implemented corrective actions to ensure information needed to make investment decisions, monitor the progress of the System's development, and evaluate whether business benefits are realized is provided to TE/GE Division managers responsible for making decisions regarding TEDS Release 2 investments. Further, TE/GE Division management has taken action to ensure actual costs are tracked and compared against estimated costs. Management's complete response to the draft report is included as Appendix V.

Copies of this report are also being sent to the Internal Revenue Service managers affected by the report recommendations. Please contact me at (202) 622-6510 if you have questions or Nancy A. Nakamura, Acting Assistant Inspector General for Audit (Headquarters Operations and Exempt Organizations Programs), at (202) 622-8500.

² The start date of TEDS Release 1 was determined to be the date the Executive Steering Committee approved the multiple release strategy for the TEDS.

However, TE/GE Division management identified shortcomings with the EDS and the determination letter process, which affected their ability to achieve their mission and strategic performance goals. For example, TE/GE Division management determined their performance was constrained by outdated technology that lacked the capacity to handle the required workload, fulfill statutory IRS responsibilities under the Internal Revenue Code, and meet expectations of the EP customers. These shortcomings included:

• Average time to work a case is too long.

• Cycle time² to respond to simple inquiries and status changes is too long.

• Level of accuracy in determinations is unacceptably low.

• The EDS can not meet expected peak volume demand.

• Significant manual processes, paper handling, and tracking are inefficient.

• Nonautomation of case grading, classification,³ or assignment is inefficient.

• Postal costs are excessive because of an inefficient practice of mailing case files.

In December 2001, the TE/GE Division Investment Executive Steering Committee⁵ approved a multiple release strategy for the new system, the Tax Exempt Determination System (TEDS). Overall, the TEDS was expected to reduce cycle time, processing time, and processing costs, while implementing process improvements that were to enhance the quality of determinations and the quality of service delivered to customers. TEDS Release 1 was to provide the technical infrastructure for all future releases of the TEDS and would redesign the process for one of the simpler EP determination request forms, Short Form Application for Determination for Employee Benefit Plan (Form 5307). TEDS Release 1 would process the Forms 5307 from receipt by the IRS to issuance of a determination letter or acknowledgment notice. Specifically, implementation of TEDS Release 1 was expected to provide the following system enhancements:

• Control the receipt and handling of Forms 5307.

• Convert hard-copy Forms 5307 and other submitted documents to electronic versions that can be accessed in the TEDS to perform determination processing.

• Automate case grading and classification of Forms 5307 for case assignment.

• Automate case closure for Forms 5307 through the use of business rules.⁶

• Automate the issuance of determination letters for Forms 5307.

• Generate management information reports for monitoring the processing of determination applications.

A prior Treasury Inspector General for Tax Administration audit⁸ found that Business Systems Planning program management's planned use of a modified Enterprise Life Cycle and added project management techniques⁹ for the development of the TEDS complied with IRS guidelines for the initial stages of systems development. However, TEDS project management did not implement these techniques as planned. This resulted in the following inefficiencies:

• Requirements gathering led to delays in project development.

• Performance monitoring did not ensure project objectives were completed on schedule or within budget.

• Risk management¹⁰ did not identify potential problems that could affect completion of TEDS development.

This review was performed at the IRS National Headquarters Business Systems Planning and EP Program offices in Washington, D.C.; the Cincinnati Submission Processing Site¹² in Covington, Kentucky; the EP Rulings and Agreements Determinations program, the EP Determinations Quality Assurance program, and the TE/GE Division Customer Account Services (CAS) call site in Cincinnati, Ohio; and the EP Examinations function in Baltimore, Maryland. The audit work was conducted during the period January through April 2006 in accordance with Government Auditing Standards. Detailed information on our audit objective, scope, and methodology is presented in Appendix I. Major contributors to the report are listed in Appendix II.

Results of Review

Critical Portions of the Tax Exempt Determination System Release 1 Did Not Operate As Expected, Which Prevented Expected Benefits From Being Realized

The delivery of TEDS Release 1 on March 16, 2004, did not significantly improve the TE/GE Division's processing of Form 5307 determination applications or assist in providing electronic EP determination information to other TE/GE Division programs, as originally envisioned. Because TEDS Release 1 was implemented a couple of months after the remedial amendment period and the EP function had already started to receive a large number of applications, the EP function would not have realized the full benefits of the system even if it had worked effectively. As a result, the delay in the delivery of the System severely limited the impact TEDS Release 1 had on the processing of Form 5307 determination applications. In addition, while electronic Form 5307 information was available to TEDS users, its impact was limited because the use of electronic images had not been fully incorporated into TE/GE Division operating procedures.

Enhancements expected from TEDS Release 1 did not significantly improve the processing of EP Form 5307 determination applications

For each of the planned system capabilities for TEDS Release 1, we determined whether the System was operating as intended as well as the reasons why any system capabilities were not operating. The TE/GE Division successfully ensured TEDS Release 1 system capabilities for controlling and converting hard-copy Form 5307 determination applications into electronic case files were implemented. It also successfully automated some of the case assignment processes by systemically grading and classifying Form 5307 determination applications. While no data existed at the time of our audit to assess the impact of these enhancements, the implementation of these additional capabilities improved Form 5307 processing by reducing the number of human resources needed to perform these functions.

However, key portions of the TEDS Release 1 functionality did not deliver the expected benefits projected in the August 7, 2003, TEDS Business Case, which greatly limited the impact the System had on the processing of determination applications. As described in more detail below, the automation of closing processes for Form 5307 determination applications that met selected criteria as well as the automation of processes for issuing determination letters were not operating as envisioned. In addition, management information reports that were to be used to monitor the determination applications after IRS receipt and initial processing were not operating as expected and were not being used.

The TE/GE Division had planned on systemically closing a portion of Form 5307 determination applications by programming selected processing procedures or business rules into the case evaluation processes. Determination applications that met specific criteria were supposed to be automatically closed with a determination letter automatically issued to the customer with no human intervention. However, during pilot testing from July 28, 2003, through March 16, 2004, the EP function determined many of the cases proposed for automatic closure were at a significant risk of being inappropriately closed because sufficient evidence was not available to ensure the plans met all of the criteria for tax-exempt status.

The automated case closure feature was designed to expedite the processing for determination applications and to eliminate the need for EP personnel to prepare and send determination letters to customers. Combined, these enhancements were expected to yield monetary benefits resulting from "productivity increases that translate into reinvestment opportunities for the IRS." For example, the TEDS Business Case from August 2003 showed the TE/GE Division had estimated achieving over $788,000 in monetary benefits for FYs 2004 and 2005 resulting from reduced labor costs¹³ for processing determination applications.

Several issues prevented the EP function from implementing the automated closure feature:

• The criteria (or business rules) for automated closure did not account for instances where Forms 5307 included additional issues or amendments, which is a common practice even for pre- approved plans.

• The criteria that govern the automated case closure system capability are custom coded and do not provide the ability for the user to update or change the criteria without the assistance of a programmer for common differences such as legislative changes.

• The pressure of a tight time constraint for TE/GE Division project management to implement the TEDS Release 1 prior to the influx of Forms 5307 due to the remedial amendment period.

• The research process during the development of the TEDS was not effective to identify all of the requirements that were needed to program the TEDS.

EP function management and TEDS project management advised us the automated case closure feature described in the March 15, 2002, TEDS Business Concept of Operations will not be available in future releases of TEDS applications unless significant changes are made to either the criteria for applying business rules or the Form 5307. The only EP Form that is expected to be closed automatically is the Notice of Merger, Consolidation or Transfer of Plan Liabilities (Form 5310-A), which is a merger notice that is not required to be reviewed because no determination or acknowledgement letter is required.

TEDS Release 1 was also intended to yield nonmonetary benefits, including the following:

• Reduced cycle time for processing EP determination applications.

• Improved customer satisfaction as measured by the customer satisfaction survey score.

• Improved quality of determination letters.

 

Figure 1: Increased Cycle Time for EP Determination

 

Applications

 

 

 

The TE/GE Business Performance Reviews showed the TE/GE Division attributed the increased cycle time to a higher than anticipated volume of determination applications received from adopters of pre- approved plans and the extension of time in which plan sponsors had to submit their determination applications. In FY 2004, the EP function estimated approximately 42,000 determination applications would be received; the actual volume was 70,610 receipts.

However, the increases in cycle time could have been significantly mitigated with successful implementation of the automated case closure feature described by the August 7, 2003, TEDS Business Case. This Business Case showed "a significant variable impacting the benefits associated with TEDS Release 1 is the percentage of cases that will be closed automatically with TEDS." The TE/GE Division expected approximately 32 percent of the Forms 5307 would be closed automatically without human intervention. Because the automated case closure feature was not operating as envisioned by the Business Case, the Forms 5307 that would have been auto-closed were identified for proposed auto-closures, which required a review by revenue agents when the System was placed into production in FY 2004. This contributed to increasing cycle time from 176 calendar days to 324 calendar days between FYs 2003 and 2005.

During this same time period, TEDS Release 1 did not contribute towards the performance measures for improving customer satisfaction and improving the quality of determination letters. Figure 2 presents the decline in these two performance measures.

 

Figure 2: Decreased Customer Satisfaction and Letter

 

Quality

 

 

 

The TE/GE Business Performance Reviews attributed a decrease in the level of satisfied customers from 69 percent in FY 2003 to 61 percent in FY 2005 primarily to the large increase in workload resulting from the determination applications received from adopters of pre-approved plans and the corresponding delay in issuing determination letters. During this same period, the quality of determination letters decreased from 80 percent to 76 percent. The TE/GE Business Performance Reviews for the EP function attribute the decrease in quality in FYs 2004 and 2005 to use of a more rigid definition of the timeliness standards, which no longer allowed for reviewer judgment or mitigating circumstances. This was coupled with a large build-up of inventory for determination applications.

The automated letter generation system capability did not deliver the expected benefits described in the August 7, 2003, TEDS Business Case because TEDS Release 1 lacked the functionality to:

• Create correspondence templates.

• Provide authorized users the ability to change the letter template as well as the effective dates associated with each change.

Project management is a carefully planned and organized effort to accomplish a specific one-time effort (e.g., implementation of a new computer system). Project management includes defining project goals and objectives, specifying tasks on how the goals will be achieved and what resources are needed, and associating budgets and timeliness for completion.

Because the effort to implement TEDS Release 1 was not sufficiently planned and monitored to ensure tasks would achieve project goals, there was less assurance that system capabilities would operate as planned, expected benefits would be delivered, and the System would be implemented within estimated project cost.

TEDS Release 1 did not benefit other TE/GE Division operations not directly associated with the processing of Form 5307 determination applications

System documentation showed several TE/GE Division operations not associated with the processing of EP determination applications would also benefit from the implementation of TEDS Release 1. The August 7, 2003, TEDS Business Case identified the TE/GE CAS function as one operation that would be using TEDS functionality, and the March 15, 2002, TEDS Business Concept of Operations identified the EP Examination and EP Quality Assurance functions as other operations that would be using TEDS information.

The functionality of TEDS Release 1 was intended to provide the TE/GE CAS call site with online access to EP Form 5307 determination applications and the ability to reproduce copies of closing letters. However, because the EP Forms 5307 are actually processed on the EDS, TEDS Release 1 did not show whether the EP determination application had actually completed processing. As a result, the TE/GE CAS function used the EDS information, not the TEDS, to answer customer inquiries regarding the status of their determination applications.

While the March 15, 2002, TEDS Business Concept of Operations showed the EP Examination function would be using information from TEDS Release 1, EP Examination management stated TEDS Release 1 would not affect its operation because that functionality was not implemented. Future releases of the TEDS are expected to provide EP revenue agents with advance copies of determination letters and with the applications documents associated with the plans prior to initiating an audit of the employee plan.

The EP Quality Assurance function expected a reduction in the error rates on determination letters and timelier issuance of determination letters with the implementation of TEDS Release 1. However, these benefits were never realized because major portions of the system functionality were not delivered. EP employees have the capability to automatically select cases for review, but they select the cases from the EDS because this is "their system of record." Although no benefits were realized with TEDS Release 1 implementation, EP Quality Assurance function management stated the nondelivered system functionality did not affect their ability to meet program goals and provide quality service.

Costs and Benefits for Developing the Tax Exempt Determination System Release 1 Were Not Appropriately Tracked and Monitored

Investments associated with the development of TEDS Release 1 were not appropriately tracked, which prevented TE/GE Division senior management from receiving the information needed to effectively evaluate their investment in the TEDS. The TE/GE Division used a business case model for estimating the cost and benefits. The business case model was intended to provide a basis for making investment funding decisions and to:

• Justify the need for the project to investment decision makers.

• Provide justification for prioritizing, selecting, and funding the investment throughout the life cycle for the project.

• Establish baseline costs, schedule, and performance goals to control and evaluate the investment of resources into the project.

Over the last decade, Congress has enacted a variety of legislation to change how the Federal Government conducts its business. This legislation seeks to improve mission performance through more effective strategic, financial, and acquisition management. One law enacted was the Clinger-Cohen Act of 1996,¹⁵ which requires agencies of the Federal Government to focus on the results that are achieved when investing in Information Technology projects. Agencies are required to evaluate technology investment decisions in a true "business context" and to analyze the investments to determine their return on investment. Even after the system has been deployed, agencies should be comparing the actual business benefits with the projected business benefits for each Information Technology investment. To meet this requirement, agencies must implement processes and maintain information needed to assess whether Information Technology projects are completed at acceptable costs, within reasonable and expected time periods, and are contributing to tangible, observable improvements in mission performance. The Clinger-Cohen Act of 1996 also requires that business benefits from each Information Technology investment be reported to the Office of Management and Budget and Congress annually. The business case model assists in satisfying this requirement.

IRS policy generally requires business cases to be updated at specific points of system development,¹⁶ although revisions to a business case can occur at any time. Conditions that require a revision to the business case include changing the scope of the project, increasing the cost of the project by more than 10 percent, or extending the targeted delivery dates by more than 10 percent of the previous estimate. The revisions are necessary to ensure the most accurate information is available for making investment decisions, monitoring the progress of the system's development, and evaluating whether business benefits are actually realized.

The original baseline Business Case for the total TEDS project was prepared in June 2001 and updated in August 2003 to include only TEDS Release 1 costs and benefits. However, the Business Case for TEDS Release 1 has not been updated since August 2003, even though all three conditions that require an update to the Business Case have been met. For example, during the pilot testing of TEDS Release 1, problems surfaced with the quality of cases meeting the rules for closing cases using automated business rules processes and with the quality of determination letters that were to be issued using automated processes of TEDS Release 1. This resulted in a significant change to the scope of the TEDS Release 1 project because processes that were designed to close some EP Forms 5307 without human intervention were changed to a process still requiring a revenue agent's review. Problems with the automated letter generation portion of TEDS Release 1 software could not be used, which resulted in a change to the scope and required additional resources to review determination letters for cases meeting selected criteria.

Since issuance of the August 7, 2003, TEDS Business Case, the cost to deliver TEDS Release 1 had also exceeded the estimated project cost by over 10 percent. Total expected project cost through FY 2006 for TEDS Release 1 was estimated at approximately $14.7 million in the Business Case updated in August 2003. However, the actual cost to implement and maintain TEDS Release 1 was about $17 million as of February 17, 2006, which exceeded the estimate by approximately $2.3 million or 16 percent.

The TE/GE Division also did not meet the delivery dates targeted in the updated Business Case, which should have triggered another update to reevaluate the System's performance. The August 2003 Business Case showed that TEDS Release 1 was scheduled to be deployed in October 2003, but it was not actually deployed until March 2004.

In addition to not updating the August 7, 2003, TEDS Business Case, TE/GE Division management did not have a process in place to appropriately evaluate investment decisions associated with the development of TEDS Release 1 because the actual cost of processing determination applications was not compared with the cost estimated in the Business Case. The August 7, 2003, TEDS Business Case presented the annual estimated cost of processing determination applications using TEDS Release 1 functionality; however, no process was in place to assess whether the estimates were accurate because the actual cost was not tracked.

TE/GE Division management advised us the Business Case was not updated because they considered TEDS Release 1 to be essentially complete at the time the System was placed into a production mode. However, we believe another update to the Business Case along with an analysis comparing the actual cost of processing determination applications with the estimate in the Business Case would have provided TE/GE Division management with the information necessary to appropriately evaluate investment decisions as required by the Clinger-Cohen Act of 1996.

The TE/GE Division started planning for TEDS Release 2 in September 2003, before TE/GE Division management was fully aware of the limitations of the System. As part of this planning process, TE/GE Division management prepared a June 7, 2004, TEDS Business Case that analyzed the benefits and costs associated with TEDS Release 2. The system functionality to be added with the implementation of this Release includes the ability to image additional documents (i.e., other than those received as part of a new Form 5307 application package) and either add those images to an existing TEDS case file or create a new electronic file for images not related to an existing TEDS case. TEDS Release 2 will also provide the ability to scan and image Exempt Organizations closed case files. We did not perform an assessment of the processes used to plan or develop system functionality for Release 2 in this audit. However, based on the failure of TEDS Release 1 to deliver many of the necessary projected benefits, TE/GE Division management should closely review the expected benefits and costs associated with TEDS Release 2 and update the June 7, 2004, TEDS Business Case as appropriate.

Recommendations

Recommendation 1: The Director, Business Systems Planning, should ensure tasks needed to achieve the system functionality for future releases of the TEDS are clearly established to provide greater assurance that expected benefits will be realized. This includes ensuring:

• Criteria for automating case closure processes are complete, to account for additional issues or amendments within the determination applications.

• Criteria that govern the automated case closure and automated letter generation system capabilities are flexible, to allow users to easily update or change the criteria for common issues such as legislative changes.

• Adequate time is provided to implement TEDS Release 2.

• Processes are effective to identify all TEDS Release 2 system requirements.

• Management's Response: The Commissioner, TE/GE Division, reported that each element of Recommendation 1 has been completed. Specifically:

  1) During development of the requirements for TEDS Release 2, the criteria for automating case closure processes were thoroughly reviewed by the TEDS user group and were determined to be as complete as possible for the available data from the form.

  2) Criteria that govern the automated case closure and automated letter generation systems in TEDS Release 2 are significantly more flexible than those in Release 1. The custom-coded business rules engine used in Release 1 has been replaced by a commercial off-the-shelf business rules engine that enables authorized users to modify specific criteria for auto-closure. This commercial off-the-shelf product also enables a programmer to easily create new business rules when needed. The custom-coded letter generation system in TEDS Release 1 has been replaced in TEDS Release 2 by another commercial off-the-shelf-based system that enables authorized users to easily add or modify standard letter paragraphs without reprogramming.

  3) TEDS Release 2 has been divided into distinct phases, and each phase is allocated an amount of time in the project schedule for development, testing, and pilot. The deployment will also be accomplished in a phased approach starting in April 2007 and concluding in October 2007. The project schedule contains time for each phase of the life cycle and is managed by weekly scheduled reviews conducted by the integrated project team.

  4) To ensure effective processes are in place to identify all requirements, a User Group with a wide background of TE/GE Division expertise has been established for TEDS Release 2. The User Group of front-line employees and managers, with additional participants providing input on an as-needed basis, is in place. To control the process, the TEDS 2 Integrated Project Team has developed and implemented a Requirements Management Plan, which includes a Change Control Board imposing configuration management via change requests.

² Cycle time is the period from when an EP case is postmarked to the closing of the application and issuance of the determination letter.

³ The case grading system capability recommends the grade level of the employee assigned to work the case (i.e., General Service 9, 11, 12, or 13). The classification system capability shows the type of case to be worked (i.e., Automated, Merit, Non-Merit, and Washington Office).

⁴ The Business Concept of Operations document provides the vision for how the TEDS will facilitate the processing of determinations within the TE/GE Division.

⁵ A committee with the authority to approve a project request and determine whether to continue the project. The committee includes executive-level members and advisors. Executive-level members, who have voting privileges, are from each part of the organization that has a stake in the success of their initiative. Advisors are from an organization such as Business Systems Planning.

⁶ The business rules are requirements that govern which cases meet the favorable determination criteria.

⁷ A favorable determination letter may no longer apply if there is a change in a statute, regulation, or revenue ruling applicable to pension plans. When this occurs, the plan must be amended to comply with the new requirements. The IRS allows plan sponsors time, called the remedial amendment period, to incorporate tax law changes without a penalty or sanction. The remedial amendment period ended on January 31, 2004.

⁸Project Management Techniques Need to Be Followed to Effectively Develop the Tax Exempt Determination System (Reference Number 2003-10-103, dated May 2003).

⁹ Project management techniques are used to ensure the development of computer systems is efficient and effective.

¹⁰ Risk management is a continual process that seeks to proactively manage uncertainties to reduce or avoid adverse project impacts.

¹¹ A business case serves as the document that summarizes numerous technical and business work products, analyses, and studies that provide the basis for making investment funding decisions and for monitoring and evaluating investment performance.

¹² Submission Processing sites process paper and electronic submissions for analysis and posting to taxpayer accounts.

¹³ The estimated labor processing savings are presented in constant dollars. Constant dollars are reported in terms of the value they had on a previous date (e.g., the dividend of $5.00 per share paid in 1986 was worth only $2.50 in constant dollars of 1976, when the stock was purchased).

¹⁴ Balancing Reports are inventory reports, one report each from the two systems, used to reconcile the inventory of determination letters between the EDS and the TEDS.

¹⁵ Clinger-Cohen Act of 1996 (Federal Acquisition Reform Act of 1996) (Information Technology Management Reform Act of 1996), Pub. L. No. 104-106, 110 Stat. 642 (codified in scattered sections of 5 U.S.C., 5 U.S.C. app., 10 U.S.C., 15 U.S.C., 16 U.S.C., 18 U.S.C., 22 U.S.C., 28 U.S.C., 29 U.S.C., 31 U.S.C., 38 U.S.C., 40 U.S.C., 41 U.S.C., 42 U.S.C., 44 U.S.C., 49 U.S.C., 50 U.S.C.).

¹⁶ IRS policy generally requires a business case to be updated after successful completion of integration, testing, acceptance, and piloting (testing the system in an actual business environment).

I. Determined what system capabilities were scheduled to be delivered as a result of the implementation of TEDS Release 1.

III. Determined whether TEDS Release 1 is functioning as intended and assessed the impact any differences have had on the TE/GE Division's programs.

² This is the process of testing a system or program to ensure it meets the original objectives outlined by the user in the requirement analysis document.

³ The Configuration and Change Management Handbook is used to identify, control, and approve changes to system documentation, custom computer source code, or off-the-shelf software.

Type and Value of Outcome Measure:

Reliability of Information -- Potential; $2.3 million (see page 9). Total project costs were not tracked and reported because TE/GE Division management did not have a process in place to appropriately evaluate investment decisions associated with the development of TEDS Release 1. As a result, information on the $2.3 million cost overrun was not available for TE/GE Division management to use in planning, monitoring, and reporting on their investment.

Methodology Used to Measure the Reported Benefit:

We calculated the total estimated TEDS Release 1 project costs through FY 2006 (as of February 17, 2006) by using the information from the August 7, 2003, TEDS Business Case. The $14.7 million estimate includes $13,451,068 to implement TEDS Release 1 and $1,200,055 in FYs 2004 through 2006 for contractor labor and other contractor-related costs.

The $17 million cost to implement and maintain TEDS Release 1 includes $12,028,694 spent to implement the System prior to issuance of the August 7, 2003, TEDS Business Case. The $17 million also includes $475,053 needed by the TE/GE Division to cover labor and other related costs to complete implementation of TEDS Release 1 and $4,488,405 in contractor costs between August 7, 2003, and February 17, 2006. The total cost overrun was determined by subtracting the estimated cost from the actual cost.

 1. Actual cost to implement and maintain TEDS Release 1 as of

TE/GE follows the ELC and ELC Lite methodology and governance processes. Rather than update the Business Case for TEDS Release 1 after the August, 2003 update, TE/GE made the decision to create a revised Business Case using the detailed evaluation criteria of OMB Circular A-11, Exhibit E-300 Lite. This was prepared over a period of months in the spring of 2004. It was issued June 7, 2004, and then updated in August, 2004, to reflect TE/GE management decisions.

The TE/GE Determination System (TEDS) Release 2 "E-300" was written as part of TE/GE management's response to your earlier audit of the TEDS project (Audit Number 2003-10-103, dated May 2003). TE/GE management relied on this revised business case to recommend additional TEDS development as a separate Release and its justification was used by the IRS Capital Planning Investment Committee (CPIC) to award funding to TEDS Release 2, currently in development.

TE/GE has implemented the process improvements for better project management you recommended in Audit 2003-10-103. As a result, where appropriate, we have rescoped TEDS Release 2 to ensure we are able to meet the highest business priorities. TEDS Release 2 is well on the way to delivering enhancements to processing all determination applications received by TE/GE. Following is our response to your specific recommendations.

Recommendation 1:

The Director, Business Systems Planning, should ensure that tasks needed to achieve the system functionality for future releases of the TEDS are clearly established to provide greater assurance that expected benefits will be realized. This includes ensuring:

• Criteria for automating case closure processes are complete to account for additional issues or amendments within the determination applications.

• Criteria that govern the automated case closure and automated letter generation system capabilities are flexible to allow users to easily update or change the criteria for common issues such as legislative changes.

• Adequate time is provided to implement TEDS Release 2.

• Processes are effective to identify all TEDS Release 2 system requirements.

We are pleased to report that each element of Recommendation 1 has been completed. Commentary on completion of Corrective Action 1;

1. During the development of the requirements for TEDS Release 2, the criteria for automating case closure processes were thoroughly reviewed by the TEDS user group, and were determined to be as complete as possible for the available data from the form.

2. Criteria that govern the automated case closure and automated letter generation systems in TEDS Release 2 are significantly more flexible than in Release 1. The custom coded business rules engine used in Release 1 has been replaced by a Commercial Off-The-Shelf (COTS) business rules engine, Blaze Advisor, which enables authorized users to modify specific criteria for autoclosure. This COTS product also enables a programmer to easily create new business rules when needed. The custom coded letter generation system in Release 1 has been replaced in Release 2 by another COTS-based system using MS-Word and PDF templates that enable authorized users to easily add or modify standard letter paragraphs without re-programming.

3. TEDS Release 2 has been divided into distinct phases, and each phase is allocated an amount of time in the project schedule for development, testing and pilot. The deployment will also be accomplished in a phased approach starting in April of 2007 and concluding in October 2007. The project schedule contains time for each phase of the life cycle and is managed by weekly scheduled reviews by the integrated project team.

4. To ensure that effective processes are in place to identify all requirements, a User Group with a wide background of TEGE expertise has been established for TEDS Release 2. The User Group of front line employees and managers, with additional participants providing input on an as-needed basis, is in place. To control the process, the TEDS 2 Integrated Project Team (IPT) has developed and implemented a Requirements Management Plan, which includes a Change Control Board (CCB) imposing configuration management via change requests. Each phase of the TEDS 2 project has a specific set of requirements defining the scope and business needs. The requirements were derived using the Use Case Methodology (UCM) and are controlled in the Rational tool set, a standard project software too).

Responsible Official:

Director, Business Systems Planning.

Recommendation 2:

The Director, Exempt Organizations, should implement processes that ensure the TEDS Business Case is revised when the changes to the project's scope occur, increases in the project's target completion date are extended by more than 10 percent, or project costs are increased by more than 10 percent, so the most current information is available to make informed investment decisions, monitor the progress of the system's development, and evaluate whether business benefits are realized.

Corrective Action 2:

We are pleased to report that Recommendation 2 is completed.

Commentary on completion of Corrective Actions 2 and 3: The Director, Exempt Organizations ensures that TEDS Release 2 follows the Enterprise-wide ELC and ELC Lite methodology and governance processes to ensure that the most current information is available to the TE/GE Investment Executive Steering Committee (IESC) to make informed investment decisions and monitor the progress of the system's development and evaluate whether business benefits are realized.

As part of this oversight, in the last year, the TEDS Release 2 project team now briefs the Director, Exempt Organizations and the Commissioner, TE/GE on a regular basis in advance of the monthly review by the TE/GE IESC. At both briefings, project activities, accomplishments, risks and issues, any significant cost, schedule and scope issues that require IESC review and decision are discussed and evaluated. At the TE/GE IESC meetings, these issues, as well as any milestone exits proposed for the TEDS project, are addressed and project decisions are made and documented.

Actual costs are tracked and compared on a weekly basis against projected estimates for the same time frame. The aggregate weekly actual costs are subsequently verified with the monthly invoices. This tracking mechanism is being used for all three TEDS Release 2 vendor/contractor teams and ensures that the most accurate financial information is available for review and management on a more timely basis than in TEDS Release 1, and can be effectively incorporated in any required updates to the TEDS business case.

Responsible Official:

Director, Exempt Organizations.

Recommendation 3:

The Director, Exempt Organizations, should ensure investment decisions are fully evaluated by adopting a business case model that includes processes for tracking actual costs of processing the determination applications to compare with the costs estimated.

Corrective Action 3:

We are pleased to report that Recommendation 3 is completed.

Please see the Commentary on Completion of Corrective Actions 2 and 3, under Corrective Action 2 above, for a discussion of TE/GE's completion of this corrective action.

Responsible Official:

Director, Exempt Organizations.

We concur with TIGTA's statement of the measurable benefit on tax administration.