IRS Automation Projects Need Development, TIGTA Says

Process Automation Benefits Are Not Being Maximized, and Development Processes Need Improvement

September 25, 2020

This report has cleared the Treasury Inspector General for Tax Administration disclosure review process and information determined to be restricted from public release has been redacted from this document.

HIGHLIGHTS

Final Audit Report issued on September 25, 2020 

What TIGTA Found

The IRS has not maximized Contracting Officer and Contract Specialist use of the Contractor Responsibility Determination Robot. For the first year of its deployment, TIGTA estimates that the Contractor Responsibility Determination Robot saved the IRS 2,740 hours totaling approximately $328,800 in processing 1,096 contracts. However, the number of hours actually saved was considerably less than the number of hours that should have been saved. If the IRS had maximized the Contractor Responsibility Determination Robot's use on the 1,618 contracts for which manual contractor responsibility determinations were potentially made during the first year of its deployment, TIGTA estimates that an additional $485,400 in unnecessary costs would have been avoided.

In addition, while the IRS allocated direct costs incurred for the Contractor Responsibility Determination Robot contract, the Robotic Process Automation Program generally did not fully allocate direct or indirect costs to specific automation projects. The allocation of these costs is critical to baseline Federal investments and determine the total costs and return on investments for each automation project.

Lastly, the Robotic Process Automation Program did not establish an effective governance structure nor a suitable development methodology for automation projects. However, the IRS outlined its plans for improving automation project deliverables. According to the IRS, its ongoing automation projects are in the process of using these deliverables and are updating them based on learning and feedback. Finalizing a well-defined automation project development methodology should help to ensure that business requirements are captured, privacy and security requirements are addressed, designs fully satisfy business requirements, solutions are properly tested and deployed in a controlled manner, and operations are closely monitored.

What TIGTA Recommended

The Chief Procurement Officer should maximize the timely use of the Contractor Responsibility Determination Robot. In addition, the Chief Information Officer should ensure that direct and indirect costs are allocated to each Robotic Process Automation and Intelligent Automation project as well as finalize an automation program governance structure and a development methodology suitable for automation projects.

The IRS agreed with all four recommendations. The IRS plans to maximize the timely use of the Contractor Responsibility Determination Robot, establish an internal order code for each automation project to track and allocate direct and indirect costs, enhance the current governance structure and process to involve additional technology and business stakeholders, and establish a Robotic Process Automation–centric Enterprise Life Cycle process.

---

September 25, 2020

MEMORANDUM FOR:
COMMISSIONER OF INTERNAL REVENUE

FROM:
Michael E. McKenney
Deputy Inspector General for Audit

SUBJECT:
Final Audit Report — Process Automation Benefits Are Not Being Maximized, and Development Processes Need Improvement (Audit # 201920018)

This report presents the results of our review to assess the effectiveness and efficiencies achieved through the Internal Revenue Service's (IRS) implementation of Robotic Process Automation and Intelligent Automation technologies. This review is part of our Fiscal Year 2020 Annual Audit Plan and addresses the major management and performance challenge of Achieving Operational Efficiencies.

Management's complete response to the draft report is included as Appendix III.

Copies of this report are also being sent to the IRS managers affected by the report recommendations. If you have any questions, please contact me or Danny R. Verneuille, Assistant Inspector General for Audit (Security and Information Technology Services).

---

Table of Contents

Background

Results of Review

Appendices

---

Background

According to Internal Revenue Service (IRS) data, as of July 2020, approximately one-third of the IRS's full-time employees will be eligible to retire within two years. If funding limitations continue to affect operational priorities, not all retiring employees will be replaced. To help compensate for the potential loss of key staffing resources, the IRS is exploring the use of Robotic Process Automation (RPA) and Intelligent Automation solutions.

RPA is the implementation of automation software¹ to perform high-volume, labor-intensive, repeatable tasks. This allows employees to focus on higher value-added work while RPA software performs the repetitive work. RPA can be deployed to increase quality, reduce human error, increase compliance, strengthen control environments, and offer new services.

Intelligent Automation imitates human learning actions to perform automated tasks used in chatbots and natural language processing. Chatbots are computer programs, known as virtual assistants, which simulate conversations with human users over the Internet. The end user makes a statement or asks a question, which is interpreted or recognized for its intent, and then the intent is mapped to a specific task. Natural language processing is the interaction between computer and human natural languages in order to program computers to process and analyze large amounts of natural language data.

The IRS is pursuing a multiyear investment strategy to prove the value of automation technologies through the establishment of an initial operating capability through Fiscal Year 2021 and the implementation of full enterprise-wide operational capability by Fiscal Year 2022. Recently developed guidance tasks the IRS with improving the delivery of its operations through automation, including:

• The President's Management Agenda (March 2018), which focused Federal agencies on shifting resources from low-value work to high-value work.

• The IRS Integrated Modernization Business Plan (April 2019), which includes plans to modernize operations by retiring and decommissioning its legacy systems and replacing them with a sustainable platform. One key objective is to strengthen organizational agility through automation and streamlining processes. Key IRS programs and initiatives include automation projects.

The IRS has deployed the Office of the Chief Procurement Officer's (hereafter referred to as the Office of Procurement) Contractor Responsibility Determination Robot (CR BOT)² as well as the initial User and Network Services function's Question and Answer (Q&A) Chatbot.

The CR BOT

In May 2019, the IRS deployed the CR BOT. The objective of the CR BOT project was to significantly reduce the time it took to manually complete the contractor responsibility determination process for each unique vendor. The contractor responsibility determination process collects data from a variety of external websites, e.g., System for Award Management, Federal Awardee Performance Integrity Information System, and Excluded Parties List System. The CR BOT checks these websites to determine if a vendor has the financial resources and capabilities to perform the proposed work and to confirm that the vendor is eligible to receive an award under applicable acquisition laws and regulations.

According to Office of Procurement management, between May 30, 2019, and May 29, 2020, the CR BOT was used 2,107 times in requests to obtain contractor determination information.³ In addition, the IRS reported that five formal training sessions on how to use the CR BOT were provided to all 224 Contracting Officers and Contract Specialists during Fiscal Year 2019.

The Q&A Chatbot

In July 2019, the IRS deployed phase 1 of the Q&A Chatbot. RPA Program management confirmed that the Q&A Chatbot uses natural language processing that analyzes, understands, and generates the language that humans use in order to interact with computers in both written and spoken form. The Q&A Chatbot provides employees with an interface that answers their questions about Windows 10 functionality. The Q&A Chatbot prioritizes employee search results, learns from their Internet search behaviors, updates presentation priorities based on the selections, and delivers personalized results. The expected benefits of the project are to identify the most frequently asked questions, provide answers that are helpful, and report the questions and answers that require maintenance or need to be retired.

According to User and Network Services function management, from August 2019 through February 2020, employees initiated 13,834 unique sessions with the Q&A Chatbot and asked a total of 22,160 questions. Employee interactions were intuitive, as the Q&A Chatbot does not require any training prior to using it.

Results of Review

Maximizing Use of the Contractor Responsibility Determination Robot Would Increase Cost Savings

The IRS estimated that by using the CR BOT, it would potentially save 11,250 hours annually when conducting contractor responsibility determinations,⁴ or approximately $1,350,000 per year.⁵ To measure the CR BOT's use and subsequent cost savings during its first year of deployment, we obtained from the IRS an extract of CR BOT usage data for May 30, 2019, through May 29, 2020.⁶ Similarly, we obtained an extract of 2,774 new IRS-administered contracts signed between May 30, 2019, and June 30, 2020, from the Procurement for Public Sector application.⁷

Using the Data Universal Numbering System (DUNS) numbers found in both extracts, we were able to associate unique CR BOT usage to specific contracts. Figure 1 presents the results of our comparison.

[Editor's Note: The reference for footnote ⁸ does not appear in the original full text.]

[Editor's Note: The reference for footnote ⁹ does not appear in the original full text.]

Comparing CR BOT usage data to the Procurement for Public Sector application contract data, we estimate that, for the first year of its deployment, the CR BOT saved the IRS 2,740 hours, which equates to a cost savings of approximately $328,800. However, the number of hours actually saved was considerably less than what the CR BOT should have saved for the same time frame. If the IRS had maximized Contracting Officer and Contract Specialist use of the CR BOT to conduct contractor responsibility determinations, the expected hour and dollar savings would have been more fully realized. For example, had the IRS maximized the CR BOT's use on the 1,618 contracts for which manual contractor responsibility determinations were potentially made during the first year of its deployment,¹⁰ we estimate that an additional $485,400 in unnecessary costs would have been avoided. If the IRS maximizes the use of the CR BOT, we estimate that it could potentially save approximately $2,427,000 over the next five years.

The President's Management Agenda states, “Federal agencies will shift time, effort, and funding currently spent performing repetitive administrative tasks and complying with unnecessary and obsolete policies, guidance, and reporting requirements, toward accomplishing mission outcomes, e.g., reducing burden through tools like integrated information technology and automation software.” It also states, “Taxpayer dollars must go to effective programs that produce results efficiently.”

Office of Procurement management stated that they did not mandate use of the CR BOT for trained Contracting Officers and Contract Specialists because it was their first RPA project, and they did not want to hurt morale by imposing a top-down policy that personnel must use the CR BOT for all contractor responsibility determinations. Specifically, they said that they prefer “a grass roots acceptance, not a top-down mandatory policy requirement.”

However, once developed, full implementation of RPA and Intelligent Automation technologies would ensure that taxpayer funds are being effectively and efficiently used.¹¹ According to an April 2019 Nextgov article on the IRS turning to automation amid a shrinking workforce,¹² Harrison Smith, the IRS Chief Procurement Officer at the time, is quoted as saying that:

“The hours they [Contracting Officers and Contract Specialists] would've spent copying and pasting information could now be used to negotiate deals and build relationships with vendors . . .” [The CR BOT is an example of] “shifting from low-value to high-value work. . . . I think our goal as individuals and as civil servants . . . is to do the absolute most we can do with the money and the support and the resources. . . . If we're not pursuing these types of things intentionally and carefully . . . we're doing everybody a disservice.”

The True Cost of Robotic Process Automation Projects Cannot Be Determined Without Detailed Cost Information

We requested the IRS's Fiscal Years 2019 and 2020 budget and expenditure data for the RPA Program. The data included overall program start-up costs as well as the development costs of the CR BOT¹³ and the Q&A Chatbot. Based on our review of RPA Program costs, we observed that the RPA Program generally does not fully allocate the direct and indirect costs for each automation project, as discussed below.

• For the CR BOT, the IRS hired a contractor to help develop and deploy this RPA technology solution into production. The direct costs of $376,450 charged by the contractor were paid out of the Information Technology organization's overall Fiscal Year 2018 budget and were not reflected in the RPA Program's expenditures. Further, no other direct or indirect costs incurred for the CR BOT were allocated to the CR BOT.

• For the Q&A Chatbot, this project did not incur any direct costs, as the labor hours for its development were absorbed by the User and Network Services function, and any indirect costs were not included in the RPA Program's expenditures and allocated to the Q&A Chatbot.

• The IRS also used Fiscal Years 2019 and 2020 funds to initiate the development of five additional automation projects including:

  1) A Robot (BOT) to automate the Offer in Compromise Aged 5M¹⁴ process in the Small Business/Self-Employed Division.

  2) A BOT to automate the Offer in Compromise payment process in the Small Business/Self-Employed Division.

  3) A BOT to improve customer hold time and customer satisfaction when providing live assistance via telephone in the User and Network Services function.

  4) A BOT to automate the e-mail referral processing in the Tax Exempt and Government Entities Division.

  5) A BOT to automate the Reporting Compliance Case Management System¹⁵ processing in the Tax Exempt and Government Entities Division.

The RPA Program budget for Fiscal Year 2019 was $5 million; however, the IRS overspent it by $105,204. RPA Program management provided a breakdown of the $5,105,204 Fiscal Year 2019 expenses as follows:

• $2,912,582 for contractors' costs to include: 1) program, platform, and project documentation; 2) licensing support; 3) development of the Offer in Compromise BOTs; and 4) UiPath¹⁶ technical support.

• $1,539,957 for contract support, platform support, development, and operations.

• $340,979 for salaries and travel for Government employees assigned to the RPA Program.

• $311,686 for platform costs including hardware, software licensing, and services.

The RPA Program budget allocated for Fiscal Year 2020 was $1 million. Similar to the expense categories for Fiscal Year 2019, the RPA Program projected expenditures of $1,000,000 for Fiscal Year 2020 include:

• $212,000 for contractors' costs to include: 1) provide licensing and support for the Offer in Compromise Aged 5M process BOT, 2) UiPath architecture and technical support, and 3) delivery of the Offer in Compromise payment process BOT.

• $144,000 for contract support, platform support, development, and operations.

• $390,000 for salaries and travel for Government employees assigned to the RPA Program.

• $254,000 for expanding the RPA platform, which includes hardware, software licensing, and services.

The IRS requested approximately $5 million for the RPA Program for Fiscal Year 2021, which includes:

• $114,000 for contractors' costs, to include: 1) provide licensing and support for the Offer in Compromise Aged 5M process BOT and 2) UiPath architecture and technical support.

• $2,300,000 for contract support, platform support, development, and operations.

• $1,580,000 for salaries and travel for Government employees assigned to the RPA Program.

• $855,000 for hardware, software licensing, and services to expand the RPA platform.

• $150,000 for training to support a federated automations model.

The President's Management Agenda provides that “Effective stewardship of taxpayer funds is a crucial responsibility of [the Federal] Government. . . .” It also states, “Congress and taxpayers have pressed for better information about how Federal IT [Information Technology] dollars are spent and the return on investment. . . . The lack of granularity makes it difficult to baseline Federal investments and show the public how the Government is spending taxpayer dollars effectively in order to drive large-scale change needed to improve business transformation and citizen services.”

The IRS explained that the cost structure establishing the RPA Program will look different from the cost structure when the program is more mature at full operations. According to RPA Program management, as the RPA Program matures, they intend to allocate the costs associated with each RPA and Intelligent Automation project to the extent practical.

Notwithstanding, the IRS believes that each automation project should be assessed on its cost, the risks mitigated, and the benefits realized. The allocation of all program and project costs will better allow for later comparison of outlaid costs to calculated benefits. If the IRS does not fully allocate both the direct and indirect costs of its RPA and Intelligent Automation projects, it cannot show the return on investment and effective stewardship of taxpayer funds.

The Robotic Process Automation Program Did Not Establish an Effective Governance Structure nor a Suitable Development Methodology for Automation Projects

An effective program governance structure has not been implemented for automation projects

In November 2018, RPA Program management presented an overview of the RPA Program to the IRS Commissioner and explained their plans to establish a governance structure to lead, develop, and operate automation development projects as well as develop a program strategy and acquire automation tools. However, as of July 2020, the RPA Program still has not established key components that comprise a proper governance structure for automation projects. For example, the RPA Program does not have:

• An approved charter to define the program's mission, vision, scope, and expected outcomes. In addition, the charter should include roles and responsibilities of the key stakeholders and supporting organizations.

• An approved program management plan to provide information on how the RPA Program will be planned, executed, monitored, and controlled. The plan should provide specific direction on the roles and responsibilities of the RPA Program office, the RPA Center of Excellence Team,¹⁷ the RPA Platform team,¹⁸ the RPA System Integrators,¹⁹ and the RPA Operations Support function²⁰ as well as handling interactions with business partners.²¹

• An approved operating model to establish the structural framework for deploying RPA at the agency level. The operating model sets agencywide standards for RPA project prioritization, development, testing, and deployment; determines controls and compliance mechanisms; and identifies and implements best practices.

• An approved roadmap to define the timeline and steps necessary to build the program, platform, processes, and procedures for developing automation projects.

• A common platform to establish an enterprise capability to develop, deploy, and operate automation projects.

In January 2020, the General Services Administration's RPA Community of Practice group published the RPA Program Playbook to provide Federal agencies detailed guidance for initiating a new RPA program or rapidly evolving an existing program. The RPA Program Playbook provides best practices on the development of foundational program guidance, including:

While the IRS had planned to establish an RPA Program office in the first quarter of Fiscal Year 2020, RPA Program efforts were paused while priorities for the Information Technology organization's funding were examined. As a result, many of the RPA Program governance and strategy documents were delayed. Subsequently, in mid-January 2020, the IRS decided to allocate some funds to continue RPA Program activities to the end of the fiscal year. Without a functioning governance structure, an organization is not in place to oversee and manage the life cycle of automation projects from origination, development, testing, and deployment through operations.

Management Action: In April 2020, RPA Program management stated that they are establishing an RPA Advisory Board with technology and business stakeholders. The RPA Advisory Board will provide oversight and key decisionmaking for the RPA Program and platform. The Board will also provide cross-functional visibility to all automation projects and will recommend project priorities and project deployments to the Chief Information Officer. In addition, RPA Program management stated that they are in the process of finalizing an RPA Program Charter, RPA Program Management Plan, RPA Operating Model, RPA Roadmap, and RPA/Intelligent Automation Platform and Architecture documentation.

A suitable development methodology has not been implemented for automation projects 

When the IRS began exploring the use of RPA and Intelligent Automation solutions, RPA Program management decided that the software development of the CR BOT would follow the Enterprise Life Cycle (ELC) methodology. Although the CR BOT's software development ultimately did not follow this methodology, RPA Program management ensured that User Acceptance Testing was conducted prior to CR BOT deployment. The purpose of the User Acceptance Testing was to test functionality to ensure that Contracting Officers and Contract Specialists received accurate output from the CR BOT for each transaction. From the testing conducted, we concluded that key requirements of the CR BOT were incorporated and tested and the results were documented showing the resolution of any exceptions.²²

Internal Revenue Manual 2.16.1, Enterprise Life Cycle, Enterprise Life Cycle Guidance, dated November 2019, provides guidance for system development projects as they move through various phases of requirements, design, testing, and deployment. Further, the ELC methodology provides guidance on the development of technical demonstrations, pilots, and proof-of-concept projects.

In January 2020, RPA Program management decided that, based on their experience with developing the CR BOT, the existing ELC methodology was not applicable or suitable for developing and deploying automation projects. However, the IRS outlined its plans for improving automation project deliverables to include documenting the project's process definition, development specification, and test plan and results as well as deployment, operations, and maintenance. According to the IRS, its ongoing automation projects are in the process of using these deliverables and are updating them based on learning and feedback. Finalizing a well-defined automation project development methodology should help to ensure that business requirements are captured, privacy and security requirements are addressed, designs fully satisfy business requirements, solutions are properly tested and deployed in a controlled manner, and operations are closely monitored.

However, as of July 2020, RPA Program management had not implemented a suitable development methodology for its ongoing automation projects.

Management Action: RPA Program management stated that they are consulting with the owners of the ELC methodology to create an RPA Project Tailoring Plan that will identify and define RPA project development artifacts that correspond to the ELC methodology.

---

Appendix I

Detailed Objective, Scope, and Methodology

The overall objective of this review was to assess the effectiveness and efficiencies achieved through the IRS's implementation of RPA and Intelligent Automation technologies. To accomplish this objective, we:

• Evaluated the effectiveness of the development methodologies used for the CR BOT and the Q&A Chatbot projects by reviewing the existing ELC methodology's suitability and applicability to the development of automation projects as well as the allocation of direct and indirect project costs.

• Assessed the CR BOT's adoption by matching the IRS-provided CR BOT usage data to the Procurement for Public Sector application contract data, and calculated the estimated labor dollar savings that the IRS has realized.

Performance of This Review

This review was performed at the Information Technology organization's Enterprise Services function and the User and Network Services function offices in Lanham, Maryland, and the Office of Procurement's Facilities Management and Security Services Division in Washington, D.C., during the period September 2019 through July 2020. We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objective. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objective.

Major contributors to the report were Danny R. Verneuille, Assistant Inspector General for Audit (Security and Information Technology Services); Bryce Kisler, Director; Carol Taylor, Audit Manager; Mark Carder, Lead Auditor; and Ashley Weaver, Senior Auditor.

Validity and Reliability of Data From Computer-Based Systems

Without access to any corroborating source data, we performed a very limited evaluation of the reliability of the IRS-provided CR BOT usage data by reviewing the output file to detect obvious errors and unexpected missing data as well as assessing the logical presentation of the data.

For the Procurement for Public Sector application contract data, we reviewed a judgmental sample¹ of the contracts and verified that the vendor name, contract number, contract signed date, contract amount, and DUNS number were accurately reported on the extract. In both cases, we determined that the data were sufficiently reliable for the purposes of this report.

Internal Controls Methodology

Internal controls relate to management's plans, methods, and procedures used to meet their mission, goals, and objectives. Internal controls include the processes and procedures for planning, organizing, directing, and controlling program operations. They include the systems for measuring, reporting, and monitoring program performance. We determined that the following internal controls were relevant to our audit objective: the Internal Revenue Manual guidance and the RPA Community of Practice group's RPA Program Playbook. We evaluated these controls by reviewing the criteria documents and interviewing Office of Procurement and the Information Technology organization's Enterprise Services function and User and Network Services function personnel as well as reviewing RPA Program and project documentation.

---

Appendix II

Outcome Measure

This appendix presents detailed information on the measurable impact that our recommended corrective actions will have on tax administration. This benefit will be incorporated into our Semiannual Report to Congress.

Type and Value of Outcome Measure:

• Funds Put to Better Use — Potential; $2,427,000 in forgone cost savings projected over the next five years from the IRS not fully using the CR BOT to perform all contractor responsibility determinations (see Recommendation 1).

Methodology Used to Measure the Reported Benefit:

If the IRS does not maximize Contracting Officer and Contract Specialist use of the CR BOT to make all contractor responsibility determinations, we estimate that the IRS will miss approximately $2,427,000 in additional cost savings over the next five years, calculated as follows:

---

Appendix III

Management's Response to the Draft Report

September 8, 2020

MEMORANDUM FOR
DEPUTY INSPECTOR GENERAL FOR AUDIT

FROM:
Nancy A. Sieger
Acting Chief Information Officer

Shanna Webbers
Chief Procurement Officer

SUBJECT:
Response to Draft Audit Report — Process Automation Benefits Are Not Being Maximized, and
Development Processes Need Improvement (Audit # 201920018)

Thank you for the opportunity to review your report. Attached is our corrective action plan to address the recommendations. Your continued assistance is valued as the Service pursues automating manual processes to achieve efficiencies in business operations and effective use of taxpayer funds.

On the outcome measure, we view maximum use of the Contractor Responsibility Robotics (CRBOT) application as an opportunity to gain an efficiency, valued at $2,427,000, instead of a forgone cost savings, over the next 5 years.

If you have any information technology questions, please contact Nancy Sieger at (202) 317-5000 or a member of your staff may contact Lisa Wilson, Associate Chief Information Officer at 240-613-4620.  For questions concerning Procurement, please contact Andrea Kadish, Director, Data Analytics and Technology Division at 202-317-3473.

Attachment

---

Attachment

Recommendation #1:  The Chief Procurement Officer should maximize the timely use of the CR BOT.

Corrective Action #1:  The IRS agrees with the recommendation. The Chief Procurement Officer will maximize the timely use of the CR BOT. To accomplish this, at least four (4) hands-on training sessions will be held by June 30, 2021, to demonstrate how and when to use the CR BOT.

Implementation Date:  June 30, 2021

Responsible Officials:  Chief Procurement Officer

Corrective Action Monitoring Plan:  We enter accepted Corrective Actions into the Joint Audit Management Enterprise System (JAMES) and monitor them on a monthly basis until completion.

Recommendation #2:  The Chief Information Officer should ensure that both direct and indirect costs are allocated to each RPA and intelligent automation project so that the true cost of each BOT is known. 

Corrective Action #2:  The IRS agrees with this recommendation. Going forward, IT will establish an internal order code for each automation project to track and allocate direct and indirect costs in the IRS’s Integrated Financial System. This guidance will be provided to the RPA Program office.

Implementation Date:  October 10, 2020

Responsible Officials:  Associate Chief Information Officer, Strategy and Planning 

Corrective Action Monitoring Plan:  We enter accepted Corrective Actions into the Joint Audit Management Enterprise System (JAMES) and monitor them on a monthly basis until completion.

Recommendation #3:  The Chief Information Officer should finalize an automation program governance structure, involving both technology and business stakeholders.

Corrective Action #3:  The IRS agrees with this recommendation and will continue with its plan to enhance its current governance structure and process to involve additional technology and business stakeholders.

Implementation Date:  September 30, 2021

Responsible Officials:  Associate Chief Information Officer, Enterprise Services

Corrective Action Monitoring Plan:  We enter accepted Corrective Actions into the Joint Audit Management Enterprise System (JAMES) and monitor them on a monthly basis until completion.

Recommendation #4.  The Chief Information Officer should finalize a development methodology suitable for automation projects.

Corrective Action #4:  IRS agrees with this recommendation, and IRS will continue as planned to establish a RPA centric ELC process.

Implementation Date:  June 30, 2021

Responsible Officials:  Associate Chief Information Officer, Enterprise Services

Corrective Action Monitoring Plan:  We enter accepted Corrective Actions into the Joint Audit Management Enterprise System (JAMES) and monitor them on a monthly basis until completion.

---

Appendix IV

Glossary of Terms

Term	Definition
Contract Specialist	Identifies and provides resolution of contracting issues based on the correct interpretation of laws, rules, and regulations.
Contracting Officer	An agent of the Federal Government empowered to execute contracts and obligate Government funds.
Data Universal Numbering System Number	A unique nine-digit identifier for businesses, generally used for credit reporting purposes.
Direct Cost	A cost that is traceable to the production of a specific item, such as a product or service.
Enterprise Life Cycle	A structured business systems development methodology that requires the preparation of specific work products during different phases of the development process. It establishes a set of repeatable processes and a system of reviews, checkpoints, and milestones that reduce the risks of systems development and ensures alignment with the overall business strategy.
Federated Automations Model	Key members of the central innovation team are embedded in the business units. This is a quasi–matrix management model. Innovation team members in the business units are responsible for funneling the business units' ideas to the central team as well as for creating awareness and testing the concepts developed by the central team.
Fiscal Year	Any yearly accounting period, regardless of its relationship to a calendar year. The Federal Government's fiscal year begins on October 1 and ends on September 30.
General Schedule	The classification and pay system established under 5 United States Code Chapter 51 and Subchapter III of Chapter 53. It is a rate of basic pay for professional, technical, administrative, and clerical professionals working for the Federal Government.
Indirect Cost	Costs used by multiple activities, which cannot be assigned to a specific cost object.
Integrated Financial System	An administrative accounting system used by the IRS.
Legacy System	An information system that may be based on outdated technologies but is critical to day-to-day operations. In the context of computing, it refers to outdated computer systems, programming languages, or application software that are used instead of more modern alternatives.
Performance Testing	Determines whether the system undergoing testing can effectively process transactions under expected normal and peak workload conditions, within acceptable response time thresholds. Performance testing will uncover any bottlenecks and capacity constraints that may not have occurred during normal functional testing.
Pilot	A limited version (limited functionality or limited number of users) of a system being deployed to discover as well as resolve problems before full implementation.
Platform	The hardware, software, and technical support for RPA applications.
Procurement for Public Sector	An application used by the IRS to request, fund, and award contracts; execute delivery orders; and verify receipt and acceptance of products and services as well as accrue procurement-related liabilities and process payments.
Project Tailoring Plan	A documented agreement between the project manager, organization, and process owner(s) regarding how the project will meet the established process requirements. This document identifies the process artifacts and reviews required to be completed by the project and any provisions or exceptions to the processes.
Proof-of-Concept	An investigative component, which demonstrates the feasibility of an idea or to prove a theory to mitigate integration, interoperability, and system-level risks.
Session	A limited time of communication between two systems.
Software	A general term that describes computer programs and consists of lines of code written by computer programmers that have been compiled into a computer program.
Systems Acceptance Testing	The process of testing a system or program to ensure that it meets the original objectives outlined by the user.
Technical Demonstration	Used for the purpose of evaluating technology or producing data in support of analyzing alternatives.
User Acceptance Testing	Testing conducted to validate that the system works as designed and implemented and satisfies the business requirements of the system.

---

Appendix V

Abbreviations

BOT	Robot
CR BOT	Contractor Responsibility Determination Robot
DUNS	Data Universal Numbering System
ELC	Enterprise Life Cycle
IRS	Internal Revenue Service
Q&A	Question and Answer
RPA	Robotic Process Automation