CBO Says It Tends to Overestimate Revenue Projections

An Evaluation of CBO's Past Revenue Projections

AUGUST 2020

Differences Between CBO's and the Administration's Budget-Year Projections of Revenues and Actual Amounts

---

Contents

Summary

How Do CBO's Revenue Projections Compare With Actual Outcomes?

How Do CBO's Projections Compare With the Administration's Projections?

What Factors Have Contributed to Differences Between CBO's Projections of

Revenues and Actual Revenue Collections?

CBO's Methods for Projecting Revenues

CBO's Methods for Assessing Its Projections

Sources of Data

Calculation of Projection Errors

BOX 1. AN EXAMPLE OF HOW CBO ADJUSTS ITS PROJECTIONS TO CALCULATE PROJECTION ERRORS

Measures of the Quality of Projections

Limitations of Projection Evaluations

Assessment of the Projections

Budget-Year Projections

Sixth-Year Projections

Projections Since the 2007–2009 Recession

BOX 2. CBO'S REVENUE PROJECTIONS AND THE CORONAVIRUS PANDEMIC

Serial Correlation in CBO's Projection Errors

Differences Between This Analysis and CBO's 2015 Report on Its Revenue Forecasting Record

Some Sources of Projection Errors

Errors Related to Projections of GDP

BOX 3. ERRORS IN REVENUE PROJECTIONS MADE NEAR THE ENACTMENT OF MAJOR TAX LEGISLATION

Errors Related to Other Factors

Comparison of Errors Related to Projections of GDP With Those Related to Other Factors

Interactions Between Different Sources of Errors

Errors in Projections of Different Sources of Federal Revenues

List of Tables and Figures

About This Document

---

Notes

Unless this report indicates otherwise, all years referred to are federal fiscal years, which run from October 1 to September 30 and are designated by the calendar year in which they end.

Numbers in the text and tables may not add up to totals because of rounding.

Some of the figures in this report use shaded vertical bars to indicate periods of recession.

A recession extends from the peak of a business cycle to its trough.

Supplemental data are posted along with this report on CBO's website (www.cbo.gov/ publication/56499).

Summary

In the course of producing baseline projections of the federal budget, the Congressional Budget Office regularly projects what federal revenues would be in the current year and in the next 10 years if current laws generally remained unchanged. To refine its methods and improve its projections, the agency routinely assesses the accuracy of its past projections.

In this report, CBO assesses the quality of the revenue projections that it has produced since 1982 — the earliest year for which the information necessary to assess the projections is available — by comparing them (after an adjustment to account for the effects of subsequent legislation) with actual revenue collections. Specifically, it looks at the projections of total revenues and of different sources of revenues for the budget year (the next fiscal year after the year in which the projections are made) and for the sixth year of the projection period (including the current year).

The agency has tended to overestimate revenues in its projections — especially those that extend further into the future — largely because it is difficult to predict the timing, depth, and duration of downturns in the business cycle. A disproportionate share of the largest projection errors (which are calculated as the difference between the adjusted projection and actual revenues, expressed as a percentage of actual revenues) are in projections made just before a recession. When the four budget-year projections that were produced at or near business cycle peaks are excluded, the accuracy of the projections, according to key statistical measures, improves significantly.¹

How Do CBO's Revenue Projections Compare With Actual Outcomes?

In evaluating its projections, CBO focused on three characteristics of forecast quality — centeredness, accuracy, and dispersion (see Table 1).

Centeredness. The tendency of a set of projections to not repeatedly err in the same direction is referred to as centeredness. To measure centeredness, CBO uses the average error — the arithmetic mean of the projection errors. A perfectly centered set of projections would have an average error of zero.

The average error for CBO's budget-year projections published since 1982 is 1.2 percent, indicating that, on average, CBO has slightly overestimated total revenues for the budget year. For context, an error of that size in the baseline projections that CBO produced in 2018 would amount to about $42 billion of the roughly $3.5 trillion in total revenues that CBO projected for 2019.²

Overestimates and underestimates offset each other in the calculation of the average error, so the measure can obscure the magnitude of the errors. The 1.2 percent average error for CBO's budget-year projections reflects projections in which CBO overestimated revenues by as much as 25 percent (the projection for 2009 published in January 2008) and others, namely those for the late 1990s and the mid-2000s, in which the agency underestimated revenues by nearly 10 percent.

Projection errors are generally larger the further into the future the projection extends. In its projections of revenues for the sixth year of the baseline projection period, CBO has, on average, overestimated revenues by 5.6 percent.

Accuracy. This characteristic refers to how close the projected values are to the actual amounts. CBO uses two standard measures of accuracy: the average absolute error (that is, the average of all errors with the negative signs removed from the underestimates) and the root mean square error (RMSE).

Table 1. Summary Measures of the Quality of CBO's and the Administration's Revenue Projections

Percent
	Projection Year
Measure	Current Year	Budget year	Third Year	Fourth Year	Fifth Year	Sixth Year
	CBO
Centeredness
Average error	*	1.2	2.5	3.7	4.8	5.6
Accuracy
Average absolute error	2.2	4.9	6.9	8.5	9.4	9.9
RMSE	2.9	7.0	9.4	10.4	11.2	11.6
Dispersion
Two-thirds spread of errors	6.5	11.9	15.9	21.3	19.0	19.8
	Administration
Centeredness
Average error	-0.2	1.7	3.2	4.7	6.0	7.4
Accuracy
Average absolute error	2.3	5.2	7.5	9.3	10.4	11.5
RMSE	2.9	7.4	9.9	11.4	12.4	13.2
Dispersion
Two-thirds spread of errors	7.0	11.7	16.5	22.5	20.8	18.6
Sources: Congressional Budget Office and Office of Management and Budget. The projection errors underlying these summary measures are calculated as the projected amount minus the actual amount, divided by the actual amount; thus, the error is expressed as a percentage of the actual amount. A negative average error is an underestimate, and a positive average error, an overestimate. The average absolute error is the average of projection errors with the negative signs removed from the underestimates. The RMSE is calculated by squaring the projection errors, averaging those squares, and taking the square root of that average. The revenue projections included in this analysis are those published in calendar years 1982 through 2019, typically in January or February. The current year is the fiscal year in which the projection is made. The budget year refers to the fiscal year beginning eight to nine months after the projection is made, the third year is the fiscal year starting one year and nine months after the projection is made, and so on. RMSE = root mean square error; * = between zero and 0.05 percent.

The average absolute error for CBO's budget-year revenue projections made since 1982 is 4.9 percent, and the RMSE is 7.0 percent. An error the size of that average absolute error in the revenue projection for 2019 that CBO released in April 2018 would correspond to an overestimate or underestimate of $171 billion.

The sixth-year projections have larger errors — an average absolute error of 9.9 percent and an RMSE of 11.6 percent. An error the size of that average absolute error in the revenue projection for 2023 that CBO released in April 2018 would mean that actual revenues were $419 billion higher or lower than the agency projected.

The average absolute error and the RMSE appear to level off by the sixth-year horizon; those two measures of CBO's sixth-year projections are roughly equal to those of the agency's fifth-year projections. The small sample size of projections extending beyond six years does not allow for a comprehensive assessment, so the apparent stabilizing of the measures cannot yet be confirmed.

Dispersion. The size of the range around the projection errors is referred to as the errors' dispersion. In this analysis, CBO uses the range formed by the middle two-thirds of errors — known as the two-thirds spread — as its measure of dispersion. A larger two-thirds spread of errors implies that any single projection for that particular time horizon may be less certain.

The dispersion of CBO's revenue projection errors increases with the projection horizon. For the agency's budget-year projections, the two-thirds spread of errors is 11.9 percent; for the sixth-year projections, it is 19.8 percent.

How Do CBO's Projections Compare With the Administration's Projections?

The errors in the Administration's budget-year projections of revenues have been similar, in both size and direction, to those in CBO's projections. In the budget-year projections that it has published since 1982 (adjusted to exclude the effects of its proposed policy changes), the Administration overestimated revenues, on average, by 1.7 percent — slightly more than the 1.2 percent that CBO overestimated by, on average (see Table 1 on page 2). The RMSE for the Administration's budget-year projections is 7.4 percent, close to the 7.0 percent RMSE of CBO's budget-year projections. The accuracy of the Administration's projections of specific revenue sources was similar to that of CBO's projections.³

The Administration's sixth-year projections are less centered than CBO's projections. Whereas CBO overestimated revenues in the sixth year of the projection period by 5.6 percent, on average, the Administration overestimated revenues in the sixth year by 7.4 percent, on average. Likewise, the Administration's sixth-year projections are slightly less accurate than CBO's: The RMSE of the Administration's sixth-year projections is 13.2 percent, and that of CBO's projections is 11.6 percent. The Administration's sixth-year projection errors were, however, less dispersed than CBO's were.

What Factors Have Contributed to Differences Between CBO's Projections of Revenues and Actual Revenue Collections?

Many of CBO's revenue projection errors can be attributed to errors in the agency's economic forecast; other errors in the revenue projections arise from differences in projected and actual income relative to the size of the economy. The largest forecast errors are associated with revenue projections produced for years near downturns in the business cycle, when gross domestic product (GDP) was significantly lower than expected. Errors have also arisen from other factors, including unexpected declines in certain types of income relative to GDP — most notably wages and salaries, corporate profits, and capital gains realizations — and changes in the share of overall income earned by the highest-earning taxpayers (who face the highest tax rates).

Projections of specific revenue sources have contributed in varying degrees to errors in the projections of total revenues. Projections of individual income taxes have contributed the most to errors in CBO's projections of total revenues because they were the largest source of revenues in every year of the period analyzed. The next two largest sources of error were the projections of corporate income taxes and payroll taxes. The projections of smaller revenue sources — excise taxes, customs duties, and estate and gift taxes — have contributed much less to the errors in the agency's projections of total revenues.

CBO's Methods for Projecting Revenues

In accordance with the rules governing baseline projections, CBO constructs its projections of federal revenues to reflect the assumption that current laws governing taxes will generally remain unchanged.⁴ The agency projects revenues largely by taking the values of the macroeconomic variables in its economic forecasts that most affect the bases on which the various federal taxes are imposed and applying the appropriate effective tax rates (which often differ significantly from the rates set by law) to them.⁵

CBO models each major source of revenues separately and projects total revenues by summing the projections of each source.⁶ Those projections are based mainly on the following macroeconomic variables:

• Wages and salaries and proprietors' income — all of which influence both individual income taxes and payroll taxes,

• Asset prices — which affect the income tax base (through capital gains realizations and distributions from tax-deferred retirement accounts) and the base for estate and gift taxes, and

• Corporate profits — the key determinant of corporate income tax revenues.

CBO also incorporates information from monthly data about actual tax collections into its projections. When tax collections from recent months deviate from the amounts that the agency estimated for the full year using its various models, CBO assesses whether the factors that may explain such deviations are temporary or are likely to persist. Whether the identified factors actually were responsible for the discrepancies often cannot be confirmed until more detailed information becomes available in the future.

When the agency determines that the differences between its projections and actual tax collections are systemic, it modifies its forecasting models and methods. Over the more than three decades covered in this report, CBO made numerous adjustments to its methods to reflect developments revealed by new data.⁷ For example, more than a decade ago, CBO incorporated into its projections of income and payroll taxes the expectation that earnings of higher-income taxpayers would grow more quickly than those of taxpayers with lower income.⁸

CBO's Methods for Assessing Its Projections

In this assessment, CBO compares its past revenue projections (adjusted to account for the effects of legislation enacted after they were prepared) with actual revenue collections and with the Administration's projections. To assess the quality of its projections, CBO focuses on three main characteristics: centeredness, accuracy, and the dispersion of projection errors. Although assessing past projections is an important part of the projection process, several factors limit the extent to which such assessments can improve the accuracy of future projections.

Sources of Data

For this assessment, CBO examined the baseline revenue projections that it has released annually (usually early in the calendar year) since 1982.⁹ Each of those projections includes estimates for the fiscal year in which it was prepared and for at least the next 5 years; since 1992, the agency has published projections for the current year and next 10 years. For example, in January 2017, CBO released revenue projections for 2017 (the current year), 2018 (the budget year), and each of the next 9 years (see Figure 1). Each October, shortly after a new fiscal year begins, the Treasury Department reports the actual amounts of revenues for the previous fiscal year (subject to later revisions, which are typically small). Thus, the baseline projections published since 1982 provide sample sizes of 37 budget-year projections (those for 1983 to 2019) and 33 sixth-year projections (those for 1987 to 2019).¹⁰

CBO also compared its estimates with those prepared by the Administration for the same years. The Administration generally publishes its projections in February when it submits the budget to the Congress.

Calculation of Projection Errors

Differences between projected amounts and actual outcomes are referred to in this report as projection errors. On the most basic level, each projection error is calculated by subtracting actual revenue collections from the projected amount and dividing that difference by actual revenues to express the error as a percentage; a positive value represents an overestimate and a negative value an underestimate.

Certain adjustments to the projections are necessary, however, to make meaningful comparisons with actual outcomes. CBO's baseline projections are meant to provide a benchmark against which proposed policy changes can be measured; they therefore reflect the assumption that laws affecting revenues and spending generally remain unchanged over the projection period.¹¹ When legislation is enacted after the projections are made, it can affect actual revenues and cause those actual amounts to differ significantly from the projected values. To remove such effects from its measure of projection errors, CBO adjusted the projections to incorporate the estimated revenue effects of legislation enacted after the projections were published. (For a detailed example of how the agency adjusts its projections to calculate errors, see Box 1.) Generally, CBO uses the estimates prepared by the Joint Committee on Taxation (JCT) near the time of the legislation's enactment to make those adjustments.

Adjusting projections to account for recently enacted legislation may introduce additional errors because the estimates of legislation's effects on revenues are themselves uncertain. In addition, by longstanding convention, estimates of the budgetary effects of legislation do not reflect the feedback to the budget that arises from broad changes in the economy; thus, any such feedback contributes to the measured projection error.

The Administration's projections must also be adjusted to make them comparable with CBO's projections. In contrast to CBO, the Administration includes in its projections the estimated revenue effects of the policies that it proposes but that are not yet law.¹² To make those projections comparable with CBO's projections for the same period, CBO subtracted the estimated effects of the proposed policies (which the Administration publishes separately) from the Administration's revenue projections. In general, the Administration does not publish additional estimates of the revenue effects of legislation after it submits the budget to the Congress. When the Administration's estimates of legislation's revenue effects were unavailable, CBO used the JCT's estimates to adjust the Administration's projections.

Both CBO's and the Administration's revenue projections include details on the different sources of federal revenues: individual income taxes, payroll taxes, corporate income taxes, excise taxes, customs duties, estate and gift taxes, and miscellaneous receipts (including remittances from the Federal Reserve System and various fees and fines). CBO evaluated those specific projections of each source of federal revenues separately.

Measures of the Quality of Projections

This evaluation focuses on three measures of CBO's forecast quality: centeredness, accuracy, and dispersion.

Centeredness. CBO intends to provide a revenue baseline that is centered — that is, one that is as likely to be higher than actual revenues as it is to be lower. To measure the centeredness of its past projections, CBO uses the average error (the arithmetic mean of the projection errors), which is the simplest and most widely used measure of forecast centeredness. But because the negative values of underestimates offset the positive values of overestimates when calculating the average error, the measure provides an imperfect view of the accuracy of projections. For example, a set of projections with small errors in both directions that largely offset one another would produce a small average error; so, too, would a set of projections with large overestimates and underestimates, as long as the errors were approximately the same size.

Accuracy. The accuracy of a set of projections is a measure of how much projected values differ from the actual outcomes. CBO evaluates the accuracy of revenue projections using two measures: the average absolute error and the root mean square error. To calculate the average absolute error, negative signs are removed from underestimates before averaging; thus, unlike in the calculation of the average error, errors in different directions do not offset one another.

The RMSE is calculated by squaring the errors (thus removing the negative signs), averaging those squares, and taking the square root of that average. Like the average absolute error, the RMSE measures the size of errors without regard to direction, but because the errors are squared, the RMSE places a greater weight on larger projection errors.

Dispersion. To measure the dispersion of its projection errors, CBO uses the two-thirds spread of errors — that is, the difference between the minimum and maximum error after removing the largest (most positive) one-sixth and smallest (most negative) one-sixth of all errors. A larger two-thirds spread implies a more uncertain forecast, whereas a smaller two-thirds spread implies that the actual outcome is likely to fall within a smaller range around the projected value.¹³

Limitations of Projection Evaluations

The measures of quality that CBO uses have limitations. To some degree, error is inherent to every projection because the data used to produce the projections may contain measurement error. Modifications that CBO has made to its projection methods over the years might have changed the nature of the errors, complicating any evaluation of the agency's projection errors. Finally, because of the highly uncertain nature of the economy, the conclusions that can be drawn from the measures are limited by the small sample size.

Measurement Error. The quality of the data that are used to project revenues can vary for many reasons. In some cases, data are collected by sampling from the population; that method of collection is useful for producing representative estimates for the entire population of the United States, but it inherently contains a margin of error.¹⁴ To avoid disclosing any confidential personal data, surveys might also use techniques — such as masking extreme values — that can introduce error into the analysis.

Methods. CBO has significantly changed its methods for projecting revenues over the years. But the agency's attempts to improve its projections by incorporating new information might have changed the characteristics of the resulting projection errors.

The calculated projection errors are themselves uncertain because the estimates of the effects of subsequently enacted legislation that are used to adjust revenue projections are not certain. What appears to be an error in a baseline projection may actually be an error in the estimate of the revenue effects of changes in law. The estimates of the effects of new legislation exhibit error in part because they use only information available at the time of enactment — not information (such as that from tax returns or actual data on GDP or other macroeconomic indicators) that became available later. Furthermore, even when all actual data are available after the fact, analysts cannot calculate the effects on revenues of past legislative changes with complete certainty.

Sample Size. The conclusions that can be drawn from this evaluation are also limited by the number of projections available. Fewer than 40 annual projections is a relatively small sample size for statistical analysis given the variability in the economy and the many different economic factors that interact with one another to determine the amount of revenues collected.

Assessment of the Projections

CBO's projections for the budget year have tended to be more centered and more accurate than its projections for the sixth year. The same is true of the Administration's revenue projections. In general, the accuracy of CBO's budget-year and sixth-year projections has remained relatively constant over time; the largest projection errors have resulted from downturns in the business cycle.

Budget-Year Projections

CBO's budget-year projections have tended to slightly overestimate revenues; most of the overestimates occur in projections made just before a business cycle downturn. Of the projections that the agency has published since 1982, the largest errors were in the projections made just before the 2001 recession and the 2007–2009 recession for the years during and following those recessions.

Centeredness. On average, CBO's budget-year revenue projections for the years 1983 to 2019 overestimated revenues by 1.2 percent; an error of that size in CBO's budget-year projection for 2019 would amount to $42 billion of the roughly $3.5 trillion in total revenues projected.¹⁵

That relatively small average error of 1.2 percent does not capture the substantial variability in the errors that occurs over time. The errors in the projections that CBO made for years that fell in the middle of major business cycle downturns or that immediately followed them are much larger than that average (see Figure 2). Only 8 of the 37 budget-year projections had errors of 1.2 percent or less (regardless of the direction of the error). Just over half of the projections were overestimates. The largest error in the budget-year projections — 24.9 percent — is that in the budget-year projection for 2009, which was published in January 2008. Indeed, if that single projection is excluded from the analysis, the average error for the budget-year projections is an overestimate of 0.6 percent rather than 1.2 percent.

Over the past few decades, the centeredness of the Administration's budget-year revenue projections has been similar to that of CBO's projections. By CBO's calculation, the errors in the Administration's budget-year revenue projections averaged 1.7 percent, slightly higher than CBO's average of 1.2 percent.

Figure 2.

Errors in CBO's and the Administration's Budget-Year Revenue Projections.

Accuracy. The average absolute error for CBO's budget-year projections is 4.9 percent, and the RMSE is 7.0 percent — both of which are much larger than the average error of 1.2 percent. In this case, the fact that both the RMSE and average absolute error are notably larger than the average error indicates a significant degree of offsetting of overestimates and underestimates.

The RMSE of CBO's budget-year projections has remained relatively constant over time — the largest errors appear to be attributable to specific hard-to-predict events in recent years rather than to a broader trend of deterioration in the accuracy of CBO's projections. In particular, the largest errors occurred in projections for years following the 2001 recession and, to an even greater extent, the 2007–2009 recession, both of which involved large and sustained declines in the stock market.

The Administration's budget-year revenue projections have been about as accurate as CBO's projections. By CBO's calculation, the Administration's budget-year projections have an average absolute error of 5.2 percent and an RMSE of 7.4 percent.

Dispersion. The two-thirds spread of CBO's budget-year projections' errors was 11.9 percent. That measure suggests that, on the basis of the historical pattern of errors, there is a two-thirds chance that future budget-year revenues would be no more than about 6 percent above or below the baseline projections if current laws generally remained unchanged. The two-thirds spread of the errors in the Administration's budget-year projections is similar to that of the errors in CBO's budget-year projections.

Sixth-Year Projections

CBO's sixth-year revenue projections have been less centered and less accurate than its budget-year projections. Unlike the budget-year projections' errors, which include a substantial amount of offsetting of underestimates and overestimates, large overestimates dominated the errors in the projection for the longer horizon. In total, 24 of the 33 sixth-year projections overestimated revenues. The predominance of overestimates and their magnitude resulted in less centered and less accurate projections.

Figure 3.

Errors in CBO's and the Administration's Sixth-Year Revenue Projections

Centeredness. CBO overestimated revenues in its sixth-year projections for 1987 to 2019 (those published between 1982 and 2014) by an average of 5.6 percent (see Figure 3). Those 33 projections include overestimates attributable to recessions (the projections for 2003 and 2004 and for 2009 to 2014) and underestimates for years in the middle of economic expansions (the projections for 1997 to 2000). If the projections for 2009 to 2014 (whose errors averaged 17.5 percent) are excluded, CBO still overestimated revenues in its sixth-year projections, on average, but by a much smaller 3.0 percent.

The average errors for CBO's projections grow progressively larger as those projections extend further into the future. On average, CBO underestimated revenues for the current year by less than 0.1 percent, but for all other projection years, the agency tended to overestimate revenues. The average error for the budget-year projections is an overestimate of 1.2 percent, and that average error increases to 2.5 percent for the third-year projections, 3.7 percent for the fourth-year projections, 4.8 percent for the fifth-year projections, and 5.6 percent for the sixth-year projections (see Table 1 on page 2).

Accuracy. The RMSE and average absolute error also tend to be larger for projections extending further into the future, although there is evidence that the errors level off after the sixth-year projections. The RMSE of CBO's current-year revenue projections is 2.9 percent; it increases to 7.0 percent for the budget-year projections and to 9.4 percent for the third-year projections. Although the RMSE continues to grow with each year that the projection is extended through the sixth year, it does so at a slower rate, reaching 10.4 percent for the fourth-year projections, 11.2 percent for the fifth-year projections, and 11.6 percent for the sixth-year projections. A similar pattern exists for the average absolute error, which is 9.9 percent for the agency's sixth-year projections.

The Administration's sixth-year projections were slightly less accurate, as measured by the average absolute error and RMSE, than CBO's sixth-year projections. The Administration's sixth-year projections have an average absolute error of 11.5 percent and an RMSE of 13.2 percent, according to CBO's calculations. In part, those values reflect the Administration's including in its projections the effects that its proposed policies are projected to have on economic outcomes. If those policies are not enacted, the economic effects do not materialize.

Dispersion. The two-thirds spread of the errors in CBO's sixth-year projections is 19.8 percent. That spread suggests that there is a two-thirds chance that actual revenue collections in the sixth year would be no more than 10 percent greater than or less than the amount in the agency's sixth-year projection if current laws generally remained unchanged. The errors in the Administration's sixth-year projections were slightly less dispersed than CBO's; they have a two-thirds spread of 18.6 percent.

Projections Since the 2007–2009 Recession

The period from the 2007–2009 recession through 2019 was marked by continuous but modest economic growth. And after falling in 2008 and 2009, revenue collections — particularly individual income taxes, payroll taxes, and corporate taxes (the three major sources of revenues) — rose steadily through 2019. Because there had not been a downturn in the business cycle between the 2007–2009 recession and 2019, CBO's budget-year revenue projections for the years since 2010 have been more centered and their errors less dispersed than those of earlier years. (For an analysis of the possible effects of the 2020 coronavirus pandemic on revenues and CBO's projections, see Box 2.)

Whereas the average error for the budget-year projections for 1983 to 2010 is 1.4 percent, the average drops to 1.2 percent when projections for budget years after 2010 are included.¹⁶ The accuracy of the projections has improved since 2009, and the errors in the projections for 2011 to 2019 were less variable than were those in the projections for the entire 1983–2019 period as a whole: The average absolute error for the projections for 2011 to 2019 is 2.1 percent, and the RMSE is 2.7 percent — significantly lower than the average absolute error of 4.9 percent and RMSE of 7.0 percent calculated for all 37 of CBO's budget-year projections that were analyzed.

The average error for CBO's sixth-year projections for 1987 to 2014 is 5.3 percent, slightly smaller than the full sample's average error of 5.6 percent. Unlike the budget-year projections, the sixth-year revenue projections do not improve soon after economic downturns, in part because many of the those projections were made in the years before the corresponding recession began and thus reflected the historical trends that were used to make projections until it became clear that a downturn had occurred. Nevertheless, the accuracy of the sixth-year projections has improved since the 2007–2009 recession, and the errors in the projections for 2015 to 2019 were less dispersed.

Serial Correlation in CBO's Projection Errors

CBO's projection errors show some degree of serial correlation — that is, projections tend to err in the same direction in consecutive years. For example, one overestimate of budget-year revenues is likely to be followed by another overestimate of budget-year revenues the next year. That tendency suggests that as new information becomes available, CBO may not be incorporating all of it into the agency's projections (in part because of the difficulty of determining whether a deviation from the historical pattern marks a persistent change in the trend or is only a temporary deviation). Projections produced shortly before and during economic downturns tended to overestimate revenues in the short term and were repeatedly revised downward, whereas projections made during periods of economic booms produced underestimates that were repeatedly revised upward. To some degree, those projection errors were related to the business cycle, but even after a shift from expansion to recession became apparent, it still took time for CBO to incorporate that information and the related effects on revenues into its projections.

Although that pattern could be evidence that CBO is not optimally incorporating new information into its projections (particularly those for the budget-year), serial correlation can result from many factors. The errors in projections for a year that is multiple years in the future will naturally exhibit some degree of serial correlation. Because projections of revenues are based on current information, the error in a projection for a year that is several years in the future will compound the random error (the error stemming from what is unknown about the future) in the projections for each of the years before it that were prepared at the same time. For example, the error in a projection for the third year of the projection period will include, to some degree, the random error in the current-year and budget-year projections as well as the random error in the third-year projection itself. Thus, all else equal, the sixth-year projections have larger errors than the budget-year projections.

Constraints on CBO's ability to acquire and incorporate new information in its projections also contribute to the serial correlation of its projection errors. Detailed data from tax returns, for example, are typically not available to CBO until at least two and a half years after the first related tax payments are made to the Treasury. Such lags in the availability of information make it difficult for CBO and other forecasters to recognize structural breaks in trends in a timely manner.¹⁷

CBO's analysis of the serial correlation of its projection errors suggests that, as expected, the budget-year projections of revenues better incorporate the latest information than do the sixth-year projections.¹⁸ In the past, analysts from outside CBO have found similar results concerning the serial correlation in the agency's projections. One such study of the projections that CBO made between 1985 and 1999 found that revisions to those projections were serially correlated and thus concluded that the agency was not optimally incorporating new information into its projections.¹⁹ Other studies have come to similar conclusions about CBO's projections.²⁰

Although it is evident that CBO's projection errors exhibit some serial correlation, that correlation has varied over the past 30 years. Adjusting for the average correlation over the entire period would therefore have improved the accuracy of some projections and worsened it for others. Overall, the accuracy of CBO's projections, as measured by the RMSE, would not have been systematically improved had the agency adjusted its projections on the basis of the past pattern of revisions; in fact, that method would have overcompensated for the correlation and caused the projections to err in the opposite direction in many cases.

Differences Between This Analysis and CBO's 2015 Report on Its Revenue Forecasting Record

This report incorporates five more years of revenue projections than the 2015 report, extending the analysis to include the agency's projections for 2019.²¹ Since the publication of the previous report, the average error for CBO's budget-year projections has increased slightly, but the average absolute error and RMSE have declined. Together, those three measures suggest that the agency has continued to systematically overestimate revenues but that its projection errors have been less dispersed in recent years than they were in the past. For the sixth-year projections, the three error measures followed a similar pattern.

The single largest change in tax policy since 2015 was the 2017 tax act (Public Law 115-97), which made significant changes to the tax code.²² (For a discussion of previous changes in tax law and how they affected CBO's projection errors, see Box 3.) CBO's budget-year projection for 2018 (made in 2017) had an error of 3.3 percent, and its sixth-year projection for that year (made in 2013) had an error of 11.9 percent. Those projections were adjusted to account for the estimated effects of the 2017 tax act before the errors were calculated. The error in the budget-year projection for 2018 is less than the average absolute error for the budget-year projections (4.9 percent), but the error in the sixth-year projection is greater than the average absolute error for the agency's sixth-year projections (9.9 percent).

Some Sources of Projection Errors

CBO's forecast of the economy — particularly the agency's projections of GDP — is a primary input into the agency's revenue projections, so errors in that forecast can translate into errors in the revenue projections. The errors in the economic forecast stem mostly from the difficulty of anticipating turning points in the business cycle and of recognizing changes in fundamental economic relationships or long-term trends.

Errors in other components of the agency's economic forecast — including changes in the composition of national income, the relationship between national income and the associated tax bases, the relative growth rates of asset prices and GDP, the distribution of income among taxpayers, and the effects of recent changes in policies — can also contribute to errors in CBO's projections of revenues as a percentage of GDP, as can other unanticipated developments that are not related to the economy.

Errors Related to Projections of GDP

CBO's revenue projections largely hinge on the projections of the overall state of the economy, typically measured by GDP and national income, and those economic forecasts, in turn, rely on forecasts of wages and salaries, corporate profits, and noncorporate business income. Historically, revenue projections made around business cycle peaks have had the largest errors. For example, in the budget-year projections that CBO released in early 1982, 1990, 2001, and 2008 — near the beginning of recessions — the agency significantly overestimated the amount of revenues that would be collected in 1983, 1991, 2002, and 2009. (In each of those cases, the lag between economic activity and the payment of taxes caused the largest discrepancies between projected and actual revenues to occur after the recession began.) Those four projections overestimated revenues by between 9 percent and 25 percent. The RMSE of those four projections is 16.8 percent, whereas the RMSE of all of CBO's other budget-year projections is 4.6 percent (see Figure 4). The errors in the Administration's projections have exhibited a similar pattern.

As it does with its revenue projections, CBO regularly assesses its past economic forecasts.²³ From those assessments, the agency has gleaned the following key insights about the errors in its economic forecasts:

• Forecast errors tend to be large around business cycle peaks, just before a recession begins. In some cases, the error occurred because the recession was prompted by shocks that were unforeseeable when the forecast was made; in others, widely different forecasts appeared equally probable because the recession occurred during a period of especially great economic uncertainty.

• Trends in productivity, one key variable underlying CBO's forecasts of potential and real GDP, can change in unexpected ways. Two important determinants of productivity growth, in particular — shifts in capital accumulation and the effects of specific policies on incentives to invest — can be difficult to identify until several years after they occur.

Errors Related to Other Factors

Errors in projections of revenues as a percentage of GDP can also be related to the difficulty of projecting the components of national income for a given level of GDP. National income comprises income from various sources that are taxed at different effective rates. Those sources of income can vary in terms of how fast they grow, how volatile they are, and to whom they accrue — all of which can affect revenues.

Further, growth rates of those income sources calculated on the basis of amounts reported on tax returns may differ from those calculated on the basis of income in the national income and product accounts compiled by the Bureau of Economic Analysis (BEA). Wages and salaries, for example, may differ because BEA includes wages of workers who do not file tax returns or who misreport their wages. Similarly, taxable profits reported on corporate tax returns may differ from the profits recorded in the national income accounts because BEA calculates aggregate profits by subtracting the losses of unprofitable businesses from the profits of profitable ones, but unprofitable businesses do not typically pay corporate income taxes. When estimating the growth rates of income sources, CBO uses data on historical patterns to partially account for those differences, but any deviations from the historical patterns introduce error into the projections.

Figure 4.

Root Mean Square Errors of CBO’s and the Administration’s Budget-Year Revenue Projections Produced

Near Business Cycle Peaks Compared With Those of Projections Produced at Other Times

The difficulty of projecting the distribution of household income — and especially of specific income sources — contributes to the errors in the revenue projections. If, for example, the amount of wages and salaries that accrues to high-income taxpayers (who face the highest tax rates) is higher or lower than CBO projects, total revenues will differ from the agency's projections. In past recessions, certain volatile categories of taxable income — such as corporate profits and capital gains realizations — declined in relation to GDP, and the highest-earning taxpayers unexpectedly earned a smaller share of overall income, which pushed down effective tax rates.²⁴ Also, errors in projections of components of the tax base that correspond to income from previous years — taxable pension distributions and capital gains, for example — contribute to errors in projections of revenues as a percentage of GDP.

Finally, certain other developments unrelated to either new legislation or to the state of the economy can also contribute to projection errors. The Administration may take unanticipated actions in setting policy or issuing regulations. For example, the President has broad discretion in setting tariff policy, which affects the amount of customs duties that are collected. Finally, legal decisions in tax cases and the Internal Revenue Service's published rulings interpreting the tax code can affect the size of the tax base in ways that CBO did not anticipate by, for example, broadening a tax provision's scope.

Comparison of Errors Related to Projections of GDP With Those Related to Other Factors

To examine the relative importance of errors in projections of the state of the economy versus other factors in determining the size of errors in its revenue projections, CBO compared the error measures of its forecasts of GDP with those of its projections of revenues as a share of GDP. The agency concluded that the errors in the projections of revenues as a percentage of GDP are larger than the errors in the GDP projections for shorter horizons but that the errors in the two types of projections tend to converge over longer horizons (see Table 2). Specifically, the agency found the following:

• The RMSE of CBO's budget-year projections of revenues as a percentage of GDP is 5.2 percent, whereas the RMSE of forecasts of GDP is 2.4 percent.²⁵

• At the fourth-year projection horizon, the average error for the GDP forecasts and the average error for the projections of revenues as a percentage of GDP are nearly equal; the RMSEs of the two sets of projections begin to converge, but that of the projections of revenues as a percentage of GDP (6.6 percent) still exceeds that of the forecasts of GDP (4.5 percent).

• At the sixth-year projection horizon, the average error for the projections of revenues as a percentage of GDP equals the average error for the fourth-year projections, whereas the average error for the GDP forecasts continues to increase. The RMSE of CBO's sixth-year projections of GDP (6.6 percent) exceeds that of the agency's sixth year projections of revenues as a percentage of GDP (6.2 percent).

Because the measure revenues as a percentage of GDP is itself a percentage, the projection errors associated with it can also be expressed in percentage points (that is, as the difference between the projected and actual percentages of GDP). CBO's sixth-year projections of revenues as a percentage of GDP (which averaged 18.6 percent of GDP over the years covered in this analysis) have an RMSE of 1.2 percentage points and an average absolute error of 1.0 percentage point.

Interactions Between Different Sources of Errors

Errors in CBO's forecasts of GDP have tended to move together with errors in its projections of revenues as a percentage of GDP (see Figure 5). At turning points in the business cycle, misestimates of revenues as a percentage of GDP have tended to be more cyclical than misestimates of GDP.

The relationship between misestimates of GDP and those of other revenue-determining factors appears strongest during business cycle downturns, but it is also evident during cyclical booms. During recessions, CBO has overestimated not only GDP but also key components of the tax base relative to GDP, such as capital gains realizations, pension distributions, and corporate profits. The overestimates of those components have often exceeded, in percentage terms, the overestimate of GDP. Misestimates of GDP and misestimates of revenues as a percentage of GDP have exhibited a similar relationship even when the economy was not experiencing a cyclical downturn. For example, in the late 1990s, an unexpected cyclical boom resulted in CBO's underestimating GDP and, to a greater extent, revenues as a percentage of GDP.

Errors in Projections of Different Sources of Federal Revenues

The amount of uncertainty about factors affecting revenues varies substantially by revenue source. Of CBO's budget-year revenue projections, those of corporate income taxes have the highest RMSE — 27.6 percent — and those of payroll taxes, the lowest — 3.3 percent (see Figure 6). The RMSE of the projections of corporate receipts is enlarged by a few sizable errors; the median absolute error is much smaller — 11.4 percent — and the average absolute error is 17.5 percent.²⁶

Table 2. Error Measures for CBO's Projections of GDP, Revenues as a Percentage of GDP, and Total Revenues

Percent
	Projection Year
	Budget Year	Fourth Year	Sixth Year
GDP
Average error	0.1	1.7	3.5
Average absolute error	1.7	3.4	5.4
RMSE	2.4	4.5	6.6
Revenues as a Percentage of GDP
Average error	1.1	1.8	1.8
Average absolute error	3.7	5.4	5.0
RMSE	5.2	6.6	6.2
Total Revenues
Average error	1.2	3.7	5.6
Average absolute error	4.9	8.5	9.9
RMSE	7.0	10.4	11.6
Addendum Revenues as a Percentage of GDP (Percentage points)
Average error	0.2	0.3	0.3
Average absolute error	0.8	1.1	1.0
RMSE	1.2	1.3	1.2
Source: Congressional Budget Office. The projection errors underlying these summary measures are calculated as the projected amount minus the actual amount, divided by the actual amount; thus, the error is expressed as a percentage of the actual amount. A positive average error is an overestimate. The average absolute error is the average of projection errors with the negative signs removed from the underestimates. The RMSE is calculated by squaring the projection errors, averaging those squares, and taking the square root of that average. CBO typically publishes its annual baseline projections in January or February. The budget year is the fiscal year beginning eight to nine months after the projection is made; the fourth year is the fiscal year beginning two years and eight to nine months after the projection was made, and the sixth year is the fiscal year beginning four years and eight to nine months after the projection is made. The errors for projections of GDP and for projections of revenues as a percentage of GDP do not add to up to the errors for projections of total revenues because of interactions between the two. RMSE = root mean square error.

CBO's projections of individual income tax receipts, the largest source of federal revenues, have an RMSE of 9.6 percent. The projections of customs duties and of excise taxes have RMSEs of 11.0 percent and 8.6 percent, respectively. And the projections of miscellaneous receipts (fees and fines and Federal Reserve earnings) and of estate and gift taxes have RMSEs of 15.1 percent and 17.0 percent, respectively.

The difference in outcomes by revenue source is largely explained by characteristics of the tax bases for the different revenue sources. The tax bases vary in terms of stability and predictability. The primary tax base for payroll taxes is wages and salaries, and it tends to change at more stable rates over time than most other tax bases, making it generally more predictable. Corporate profits, on the other hand, fluctuate significantly over the business cycle and are notoriously difficult to project accurately. It is therefore not surprising that projections of payroll taxes have been the most accurate and projections of corporate income taxes the least accurate. Individual income taxes — the largest single source of revenues (accounting for 45.9 percent of total revenue collections, on average, over the 1983–2019 period) — are based primarily on wages and salaries, but they also include taxes on other sources of income, such as proprietor's business income and capital gains realizations, which can vary significantly from year to year because of price changes and other market forces.

Figure 5.

Errors in CBO's Budget-Year Projections of GDP and Revenues as a Percentage of GDP

The projections of the smaller sources of revenues — excise taxes, estate and gift taxes, customs duties, fees, and other miscellaneous sources, all of which combined have averaged less than the amount of receipts from corporate income taxes — depend on factors related to specific areas of the economy that can vary significantly and change unexpectedly. For example, estate and gift taxes are assessed on the wealth of large estates, which can fluctuate widely with asset prices. Revenues from miscellaneous fees and fines can vary substantially for industry-specific reasons.

The relative contribution of each source to the total projection error depends on the size of the source's share of total revenues and the size of the projection error associated with the source. The projections for specific tax sources that contribute the most to the errors in projections of total revenues are not necessarily those with the largest errors. A relatively small error in a projection of a source that accounts for a large share of federal revenues can result in a large error in the projection of total revenues, just as a large error in a projection of a source that accounts for a small share of revenues can result in a small error in the projection of total revenues.

CBO calculated the error in the projections of each revenue source relative to the error in projections of total revenues to determine which source-specific projections have contributed the most to the errors in the projections for total revenues. By that metric, projections of revenues from the largest source, individual income taxes, which have an RMSE relative to total revenues of 4.4 percent, have contributed the most to the errors in CBO's budget-year projections of total revenues (see Table 3). The projections of revenues from the third largest source, corporate income taxes, have an RMSE relative to total revenues (2.3 percent) that is larger than that of the second-largest source, payroll tax receipts (1.3 percent) — even though corporate income tax receipts have averaged less than one-third of the amount of payroll tax receipts over the past few decades. The relatively large variability of corporate income taxes over the business cycle magnifies their importance in determining total accuracy. The projections of smaller revenue sources have contributed much smaller amounts to projection errors for total revenues; the RMSE relative to total revenues of projections of each of those sources — excise taxes and customs duties, for example — is 0.5 percent or less.

Figure 6.

Root Mean Square Errors of CBO's and the Administration's Budget-Year Revenue Projections, by Revenue Source

Table 3. Root Mean Square Errors of CBO's Budget-Year Projections, by Revenue Source

Percent
	RMSE	Average Share of Total Revenues	RMSE Relative to Total Revenues
Individual Income Taxes	9.6	45.9	4.4
Payroll Taxes	3.3	35.9	1.3
Corporate Income Taxes	27.6	9.7	2.3
Excise Taxes	8.6	3.6	0.4
Customs Duties	11.0	1.3	0.2
Estate and Gift Taxes	17.0	1.0	0.2
Miscellaneous Receiptsa	15.1	2.6	0.5
Total Revenues	7.0	100	n.a.
Source: Congressional Budget Office. The projection errors underlying the RMSE of projections of revenues from each source are calculated as the projected amount of revenues from the specific source minus the actual amount, divided by the actual revenues from that source. The projection errors underlying the RMSE relative to total revenues are calculated as the projected amount of revenues from the specific source minus the actual amount, divided by actual total revenues from all sources; that measure allows the errors in projections of each source to be compared with one another. The RMSE is calculated by squaring the projection errors, averaging those squares, and taking the square root of that average. The RMSE of projections of revenues from a single tax source relative to total revenues can be approximated by multiplying together the RMSE of projections of revenues from that source and that source's average share of total revenues over the period. However, the more that a revenue source's share of total revenues varies over the period, the less precise that approximation becomes. The amount shown for the RMSE relative to total revenues is the precise calculation. The revenue projections included in this analysis are those for fiscal years 1983 to 2019, which were generally published in January or February of the previous year. RMSE = root mean square error. a. Includes remittances from the Federal Reserve System and various fees and fines.

List of Tables and Figures

Tables

1.Summary Measures of the Quality of CBO's and the Administration's Revenue Projections2

2. Error Measures for CBO's Projections of GDP, Revenues as a Percentage of GDP, and Total Revenues

3. Root Mean Square Errors of CBO's Budget-Year Projections, by Revenue Source

Figures

1. Sample Timeline for Measuring Errors in CBO's Budget-Year Revenue Projections

2. Errors in CBO's and the Administration's Budget-Year Revenue Projections

3. Errors in CBO's and the Administration's Sixth-Year Revenue Projections

4. Root Mean Square Errors of CBO's and the Administration's Budget-Year Revenue Projections Produced Near Business Cycle Peaks Compared

With Those of Projections Produced at Other Times

5. Errors in CBO's Budget-Year Projections of GDP and Revenues as a Percentage of GDP

6. Root Mean Square Errors of CBO's and the Administration's Budget-Year Revenue Projections, by Revenue Source

About This Document

Each winter, the Congressional Budget Office issues a report on the state of the budget and the economy. This document provides background information on the centeredness, accuracy, and dispersion of the revenue projections included in those reports. In keeping with CBO's mandate to provide objective, impartial analysis, the report makes no recommendations.

James Williamson prepared the report with guidance from John McClelland, Joseph Rosenberg, and Joshua Shakin. Aaron Feinstein and Edward Gamber provided comments. Natsuki Arai from the National Chengchi University (Taiwan) and Byron Lutz from the Board of Governors of the Federal Reserve System also provided feedback. (The assistance of external reviewers implies no responsibility for the final product, which rests solely with CBO.) Tess Prendergast fact-checked the report.

Jeffrey Kling and Robert Sunshine reviewed the report. The editor was Bo Peery, and the graphics editor was Robert Rebach. This report is available on CBO's website (www.cbo.gov/ publication/56499).

CBO continually seeks feedback to make its work as useful as possible. Please send any comments to communications@cbo.gov.

Phillip L. Swagel
Director
August 2020