Are BHCs Mimicking the Fed’s Stress Test Results?   Liberty Street Economics
Liberty Street Economics

« From the Vault: Understanding Puerto Rico’s Economic Challenges | Main | Do Better Bank Services Have a Hidden Cost? »

September 21, 2015

Are BHCs Mimicking the Fed’s Stress Test Results?



LSE_2015_are-bhcs-mimicking-the-feds-stress-test-results_460_art

In March, the Federal Reserve and thirty-one large bank holding companies (BHCs) disclosed their annual Dodd-Frank Act stress test (DFAST) results. This is the third year in which both the BHCs and the Fed have published their projections. In a previous post, we looked at whether the Fed’s and the BHCs’ stress test results are converging in the aggregate and found mixed results. In this post, we look at stress test projections made by individual BHCs. If the Fed’s projections are very different from a BHC’s in one year, do the BHC projections change in the following year to close this gap? Or are year-to-year changes in BHC stress test projections driven more by changes in underlying risk factors? Evidence of BHCs mimicking the Fed would be problematic if it meant that the BHCs are not really independently modelling their own risks. Convergence poses a potential risk to the financial system, since a financial system with monoculture in risk measurement models could be less stable than one in which firms use diverse models that collectively might be more likely to identify emerging risks.

Persistence over Time
As a first step, we look at the correlation in BHC stress test projections over time, asking whether a BHC’s stress test projections are more highly correlated with its own lagged projections or with the Fed’s previous projections. Specifically, we estimate the following equation:

Equation 1

where Yi,tBHC is BHC i’s projection of some stress test variable for year t, Yi,tFED is the Fed’s projection of that variable for BHC i in year t, and ϒt is a dummy variable for each year t. The coefficients βBHC and βFED measure the correlation between the BHCs’ projections in year t and the BHCs’ and Fed’s projections in year t-1, respectively. This equation essentially creates a “horse race” to identify whether the BHCs’ projections are more correlated with their past projections or with the Fed’s. A finding that the Fed’s lagged projections mattered more would be consistent with the idea that the BHCs are trying to mimic the Fed’s stress test results. We estimated the equation using data from all DFAST BHCs between 2013 and 2015, but since we have lagged variables on the right-hand side, the dependent variables are based on 2014 and 2015 data.

The table below presents the results of these estimates for three important variables in the severely adverse macroeconomic scenario: pre-tax net income, pre-provision net revenue, and total loan losses, all scaled by total assets or total loans. The results indicate that the BHCs’ projections are significantly correlated with their own past projections, but not with the Fed’s; the coefficients on the lagged BHC projections are all positive and statistically significant, while the coefficients on the lagged Fed variables are much smaller and are statistically insignificant.


Correlation between Current and Lagged BHC and Federal Reserve Stress Test Projections

Closing the Gap?
These results suggest a relatively high degree of persistence in the BHCs’ projections, similar to the persistence in the Fed’s projections found by Glasserman and Tangirala. We therefore look to see if the change in the BHCs’ forecasts in a given year is associated with the gap between the BHC’s and Fed’s projections the year before; that is, if the Fed’s projection of net income, PPNR, or losses is much higher than a BHC’s, does the BHC’s projection tend to increase in the following year? Or are year-over-year changes more strongly related to factors such as changes in condition or risk at the BHC or changes in the macroeconomic scenario envisioned in each stress test, as one would hope?

The three charts plot the change in the BHCs’ projections of net income, PPNR, and total loan losses against the difference between the Fed’s and the BHCs’ projections of these variables in the previous year. If the BHCs were attempting to mimic the Fed’s results, we would expect a positive correlation between these two variables. At the extreme, all points would be in non-shaded regions along the 45 degree line, indicating a one-for-one change in BHC projections to close the Fed–BHC gap. Instead, we see many points in the shaded regions of charts. For example, for loan loss rates, when the Fed has higher loan loss rate forecasts than the BHC, BHCs actually have lower loss projections in the next year almost half of the time (these are the points in the lower right-hand quadrant of the chart).


BHCs projections of net income, PPNRR


PPNR


Total loan losses against


The table below goes further by presenting results from a regression that controls for other factors that could affect how projections change from year-to-year:

Equation 2

where ΔYi,tBHC and ΔYi,tFED are the changes in BHC i’s and the Fed’s projection of variable Y between year t and year t-1, and Yi,t-1BHC and Yi,t-1FED are the BHC’s and Fed’s projections of stress test variable Y in year t-1. The sign of βGAP indicates how changes in a BHC’s projection relate to the gap between the Fed’s and the BHC’s projections the year before; a positive sign is consistent with mimicking. The equation includes the change in the Fed’s projection from between year t and year t-1 to control for other factors affecting the projections, such as changes to the macroeconomic scenario or the risk profiles of the firms.


Table 2


The findings are mixed. For pre-tax net income and the total loan loss rate, the lagged gap in Fed versus BHC projections is not significantly related to the change in BHC projections the following year. For PPNR, however, there does appear to be a significant relationship, with a higher gap associated with increases in PPNR in the following year.

Notably, there is also little evidence that the changes in BHC projections are correlated with changes in the Fed projections, our control for changes in underlying risk exposures, and other factors that could affect the projections; only the BHCs’ PPNR projections are correlated with the change in the Fed’s projections.

Is the Glass Half Full, or Half Empty?
As with our earlier aggregate results, we find mixed evidence at the individual BHC level on whether BHCs’ stress test projections are converging toward the Fed’s. Schuermann has pointed out that BHCs have incentives to imitate the Fed’s stress test models. We observe convergence in some areas but not in others—a mixed message for those concerned about model monoculture. We do see some convergence in revenue projections; BHC’s forecasts move closer to what the Fed projected in the previous year. However, little of the explanatory power in the regression comes from the gap between the BHC and Fed’s prior year forecast.

By contrast, the BHCs seem to be sticking to their own models for loan losses, with projections that are closer to their own prior projections than the Fed’s. Nor did we find any evidence of convergence in pretax net income, the ratio that matters most to capital, and hence, to solvency.


Disclaimer
The views expressed in this post are those of the authors and do not necessarily reflect the position of the Federal Reserve Bank of New York or the Federal Reserve System. Any errors or omissions are the responsibility of the authors.





Angela Deng is a senior research analyst in the Bank’s Research and Statistics Group.

Beverly HirtleBeverly Hirtle is a senior vice president in the Federal Reserve Bank of New York’s Research and Statistics Group.


Anna KovnerAnna Kovner is an assistant vice president in the Bank’s Research and Statistics Group.
Posted by Blog Author at 07:00:00 AM in Bank Capital, Financial Institutions, Forecasting
Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

The comments to this entry are closed.

About the Blog
Liberty Street Economics features insight and analysis from New York Fed economists working at the intersection of research and policy. Launched in 2011, the blog takes its name from the Bank’s headquarters at 33 Liberty Street in Manhattan’s Financial District.

The editors are Michael Fleming, Andrew Haughwout, Thomas Klitgaard, and Donald Morgan, all economists in the Bank’s Research Group.

The views expressed are those of the authors, and do not necessarily reflect the position of the New York Fed or the Federal Reserve System.


Economic Research Tracker

Liberty Street Economics is now available on the iPhone® and iPad® and can be customized by economic research topic or economist.


Useful Links
Comment Guidelines
We encourage your comments and queries on our posts and will publish them (below the post) subject to the following guidelines:
Please be brief: Comments are limited to 1500 characters.
Please be quick: Comments submitted after COB on Friday will not be published until Monday morning.
Please be aware: Comments submitted shortly before or during the FOMC blackout may not be published until after the blackout.
Please be on-topic and patient: Comments are moderated and will not appear until they have been reviewed to ensure that they are substantive and clearly related to the topic of the post. We reserve the right not to post any comment, and will not post comments that are abusive, harassing, obscene, or commercial in nature. No notice will be given regarding whether a submission will or will not be posted.‎
Disclosure Policy
The LSE editors ask authors submitting a post to the blog to confirm that they have no conflicts of interest as defined by the American Economic Association in its Disclosure Policy. If an author has sources of financial support or other interests that could be perceived as influencing the research presented in the post, we disclose that fact in a statement prepared by the author and appended to the author information at the end of the post. If the author has no such interests to disclose, no statement is provided. Note, however, that we do indicate in all cases if a data vendor or other party has a right to review a post.
Archives