Liberty Street Economics

« | Main | »

April 11, 2016

Supervising Large, Complex Financial Institutions: Defining Objectives and Measuring Effectiveness

Supervising Large, Complex Financial Institutions: Defining Objectives and Measuring Effectiveness

Last month the New York Fed held a conference on supervising large, complex financial institutions. The event featured presentations of empirical and theoretical research by economists here, commentary by academic researchers, and panel discussions with policymakers and senior supervisors. The conference was motivated by the recognition that supervision is distinct from regulation, but that the difference between them is often not well understood. The discussion focused on defining objectives for supervising the large, complex financial companies that figure so prominently in our financial system and ways of measuring how effectively supervision achieves these goals. This post summarizes the key themes from the conference and introduces the more in-depth posts that will follow in this blog series.

The Goals of Supervision

After opening remarks by William Dudley, the Bank’s president, the agenda began with two sessions exploring the goals of supervision. The first theme to emerge was the crucial, but sometimes misunderstood distinction between supervision and regulation. As Dudley observed, regulation involves defining the rules by which banks operate, including permissible and impermissible activities, minimum capital and liquidity standards, and rules governing ownership and control of commercial banks. Supervision involves monitoring and overseeing the firms. Key elements of supervision include ensuring compliance with law and regulation, assessing whether the firms are involved in practices that could threaten their stability or long-run viability, and ensuring that firms remediate such “unsafe or unsound” practices or conditions. Importantly, a lot of what bank supervisors do is confidential, both the information they assess and their remediation actions.

A second theme to surface was about defining the proper objectives of supervision. What should supervisors be striving for, at each firm and for the financial system as a whole? The Federal Reserve has outlined broad objectives for the supervision of large financial institutions—to lower the probability of a firm’s failure or inability to serve as a financial intermediary and to reduce the impact of such an event on the financial system and economy. Speakers agreed that supervision should address both goals, not merely the first. Further, panelists argued that the goals should also include ensuring that the financial system functions efficiently and provides the credit and other financial services needed to foster a robust economy.

Participants also discussed how supervision might complement regulation by, for example, being more flexible, or adaptable, as market practices evolve and being better able to incorporate and act on “soft information” that might not be codified in “hard” rules. Several speakers stressed the importance of judgment by supervisors, arguing that a key part of their role is to be forward-looking in identifying risks at firms and in the financial system. They asserted that effective supervisors combine technical and interpersonal skills and, importantly, are willing to act under uncertainty even when facing resistance from the firms they supervise. Clear regulatory standards can enhance and empower supervision by providing benchmarks to support supervisory actions. Effective supervision also evolves with events; several speakers cited the introduction of stress testing as a critical new tool arising from the financial crisis.

Assessing Supervisory Effectiveness

The afternoon sessions focused on measuring the impact and effectiveness of supervision. Several participants pointed out the importance of having clear supervisory objectives and defining metrics to measure against those goals, while recognizing the challenge in identifying measures that are prospective or forward-looking. Several speakers endorsed taking a “horizontal perspective” to facilitate the spread of good practices across firms, pointing to programs such as the Comprehensive Capital Analysis and Review (CCAR) and Comprehensive Liquidity Analysis and Review (CLAR) as supervisory innovations that promote such perspectives. Participants also observed that providing bank management and boards of directors with a sense of their firm’s performance relative to peers can be a powerful way to incentivize change. That said, speakers also cited the importance of absolute, not just relative, judgments about whether the practices of each institution are sufficient to ensure stability.

Another issue highlighted by participants was the confidentiality of much of the information used by supervisors and the analysis and actions that derive from it. This confidentiality limits the ability of analysts and researchers outside of the supervisory agencies to independently assess the impact and effectiveness of supervisory actions—or in many cases, to even know what they are. Participants pointed to the potential value of independent assessments and argued that overcoming this opacity without sacrificing essential confidentiality would advance understanding of what supervisors do and how well they do it. In particular, some speakers recommended disclosing issues about individual banks identified by supervisors, perhaps with a delay, or developing methods for researchers to access confidential supervisory data, such as ratings, in controlled settings.

Research on Supervision

The conference featured several papers by New York Fed economists. We provide synopsis of the papers and links below. In the rest of this blog series, the papers’ coauthors will provide more detailed treatments of each topic.

  • The first paper, summarized in an earlier post, describes the Fed’s supervisory approach for large, complex financial institutions and how supervision of such firms is conducted on a day-to-day basis at the New York Fed as part of this broader supervisory program.
  • The second paper presents a theoretical model of supervision and regulation, emphasizing how the two complement one another. The authors use the model to interpret results in the data, in particular, key drivers and trade-offs faced by supervisors in allocating resources across banks holding companies (BHC), as measured by actual hours logged by supervisors.
  • The third paper provides a “look behind the curtain” using computational linguistics techniques to parse the content of thousands of supervisory actions issued by supervisors to banks. It extracts and summarizes the content of the textual messages given by supervisors, and then examines how the frequency of topics (for example, “risk modeling”) varies with key characteristics of the BHCs like size and risk.
  • The final paper attempts to measure whether the intensity of supervision across BHCs affects risk or performance of these firms. Using an assumption about asset-size rank within Federal Reserve districts to identify banks more likely to receive additional supervisory attention, the paper examines a range of outcome measures and finds that more supervisory attention is associated with lower risk, but not disproportionately lower returns or slower asset growth.

Next Posts in This Series:

The Economics of Bank Supervision: So Much to Do, So Little Time
Thomas Eisenbach, David O. Lucca, and Robert Townsend

How Does Supervision Affect Banks?
Beverly Hirtle, Anna Kovner, and Matthew Plosser

A Peek behind the Curtain of Bank Supervision
Paul Goldsmith-Pinkham, Beverly Hirtle, and David O. Lucca


The views expressed in this post are those of the authors and do not necessarily reflect the position of the Federal Reserve Bank of New York or the Federal Reserve System. Any errors or omissions are the responsibility of the authors.

Berverly Hirtle, Federal Reserve Bank of NY
Beverly Hirtle is a senior vice president in the Federal Reserve Bank of New York’s Research and Statistics Group.

Lucca_davidDavid O. Lucca is a research officer in the Federal Reserve Bank of New York’s Research and Statistics Group.

About the Blog

Liberty Street Economics features insight and analysis from New York Fed economists working at the intersection of research and policy. Launched in 2011, the blog takes its name from the Bank’s headquarters at 33 Liberty Street in Manhattan’s Financial District.

The editors are Michael Fleming, Andrew Haughwout, Thomas Klitgaard, and Asani Sarkar, all economists in the Bank’s Research Group.

Liberty Street Economics does not publish new posts during the blackout periods surrounding Federal Open Market Committee meetings.

The views expressed are those of the authors, and do not necessarily reflect the position of the New York Fed or the Federal Reserve System.

Economic Research Tracker

Image of NYFED Economic Research Tracker Icon Liberty Street Economics is available on the iPhone® and iPad® and can be customized by economic research topic or economist.

Economic Inequality

image of inequality icons for the Economic Inequality: A Research Series

This ongoing Liberty Street Economics series analyzes disparities in economic and policy outcomes by race, gender, age, region, income, and other factors.

Most Read this Year

Comment Guidelines


We encourage your comments and queries on our posts and will publish them (below the post) subject to the following guidelines:

Please be brief: Comments are limited to 1,500 characters.

Please be aware: Comments submitted shortly before or during the FOMC blackout may not be published until after the blackout.

Please be relevant: Comments are moderated and will not appear until they have been reviewed to ensure that they are substantive and clearly related to the topic of the post.

Please be respectful: We reserve the right not to post any comment, and will not post comments that are abusive, harassing, obscene, or commercial in nature. No notice will be given regarding whether a submission will or will
not be posted.‎

Comments with links: Please do not include any links in your comment, even if you feel the links will contribute to the discussion. Comments with links will not be posted.

Send Us Feedback

Disclosure Policy

The LSE editors ask authors submitting a post to the blog to confirm that they have no conflicts of interest as defined by the American Economic Association in its Disclosure Policy. If an author has sources of financial support or other interests that could be perceived as influencing the research presented in the post, we disclose that fact in a statement prepared by the author and appended to the author information at the end of the post. If the author has no such interests to disclose, no statement is provided. Note, however, that we do indicate in all cases if a data vendor or other party has a right to review a post.