AI in Finance: How to Approach Financial Regulation
Ask about Knowledge at Wharton
Episode Description
"The Regulatory Challenges of AI in Finance": Episode 3 of 4 in our "Future of Finance" series, hosted by Wharton School finance professor Itay Goldstein
Artificial intelligence is transforming the world of investing. Financial leaders are looking to AI technology to identify and create new sources of value – but at what risk?
In this episode, host Itay Goldstein discusses the benefits of AI in the investment industry – and the ways AI might lead to unintentional collusion. Goldstein is joined by Tobias Adrian, Financial Counsellor and Director of the Monetary and Capital Markets Department of the International Monetary Fund).
----- The Wharton School of the University of Pennsylvania is committed to sharing its intellectual capital through Knowledge at Wharton, the school’s online business analysis journal.
Launched in 1999, Knowledge at Wharton offers free access to:
- Articles, podcasts, and videos highlighting Wharton faculty research and analysis of current business trends - Interviews with book authors from Wharton School Press - In-depth series of curated content like Wharton Executive Education’s Nano Tools for Leaders - A searchable database of more than 10,000 articles covering all aspects of business - A weekly newsletter that delivers Knowledge at Wharton insights directly to your inbox
Visit the Knowledge at Wharton homepage: https://whr.tn/3qUpSBE Subscribe to the Knowledge at Wharton YouTube channel: https://whr.tn/3DEAgAU 📂 Finance & Accounting;Fintech;AI 🏷️ Future of Finance;PTW Related Articles
Brief Summary
Itay Goldstein and Tobias Adrian discuss AI's impact on finance, focusing on opportunities, risks, and regulatory challenges.
Key points
- AI's Impact on Finance. AI is reshaping finance through algorithmic trading and generative models, but risks remain.
- The Black Box of AI. Generative AI presents challenges in understanding decision-making processes and accountability.
- Regulatory Challenges Ahead. Regulators face difficulties in defining and quantifying the risks posed by AI in finance.
- AI Regulation in Finance. Exploring the need for explainability in AI-driven decision making in finance.
- Fairness and Ethics in AI. Singapore's principles focus on responsible AI usage to combat bias.
- GDPR and AI. The adaptation of GDPR to AI practices enhances data protection.
Episode highlights
- AI's Impact on FinanceAI is reshaping finance through algorithmic trading and generative models, but risks remain.“AI is promising to change the way that the financial industry works.”0:26Nov 25, 2024
- The Black Box of AIGenerative AI presents challenges in understanding decision-making processes and accountability.“Generative AI is a bit of a black box.”15:18Nov 25, 2024
- Regulatory Challenges AheadRegulators face difficulties in defining and quantifying the risks posed by AI in finance.“The risk management around using generative AI is certainly a key priority.”15:41Nov 25, 2024
- AI Regulation in FinanceExploring the need for explainability in AI-driven decision making in finance.“AI in finance must be explainable to mitigate risks.”28:00Nov 25, 2024
- Fairness and Ethics in AISingapore's principles focus on responsible AI usage to combat bias.“Responsible usage of AI systems is crucial to avoid bias and discrimination.”28:31Nov 25, 2024
- GDPR and AIThe adaptation of GDPR to AI practices enhances data protection.“GDPR adapts to AI practices for data protection and privacy.”28:47Nov 25, 2024
Transcript
Itay Goldstein: Hello. This is <i>The Future of Finance</i> miniseries here at the Wharton School of the University of Pennsylvania. I am Itay Goldstein. I'm a Finance Professor and the Chair of the Finance Department. The topic that we are going to dive into today is AI in Finance. AI stands for artificial intelligence, and AI is promising to change the way that the financial industry works in many different ways. There are many opportunities, many risks, and there are regulatory concerns, and we're going to talk about all this with Tobias Adrian. Tobias is the financial counselor and the Director of the Monetary and Capital Markets Department at the International Monetary Fund, IMF. Hello,Tobias.
Hi, Itay. - So it's great to have you, and I look forward to hearing your insights. So your division at the IMF has written a report about the use of AI in finance. Can you tell us a little more about what is at stake? What are the opportunities? What are the risks? What issues do you see? Yeah, thanks so much for having me with you today, Itay. So we are— we have a report, which is called the Global Financial Stability Report, that has a special focus on artificial intelligence, or AI in finance. So there are different modes of AI, and some AI techniques have been deployed in finance for many years. In fact, many decades. And have really reshaped the way that finance worked already. And here I'm
particularly thinking about algorithmic trading, which is primarily based on machine learning techniques. So when we're looking at liquid capital markets today, we already see a tremendous impact of artificial intelligence on how those markets are working. So you know, trading activity in— in places such as equity markets in in the US and other advanced economies, or Treasury markets, the most liquid securities in those markets are already largely traded in algorithmic fashion based on artificial intelligence. And, you know, really trading at very high frequencies. What is new in recent years is, of course, the arrival of generative artificial intelligence. And that is, you know, quite different from the
machine learning that has been previously deployed. And so the focus on— of our report is really on understanding to what extent generative AI, including the large language models that many of the listeners today will have used already— to what extent is this going to impact the financial industry? And our particular focus is on trading. So, on capital markets activity. And so I'm happy to go deeper into that. There are also other areas of finance that are already being impacted. And also happy to talk about that a little bit. Yeah. So you know, one of the things that we think about when we think about AI is the agency. That now the AI will basically be a decision maker and will start doing things on their own.
I wonder if some of this is also reflected in the way you're thinking about it, and in your report. Yeah, absolutely. So it is certainly true that in algorithmic trading, you do see a lot of automated decision making at very high frequencies. So there could be, you know, a very large number— you know, thousands of millions of decisions that are— that are fully automated. And, you know, to what extent that will be changed or influenced by generative AI is still work in progress. So let me dig a little bit deeper about, you know, what generative AI is, and how it is being used already, and how it may be used going forward. So, generative AI really is using very large models that are calibrated to
very, very large data in order to elicit answers that are somewhat similar to the way that human intelligence works, which is why the term "artificial intelligence" is being used. And in the large language models, the interaction with those models is via natural language. So the logic is quite different from programming traditionally, where traditionally, you write code, then the computer is doing something, and it's generating output. Here you have— so, like a model that has been calibrated, and then you're working with using the information from that model in order to generate output. And I think the challenge here is twofold. So number one, we don't fully understand how these models really work. They are extremely
powerful, but they're a little bit of a blank box at the moment. So a lot of research today is actually discovering what kind of knowledge is embedded in the generative AI models. And so we are still learning about the technology that is very powerful, but it's still very recent. As a result, the financial sector is still in the process of understanding how those models can be used across the financial sector and for trading activity in particular. And so one policy concern is the lack of explainability and the unpredictability, to some degree, right? Because we know they are so, like— you know, highly complex information is used in a way that may be working similar to the human brain. But how it performs in terms of trading,
for example, when new situations are feeding into— into financial market activity, we don't fully understand. And so there's a certain amount of unpredictability and lack of explainability. And, you know, what the impact is, is— is work in progress. So our sense is that there are tremendous opportunities. So we already see some areas where we think the new technology has a first order impact. So for example, when complex reports are being published. Say, an SEC filing of a publicly traded company, right? These can be very thick, you know, 100 pages long reports. And a generative AI model, you know, can very quickly extract information from the report. Another example is statements by monetary policy makers like the
Federal Open Market Committee of the Federal Reserve. When they publish their policy decision that comes alongside the statement, and the statement now can be analyzed very quickly by generative AI. And so the informational efficiency of markets can be increased, in principle. And I think we see some evidence that that may be the case. So it could be very good for market efficiency. But then there are other areas where we sort of don't know exactly how much efficiency would be improved and what potential pitfalls could be. So there are efficiency gains, but there are also new risks that we have to take into account. So how widely used is it at this point? So our understanding is that— so, like, every financial firm is
exploring it. And there are many startups that are being built around the technology. And in some areas, we do see traction. So for example, what we are hearing is that to do compliance checks, right, or credit risk analysis— so, you know, analytics, where, you know, complex data, some of it from the public domain, like from the internet, some of it from reports such as credit reports, have to be combined, and activity has to be detected. So for example, for payments activity, right? You know, financial institutions put a lot of effort into understanding whether a certain payment may be legitimate or illegitimate. We understand that in those areas, newer AI models can be very effective. And detection rate for fraudulent
activity can— can increase tremendously. And our understanding is that there is already a deployment that is very beneficial. On trading— so, on the capital market activity, we think that is more of an emerging area. And it is not clear to us that it's very widely used at the moment, say, for high frequency trading. So what we have heard is that, you know, to venture into new potential trading activities, generative AI can be very good because, you know, it can generate, say, new trading strategies very quickly, right? So it understands— so, like, the trading strategies that are out there, or the potential trading strategies that are out there, and it can give you trading ideas, and then you can work
with those ideas, right? It can literally generate code or— but— you know, to what extent that is then actually being used, we don't fully, fully know. So— so what we're hearing from market intelligence is that it's helpful, but how it's being used is still somewhat work in progress. So, you know, when you think about the financial firm— so when you think about a risk manager in a financial firm, you would certainly have some sort of concerns about accountability and decision- making accuracy, right? Because in traditional automated trading, you can go through the code, right? And you understand how the code works. And in a generative AI, right, I mean, it's not— you know, you can't trace back the code as to what
decisions are being taken. Now, if the code— if the model generates code, then that you can check. So it is— it is a bit of a complex undertaking. So this lack of explainability, the magnitude of unpredictability and this lack of— of accountability and decision making are certainly issues. Let me— let me flag two more things. So you know, there is some literature that— that is documenting some bias that's not necessarily related to financial activity. But we don't know whether— so, like, the time series properties of what is coming out of the model has good econometric properties from a trading perspective, right? So when you think about trading strategies or capital market, market making, right, you really want to make
sure that, you know, your— your strategy is resilient relative to extreme events in the future, relative to structural changes. And we don't fully understand to what extent— so, like, the length of the data to which the model is calibrated, the kind of data, you know, to what extent that is so, like— a good predictor for— for future performance. And, you know another— another issue for financial firms that we've heard is that they— you know, the technology firms ultimately develop and operate the large models. And the financial firms are users of the models. But it's— it's— it's not— you know, and then they can possibly use variants of the model or calibrate the model to specific data. But there's a certain amount of reliance on these
outside parties that are developing models that are not specifically generated for the financial industry. So the reliance on third-party service providers that may not have the same challenges in mind when generating the models as a financial firm or trading firm would have— you know, that generates a whole new set of issues in terms of third-party dependency that the financial industry is trying to tackle. So basically, what you're saying is there is potential here for better information processing that is going to contribute to market efficiency, and this is something that we all want. On the negative side, on the risk side, I hear you saying mostly that it's hard to explain what's going on. It's kind of a black
box. It is delegated to other parties, and we don't know exactly what they're going to do with it. So it's mostly, I would say, kind of the risk of the unknown that— things that we can't fully think about right now might happen because of deploying these new machines and potentially new agents in the trading process. Exactly. So it is a bit of a black box. So, you know, we may start to use it, but we may not fully understand where results are coming from and how they're going to behave in the future. So the risk management around using generative AI is certainly a key priority. I would add another aspect, which is the malicious use of— of AI. You know, it is not easily
verifiable for use of AI, what kind of information has been fed into the model, and what is the relative importance of— so, like, good data and malicious data. So it is, in principle, possible that the models were calibrated to malicious data where, you know, some actors may want to manipulate the outcome of— of the— of the model. And, you know, it's difficult to sort of assess to what extent your model is robust to such forms of manipulation. I think that is another area of concern, and I think the financial industry is working on that, but it's certainly not an easy challenge to overcome due to the complexity and the magnitude of the models. You know, another issue, of course, is cyber vulnerability, right? So we all
know that operational risk through cyber attacks is a first order concern for the financial sector. And, you know, in fact, regulators are asking financial services firms to hold capital and liquidity against cyber incidents. And you know, IT departments in financial sector firms put a huge amount of resources into protecting firms against cyber attacks. Now, I already gave an example where generative AI is used to detect, say, malicious or criminal activity in, say, financial flows. The challenge is that, of course, generative AI can also be used for criminal purposes, right? So you may use it for better defense, but you can also— it can also be used for more malicious and more effective attacks. So you know, we worry
that cyber incidents may become more effective. And let me give you an example of that. You and I and the listeners are probably receiving all kinds of emails that are trying to phish our— our confidential information. Are trying to gain access to our computer systems. And you know, in the past, oftentimes you can sort of fairly easily detect these phishing attacks, say through emails or social media. There are oftentimes spelling mistakes or things don't look quite right. But generative AI could be very good at actually fixing those issues, and so make those phishing emails or text messages, or in whatever other way you get information, you know, seem— seem more real. And, you know, in cyber— in cyber attacks, it's typically the
interface between technology and the human element that is the vulnerability. The cyber criminals tend to exploit the weak— the human weaknesses, in order to gain access to the technology. And so that could be much more effective. So on the one hand, as a sort of, you know, risk manager or IT department in the financial firm, you know, you have better tools to protect yourself. But probably the attacks are also more effective. So, you know that's another— another first order concern that we hear from the industry. Yeah. So when you're thinking about regulators, they are in a pretty tough position right now, because, on the one hand, opportunity is coming from AI that we all recognize. On the
other hand, if you think about the risks, the risks are pretty big, but they're not easy to define. They're not easy to quantify. A lot of it is just that, you know, it's hard to explain. It's a black box. We kind of don't know exactly what to— what to expect. So what do you think regulators should do in light of all this? Yeah, so I think the first order work that regulators have to do is just to understand. You know, do outreach to firms. You know, meet with participants, to understand what is happening. And our chapter is trying to do some of that. We did a lot of market outreach just to understand where financial sector firms are using generative AI, what they are
exploring, and what they see as risks and challenges, right? And you know, in many ways, we're at the junction time where— so, like, the perspective of the policy maker and the perspective of the financial firms is not that different, right? In— traditionally, in regulation, you often have sort of a big conflict in between the policy objective and the individual firm perspective, because the policy objectives are not necessarily closely aligned with the individual firm perspective. But in this area, I think there's actually a fairly close alignment so that conversations are fairly straightforward. Now, having said that, a lot of it is work in progress, right? I mean, the technology, we are getting updates that are first order
improvements. And, you know, the technology is evolving very quickly, and the applications are evolving very quickly. So it is— it is still working progress. So, you know, for example, in the US, the Securities and Exchange Commission proposed a rule in July 2023, last year, on the use of predictive analytics by broker dealers and investment advisors, which would require regulated entities to take steps to address conflicts of interest in the use of predictive data. And, you know, this is still in the— in the rule making progress. But that would be one example where there's a— there's sort of like a regulatory step by a security market regulator that— that has been undertaken. You know, we're working closely with the
Financial Stability Board and the International Organization of Security Commissions, as well as the Bank for International Settlement, in order to, you know, continue to— to assess, you know, the developments. So is there room for more coordination across regulators, across different countries, given that I think the nature of technology is that the cross border spillovers are very significant? So, is that something that you're trying to do here? Yeah. So I would say that for financial regulators, there's a very well-developed mode of cooperation across countries and across different types of regulatory bodies. So for
example, the Basel Committee, which works on regulating banks and the security market regulators— you know, which is this IASCO, they are both members of the Financial Stability Board. So the Financial Stability Board is bringing together all the different regulatory bodies, including also insurance regulators, for example, in order to have sort of like an umbrella of policy framework. So— so that is very well developed I think in this area of— of generative AI and large language models, there are other regulatory issues that don't fall within the financial sector powers and mandates, right? So, for example, the use of data, right? So as a financial sector firm, you may use a model that
has been calibrated to data where there could be, potentially, governance problems around the data in certain jurisdictions, right? So, you know, in some jurisdictions, for example, individuals have a right to withdraw their data from being used in the public domain. Now, what does that mean for the calibration of the model? If there's an individual that was being used— whose data was being used for the calibration that is then withdrawing that data, right? So these are sort of complicated model that are falling to some degree outside of the policy framework and the powers of the financial regulators, but that are highly relevant for the— for the financial regulatory sphere. So it's the interdependence between data
models and the technological infrastructure that is— that is at issue here. So— so I think data, intellectual property, you know, those are— those are some of the issues. So should we worry that the next financial crisis is going to be a result of AI, or am I going too far with that? It's— it's a little bit too early to say. I— I do think we have made first order progress in financial regulation, you know, particularly in the regulation of banks and insurance companies. But also, you know, important security market participants such as broker dealers and— and— and funds, open end funds, closed end funds. So I do think we're in a much better place than we
were ten or 15 years ago. And you know, for example, with respect to this operational risk, there has been a lot of progress. You know, that is true for banks, for funds, but also for infrastructures. I think the challenge is really around those issues that are not readily captured in the existing regulatory approaches. And I think regulators themselves are working on upscaling their own technology in order to meet these technological challenges. So we hear quite a bit about, you know, regulatory upskilling. So for example, SupTech and Reg- Tech. So, where supervisors and regulators are using, themselves, technologies to detect risks better. So, you know, on the one hand, there could be new risks. On the other hand, policy makers
could be more effective. But let me— let me be a little bit more specific about, you know, some regulatory initiatives that we have seen already. So for example, in Hong Kong, there is an AI governance framework that the Hong Kong monetary authority has put out, which is really about the risk management and the reliability of AI systems used in the banking sector. That's one concrete example. The second one in the UK, the Financial Conduct Authority has implemented requirements for the explainability of AI-driven decision making. So for example, in credit scoring. So, it's fine to use those models, but you need to have sort of a rationale or an explainability around that. And that, in turn, helps to mitigate the risks associated
with the opaqueness of the models. In Singapore, there are principles called the Fairness, Ethics, Accountability and Transparency principles for AI in finance. And that really helps the financial institutions in Singapore to have a responsible usage of AI systems. And that is particularly focused also on bias and discrimination. And I would finally mention, in the European Union, there's the General Data Protection Regulation, GDPR, and that has been adapted to the use of AI practices in financial services, particularly for focusing on the on data protection and privacy practices. So this is some of the examples I alluded to earlier, aimed at reducing the risk of data breaches and the misuse of personal information.
Okay. All very fascinating. Certainly a lot more to hear about and learn about from this AI revolution in financial markets and all the challenges it poses when thinking about risks and regulation. Thank you very much, Tobias, for all the great work that you're doing at the IMF and for joining us for this podcast. Thanks so much, Itay. Very good to see you. And warm regards from Washington to Pennsylvania. Thank you. And thank you, everyone, for listening.





