00:00:05
Itay Goldstein: Hello. This is <i>The
Future of Finance</i> miniseries
00:00:11
here at the Wharton School of
the University of Pennsylvania. I am Itay Goldstein. I'm a
Finance Professor and the Chair
00:00:18
of the Finance Department. The
topic that we are going to dive into today is AI in Finance. AI
stands for artificial
00:00:26
intelligence, and AI is promising
to change the way that the financial industry works in many
different ways. There are many
00:00:35
opportunities, many risks, and
there are regulatory concerns, and we're going to talk about
all this with Tobias Adrian.
00:00:44
Tobias is the financial
counselor and the Director of the Monetary and Capital Markets
Department at the International
00:00:51
Monetary Fund, IMF.
Hello,Tobias. Hi, Itay. - So it's great to have you, and I
look forward to hearing your
00:01:02
insights. So your division at
the IMF has written a report about the use of AI in finance.
Can you tell us a little more
00:01:10
about what is at stake? What are
the opportunities? What are the risks? What issues do you see?
00:01:17
Yeah, thanks so much for having
me with you today, Itay. So we are— we have a report, which is
called the Global Financial
00:01:26
Stability Report, that has a
special focus on artificial intelligence, or AI in finance.
So there are different modes of
00:01:44
AI, and some AI techniques have
been deployed in finance for many years. In fact, many
decades. And have really
00:01:53
reshaped the way that finance
worked already. And here I'm particularly thinking about
algorithmic trading, which is
00:02:03
primarily based on machine
learning techniques. So when we're looking at liquid capital
markets today, we already see a
00:02:12
tremendous impact of artificial
intelligence on how those markets are working. So you
know, trading activity in— in
00:02:25
places such as equity markets in
in the US and other advanced economies, or Treasury markets,
the most liquid securities in
00:02:37
those markets are already
largely traded in algorithmic fashion based on artificial
intelligence. And, you know,
00:02:48
really trading at very high
frequencies. What is new in recent years is, of course, the
arrival of generative artificial
00:02:59
intelligence. And that is, you
know, quite different from the machine learning that has been
previously deployed. And so the
00:03:14
focus on— of our report is really
on understanding to what extent generative AI, including the
large language models that many
00:03:25
of the listeners today will have
used already— to what extent is this going to impact the
financial industry? And our
00:03:32
particular focus is on trading.
So, on capital markets activity. And so I'm happy to go deeper
into that. There are also other
00:03:41
areas of finance that are
already being impacted. And also happy to talk about that
a little bit.
00:03:48
Yeah. So you know, one of the
things that we think about when we think about AI is the agency.
That now the AI will basically
00:03:57
be a decision maker and will
start doing things on their own. I wonder if some of this is also
reflected in the way you're
00:04:05
thinking about it, and in your
report. Yeah, absolutely. So it is
certainly true that in
00:04:11
algorithmic trading, you do see
a lot of automated decision making at very high frequencies.
So there could be, you know, a
00:04:20
very large number— you know,
thousands of millions of decisions that are— that are fully
automated. And, you know, to what
00:04:29
extent that will be changed or
influenced by generative AI is still work in progress. So let
me dig a little bit deeper about,
00:04:42
you know, what generative AI is,
and how it is being used already, and how it may be used
going forward. So, generative AI
00:04:52
really is using very large
models that are calibrated to very, very large data in order
to elicit answers that are
00:05:07
somewhat similar to the way that
human intelligence works, which is why the term "artificial
intelligence" is being used. And
00:05:19
in the large language models,
the interaction with those models is via natural language.
So the logic is quite different
00:05:30
from programming traditionally,
where traditionally, you write code, then the computer is doing
something, and it's generating
00:05:39
output. Here you have— so, like a
model that has been calibrated, and then you're working with
using the information from that
00:05:50
model in order to generate
output. And I think the challenge here is twofold. So
number one, we don't fully
00:05:59
understand how these models
really work. They are extremely powerful, but they're a little
bit of a blank box at the
00:06:10
moment. So a lot of research
today is actually discovering what kind of knowledge is
embedded in the generative AI
00:06:20
models. And so we are still
learning about the technology that is very powerful, but it's
still very recent. As a result,
00:06:32
the financial sector is still in
the process of understanding how those models can be used across
the financial sector and for
00:06:41
trading activity in particular.
And so one policy concern is the lack of explainability and the
unpredictability, to some degree,
00:06:51
right? Because we know they are
so, like— you know, highly complex information is used in a
way that may be working similar
00:07:02
to the human brain. But how it
performs in terms of trading, for example, when new situations
are feeding into— into financial
00:07:15
market activity, we don't fully
understand. And so there's a certain amount of
unpredictability and lack of
00:07:23
explainability. And, you know,
what the impact is, is— is work in progress. So our sense is
that there are tremendous
00:07:35
opportunities. So we already see
some areas where we think the new technology has a first order
impact. So for example, when
00:07:46
complex reports are being
published. Say, an SEC filing of a publicly traded company,
right? These can be very thick,
00:07:54
you know, 100 pages long reports.
And a generative AI model, you know, can very quickly extract
information from the report.
00:08:05
Another example is statements by
monetary policy makers like the Federal Open Market Committee of
the Federal Reserve. When they
00:08:14
publish their policy decision
that comes alongside the statement, and the statement now
can be analyzed very quickly by
00:08:24
generative AI. And so the
informational efficiency of markets can be increased, in
principle. And I think we see
00:08:31
some evidence that that may be
the case. So it could be very good for market efficiency. But
then there are other areas where
00:08:41
we sort of don't know exactly
how much efficiency would be improved and what potential
pitfalls could be. So there are
00:08:51
efficiency gains, but there are
also new risks that we have to take into account.
00:08:56
So how widely used is it at this
point? So our understanding is that— so,
like, every financial firm is
00:09:04
exploring it. And there are many
startups that are being built around the technology. And in
some areas, we do see traction.
00:09:14
So for example, what we are
hearing is that to do compliance checks, right, or credit risk
analysis— so, you know, analytics,
00:09:26
where, you know, complex data,
some of it from the public domain, like from the internet,
some of it from reports such as
00:09:37
credit reports, have to be
combined, and activity has to be detected. So for example, for
payments activity, right? You
00:09:48
know, financial institutions put
a lot of effort into understanding whether a certain
payment may be legitimate or
00:09:55
illegitimate. We understand that
in those areas, newer AI models can be very effective. And
detection rate for fraudulent
00:10:07
activity can— can increase
tremendously. And our understanding is that there is
already a deployment that is
00:10:17
very beneficial. On trading— so, on
the capital market activity, we think that is more of an
emerging area. And it is not
00:10:29
clear to us that it's very
widely used at the moment, say, for high frequency trading. So
what we have heard is that, you
00:10:37
know, to venture into new
potential trading activities, generative AI can be very good
because, you know, it can
00:10:46
generate, say, new trading
strategies very quickly, right? So it understands— so, like, the
trading strategies that are out
00:10:53
there, or the potential trading
strategies that are out there, and it can give you trading
ideas, and then you can work
00:10:59
with those ideas, right? It can
literally generate code or— but— you know, to what extent that is
then actually being used, we
00:11:08
don't fully, fully know. So— so
what we're hearing from market intelligence is that it's
helpful, but how it's being used
00:11:17
is still somewhat work in
progress. So, you know, when you think about the financial firm—
so when you think about a risk
00:11:26
manager in a financial firm,
you would certainly have some sort of concerns about
accountability and decision-
00:11:33
making accuracy, right? Because
in traditional automated trading, you can go through the
code, right? And you understand
00:11:41
how the code works. And in a
generative AI, right, I mean, it's not— you know, you can't
trace back the code as to what
00:11:50
decisions are being taken. Now,
if the code— if the model generates code, then that you
can check. So it is— it is a bit
00:11:59
of a complex undertaking. So
this lack of explainability, the magnitude of unpredictability
and this lack of— of
00:12:11
accountability and decision
making are certainly issues. Let me— let me flag two more things.
So you know, there is some
00:12:24
literature that— that is
documenting some bias that's not necessarily related to financial
activity. But we don't know
00:12:33
whether— so, like, the time series
properties of what is coming out of the model has good econometric
properties from a trading
00:12:42
perspective, right? So when you
think about trading strategies or capital market, market making,
right, you really want to make
00:12:50
sure that, you know, your— your
strategy is resilient relative to extreme events in the future,
relative to structural changes.
00:13:02
And we don't fully understand to
what extent— so, like, the length of the data to which the model
is calibrated, the kind of data,
00:13:12
you know, to what extent that is
so, like— a good predictor for— for future performance. And, you know
another— another issue for
00:13:23
financial firms that we've heard
is that they— you know, the technology firms ultimately
develop and operate the large
00:13:39
models. And the financial firms
are users of the models. But it's— it's— it's not— you know,
and then they can possibly use
00:13:52
variants of the model or
calibrate the model to specific data. But there's a certain
amount of reliance on these
00:14:02
outside parties that are
developing models that are not specifically generated for the
financial industry. So the
00:14:10
reliance on third-party service
providers that may not have the same challenges in mind when
generating the models as a financial
00:14:21
firm or trading firm would have—
you know, that generates a whole new set of issues in terms of
third-party dependency that the
00:14:32
financial industry is trying to
tackle. So basically, what you're saying is
there is potential here for
00:14:39
better information processing
that is going to contribute to market efficiency, and this is
something that we all want. On
00:14:48
the negative side, on the risk
side, I hear you saying mostly that it's hard to explain what's
going on. It's kind of a black
00:14:55
box. It is delegated to other
parties, and we don't know exactly what they're going to do
with it. So it's mostly, I would
00:15:04
say, kind of the risk of the
unknown that— things that we can't fully think about right
now might happen because of
00:15:13
deploying these new machines and
potentially new agents in the trading process. Exactly. So it is a bit of
a black box.
00:15:22
So, you know, we may start to use it, but we may not fully
understand where results are
00:15:31
coming from and how they're
going to behave in the future. So the risk management around
using generative AI is certainly
00:15:41
a key priority. I would add
another aspect, which is the malicious use of— of AI. You
know, it is not easily
00:15:56
verifiable for use of AI, what
kind of information has been fed into the model, and what is the
relative importance of— so, like,
00:16:09
good data and malicious data. So
it is, in principle, possible that the models were calibrated
to malicious data where, you
00:16:25
know, some actors may want to
manipulate the outcome of— of the— of the model. And, you know,
it's difficult to sort of assess
00:16:38
to what extent your model is
robust to such forms of manipulation. I think that is
another area of concern, and I
00:16:48
think the financial industry is
working on that, but it's certainly not an easy challenge
to overcome due to the
00:16:55
complexity and the magnitude of
the models. You know, another issue, of course, is cyber
vulnerability, right? So we all
00:17:07
know that operational risk
through cyber attacks is a first order concern for the financial
sector. And, you know, in fact,
00:17:20
regulators are asking financial
services firms to hold capital and liquidity against cyber
incidents. And you know, IT
00:17:31
departments in financial sector
firms put a huge amount of resources into protecting firms
against cyber attacks. Now, I
00:17:44
already gave an example where
generative AI is used to detect, say, malicious or criminal
activity in, say, financial
00:17:51
flows. The challenge is that, of
course, generative AI can also be used for criminal purposes,
right? So you may use it for
00:18:03
better defense, but you can also—
it can also be used for more malicious and more effective
attacks. So you know, we worry
00:18:16
that cyber incidents may become
more effective. And let me give you an example of that. You and
I and the listeners are probably
00:18:30
receiving all kinds of emails
that are trying to phish our— our confidential information. Are
trying to gain access to our
00:18:40
computer systems. And you know,
in the past, oftentimes you can sort of fairly easily detect
these phishing attacks, say
00:18:50
through emails or social media.
There are oftentimes spelling mistakes or things don't look
quite right. But generative AI
00:18:59
could be very good at actually
fixing those issues, and so make those phishing emails or text
messages, or in whatever other
00:19:12
way you get information, you
know, seem— seem more real. And, you know, in cyber— in cyber
attacks, it's typically the
00:19:23
interface between technology and
the human element that is the vulnerability. The cyber
criminals tend to exploit the
00:19:31
weak— the human weaknesses, in
order to gain access to the technology. And so that could be
much more effective. So on the
00:19:38
one hand, as a sort of, you
know, risk manager or IT department in the financial
firm, you know, you have better
00:19:46
tools to protect yourself. But
probably the attacks are also more effective. So, you know
that's another— another first
00:19:52
order concern that we hear from
the industry. Yeah. So when you're thinking about
regulators, they are in a pretty
00:20:00
tough position right now,
because, on the one hand, opportunity is coming from AI
that we all recognize. On the
00:20:08
other hand, if you think about
the risks, the risks are pretty big, but they're not easy to
define. They're not easy to
00:20:17
quantify. A lot of it is just
that, you know, it's hard to explain. It's a black box. We
kind of don't know exactly what
00:20:26
to— what to expect. So what do
you think regulators should do in light of all this?
00:20:31
Yeah, so I think the first order
work that regulators have to do is just to understand. You know,
do outreach to firms. You know,
00:20:43
meet with participants, to
understand what is happening. And our chapter is trying to do
some of that. We did a lot of
00:20:52
market outreach just to
understand where financial sector firms are using
generative AI, what they are
00:21:02
exploring, and what they see as
risks and challenges, right? And you know, in many ways, we're at
the junction time where— so, like,
00:21:15
the perspective of the policy
maker and the perspective of the financial firms is not that
different, right? In—
00:21:21
traditionally, in regulation,
you often have sort of a big conflict in between the policy
objective and the individual
00:21:29
firm perspective, because the
policy objectives are not necessarily closely aligned with
the individual firm perspective.
00:21:37
But in this area, I think
there's actually a fairly close alignment so that conversations
are fairly straightforward. Now,
00:21:44
having said that, a lot of it is
work in progress, right? I mean, the technology, we are getting
updates that are first order
00:21:52
improvements. And, you know, the
technology is evolving very quickly, and the applications
are evolving very quickly. So it
00:22:01
is— it is still working
progress. So, you know, for example, in the US, the
Securities and Exchange
00:22:08
Commission proposed a rule in
July 2023, last year, on the use of predictive analytics by
broker dealers and investment
00:22:17
advisors, which would require
regulated entities to take steps to address conflicts of interest
in the use of predictive data.
00:22:27
And, you know, this is still in
the— in the rule making progress. But that would be one example
where there's a— there's
00:22:33
sort of like a regulatory step
by a security market regulator that— that has been undertaken. You know,
we're working closely with the
00:22:42
Financial Stability Board and
the International Organization of Security Commissions, as well
as the Bank for International
00:22:48
Settlement, in order to, you
know, continue to— to assess, you know, the developments.
00:22:58
So is there room for more
coordination across regulators, across different countries,
given that I think the nature
00:23:08
of technology is that the cross
border spillovers are very significant? So, is that
00:23:14
something that you're trying to
do here? Yeah. So I would say that for
financial regulators, there's a
00:23:22
very well-developed mode of
cooperation across countries and across different types of
regulatory bodies. So for
00:23:30
example, the Basel Committee,
which works on regulating banks and the security market
regulators— you know, which is
00:23:40
this IASCO, they are both
members of the Financial Stability Board. So the
Financial Stability Board is
00:23:46
bringing together all the
different regulatory bodies, including also insurance
regulators, for example, in
00:23:52
order to have sort of like an
umbrella of policy framework. So— so that is very well developed
I think in this area of— of
00:24:01
generative AI and large language
models, there are other regulatory issues that don't
fall within the financial sector
00:24:11
powers and mandates, right? So,
for example, the use of data, right? So as a financial sector
firm, you may use a model that
00:24:22
has been calibrated to data
where there could be, potentially, governance problems around the
data in certain jurisdictions,
00:24:31
right? So, you know, in some
jurisdictions, for example, individuals have a right to
withdraw their data from being
00:24:40
used in the public domain. Now,
what does that mean for the calibration of the model? If
there's an individual that was
00:24:48
being used— whose data was being
used for the calibration that is then withdrawing that data,
right? So these are sort of
00:24:54
complicated model that are
falling to some degree outside of the policy framework and the
powers of the financial
00:25:01
regulators, but that are highly
relevant for the— for the financial regulatory sphere. So it's the
interdependence between data
00:25:11
models and the technological
infrastructure that is— that is at issue here. So— so I think
data, intellectual property, you
00:25:21
know, those are— those are some
of the issues. So should we worry that the next
financial crisis is going to be
00:25:27
a result of AI, or am I going
too far with that? It's— it's a little bit too
early to say.
00:25:36
I— I do think we
have made first order progress in financial
regulation, you know, particularly in the
00:25:48
regulation of banks and
insurance companies. But also, you know, important security
market participants such as
00:25:57
broker dealers and— and— and
funds, open end funds, closed end funds. So I do think we're
in a much better place than we
00:26:05
were ten or 15 years ago. And you
know, for example, with respect to this operational risk, there
has been a lot of progress. You
00:26:16
know, that is true for banks,
for funds, but also for infrastructures. I think the
challenge is really around those
00:26:28
issues that are not readily
captured in the existing regulatory approaches. And I
think regulators themselves are
00:26:43
working on upscaling their own
technology in order to meet these technological challenges.
So we hear quite a bit about, you
00:26:56
know, regulatory upskilling. So
for example, SupTech and Reg- Tech. So, where supervisors and
regulators are using, themselves,
00:27:03
technologies to detect risks
better. So, you know, on the one hand, there could be new risks.
On the other hand, policy makers
00:27:12
could be more effective. But let
me— let me be a little bit more specific about, you know, some
regulatory initiatives that we
00:27:22
have seen already. So for
example, in Hong Kong, there is an AI governance framework that
the Hong Kong monetary authority
00:27:31
has put out, which is really
about the risk management and the reliability of AI systems
used in the banking sector.
00:27:41
That's one concrete example. The
second one in the UK, the Financial Conduct Authority has
implemented requirements for the
00:27:51
explainability of AI-driven
decision making. So for example, in credit scoring. So, it's fine
to use those models, but you
00:28:00
need to have sort of a rationale
or an explainability around that. And that, in turn, helps to
mitigate the risks associated
00:28:11
with the opaqueness of the
models. In Singapore, there are principles called the Fairness,
Ethics, Accountability and
00:28:22
Transparency principles for AI
in finance. And that really helps the financial institutions
in Singapore to have a
00:28:31
responsible usage of AI systems.
And that is particularly focused also on bias and discrimination.
And I would finally mention, in
00:28:43
the European Union, there's the
General Data Protection Regulation, GDPR, and that has
been adapted to the use of AI
00:28:57
practices in financial services,
particularly for focusing on the on data protection and privacy
practices. So this is some of
00:29:06
the examples I alluded to
earlier, aimed at reducing the risk of data breaches and the
misuse of personal information.
00:29:16
Okay. All very fascinating.
Certainly a lot more to hear about and learn about from this
AI revolution in financial
00:29:27
markets and all the challenges
it poses when thinking about risks and regulation. Thank you
very much, Tobias, for all the
00:29:36
great work that you're doing at
the IMF and for joining us for this podcast.
00:29:42
Thanks so much, Itay.
Very good to see you. And warm regards from
Washington to Pennsylvania.
00:29:51
Thank you. And thank you, everyone,
for listening.