00:00:05
Itay Goldstein: Hello. This is <i>The
Future of Finance</i> miniseries
00:00:11
here at the Wharton School of
the University of Pennsylvania.
00:00:14
I am Itay Goldstein. I'm a
Finance Professor and the Chair
00:00:18
of the Finance Department. The
topic that we are going to dive
00:00:21
into today is AI in Finance. AI
stands for artificial
00:00:26
intelligence, and AI is promising
to change the way that the
00:00:31
financial industry works in many
different ways. There are many
00:00:35
opportunities, many risks, and
there are regulatory concerns,
00:00:40
and we're going to talk about
all this with Tobias Adrian.
00:00:44
Tobias is the financial
counselor and the Director of
00:00:48
the Monetary and Capital Markets
Department at the International
00:00:51
Monetary Fund, IMF.
Hello,Tobias.
00:00:56
Hi, Itay.
00:00:58
- So it's great to have you, and I
look forward to hearing your
00:01:02
insights. So your division at
the IMF has written a report
00:01:06
about the use of AI in finance.
Can you tell us a little more
00:01:10
about what is at stake? What are
the opportunities? What are the
00:01:14
risks? What issues do you see?
00:01:17
Yeah, thanks so much for having
me with you today, Itay. So we
00:01:22
are— we have a report, which is
called the Global Financial
00:01:26
Stability Report, that has a
special focus on artificial
00:01:33
intelligence, or AI in finance.
So there are different modes of
00:01:44
AI, and some AI techniques have
been deployed in finance for
00:01:49
many years. In fact, many
decades. And have really
00:01:53
reshaped the way that finance
worked already. And here I'm
00:01:58
particularly thinking about
algorithmic trading, which is
00:02:03
primarily based on machine
learning techniques. So when
00:02:08
we're looking at liquid capital
markets today, we already see a
00:02:12
tremendous impact of artificial
intelligence on how those
00:02:17
markets are working. So you
know, trading activity in— in
00:02:25
places such as equity markets in
in the US and other advanced
00:02:31
economies, or Treasury markets,
the most liquid securities in
00:02:37
those markets are already
largely traded in algorithmic
00:02:41
fashion based on artificial
intelligence. And, you know,
00:02:48
really trading at very high
frequencies. What is new in
00:02:54
recent years is, of course, the
arrival of generative artificial
00:02:59
intelligence. And that is, you
know, quite different from the
00:03:08
machine learning that has been
previously deployed. And so the
00:03:14
focus on— of our report is really
on understanding to what extent
00:03:19
generative AI, including the
large language models that many
00:03:25
of the listeners today will have
used already— to what extent is
00:03:29
this going to impact the
financial industry? And our
00:03:32
particular focus is on trading.
So, on capital markets activity.
00:03:37
And so I'm happy to go deeper
into that. There are also other
00:03:41
areas of finance that are
already being impacted. And also
00:03:45
happy to talk about that
a little bit.
00:03:48
Yeah. So you know, one of the
things that we think about when
00:03:51
we think about AI is the agency.
That now the AI will basically
00:03:57
be a decision maker and will
start doing things on their own.
00:04:01
I wonder if some of this is also
reflected in the way you're
00:04:05
thinking about it, and in your
report.
00:04:08
Yeah, absolutely. So it is
certainly true that in
00:04:11
algorithmic trading, you do see
a lot of automated decision
00:04:16
making at very high frequencies.
So there could be, you know, a
00:04:20
very large number— you know,
thousands of millions of decisions
00:04:24
that are— that are fully
automated. And, you know, to what
00:04:29
extent that will be changed or
influenced by generative AI is
00:04:36
still work in progress. So let
me dig a little bit deeper about,
00:04:42
you know, what generative AI is,
and how it is being used
00:04:47
already, and how it may be used
going forward. So, generative AI
00:04:52
really is using very large
models that are calibrated to
00:04:58
very, very large data in order
to elicit answers that are
00:05:07
somewhat similar to the way that
human intelligence works, which
00:05:13
is why the term "artificial
intelligence" is being used. And
00:05:19
in the large language models,
the interaction with those
00:05:22
models is via natural language.
So the logic is quite different
00:05:30
from programming traditionally,
where traditionally, you write
00:05:35
code, then the computer is doing
something, and it's generating
00:05:39
output. Here you have— so, like a
model that has been calibrated,
00:05:45
and then you're working with
using the information from that
00:05:50
model in order to generate
output. And I think the
00:05:55
challenge here is twofold. So
number one, we don't fully
00:05:59
understand how these models
really work. They are extremely
00:06:06
powerful, but they're a little
bit of a blank box at the
00:06:10
moment. So a lot of research
today is actually discovering
00:06:15
what kind of knowledge is
embedded in the generative AI
00:06:20
models. And so we are still
learning about the technology
00:06:27
that is very powerful, but it's
still very recent. As a result,
00:06:32
the financial sector is still in
the process of understanding how
00:06:38
those models can be used across
the financial sector and for
00:06:41
trading activity in particular.
And so one policy concern is the
00:06:47
lack of explainability and the
unpredictability, to some degree,
00:06:51
right? Because we know they are
so, like— you know, highly
00:06:56
complex information is used in a
way that may be working similar
00:07:02
to the human brain. But how it
performs in terms of trading,
00:07:07
for example, when new situations
are feeding into— into financial
00:07:15
market activity, we don't fully
understand. And so there's a
00:07:20
certain amount of
unpredictability and lack of
00:07:23
explainability. And, you know,
what the impact is, is— is work
00:07:29
in progress. So our sense is
that there are tremendous
00:07:35
opportunities. So we already see
some areas where we think the
00:07:42
new technology has a first order
impact. So for example, when
00:07:46
complex reports are being
published. Say, an SEC filing of
00:07:51
a publicly traded company,
right? These can be very thick,
00:07:54
you know, 100 pages long reports.
And a generative AI model, you
00:08:00
know, can very quickly extract
information from the report.
00:08:05
Another example is statements by
monetary policy makers like the
00:08:10
Federal Open Market Committee of
the Federal Reserve. When they
00:08:14
publish their policy decision
that comes alongside the
00:08:19
statement, and the statement now
can be analyzed very quickly by
00:08:24
generative AI. And so the
informational efficiency of
00:08:27
markets can be increased, in
principle. And I think we see
00:08:31
some evidence that that may be
the case. So it could be very
00:08:36
good for market efficiency. But
then there are other areas where
00:08:41
we sort of don't know exactly
how much efficiency would be
00:08:47
improved and what potential
pitfalls could be. So there are
00:08:51
efficiency gains, but there are
also new risks that we have to
00:08:55
take into account.
00:08:56
So how widely used is it at this
point?
00:09:00
So our understanding is that— so,
like, every financial firm is
00:09:04
exploring it. And there are many
startups that are being built
00:09:08
around the technology. And in
some areas, we do see traction.
00:09:14
So for example, what we are
hearing is that to do compliance
00:09:20
checks, right, or credit risk
analysis— so, you know, analytics,
00:09:26
where, you know, complex data,
some of it from the public
00:09:33
domain, like from the internet,
some of it from reports such as
00:09:37
credit reports, have to be
combined, and activity has to be
00:09:45
detected. So for example, for
payments activity, right? You
00:09:48
know, financial institutions put
a lot of effort into
00:09:51
understanding whether a certain
payment may be legitimate or
00:09:55
illegitimate. We understand that
in those areas, newer AI models
00:10:03
can be very effective. And
detection rate for fraudulent
00:10:07
activity can— can increase
tremendously. And our
00:10:13
understanding is that there is
already a deployment that is
00:10:17
very beneficial. On trading— so, on
the capital market activity, we
00:10:23
think that is more of an
emerging area. And it is not
00:10:29
clear to us that it's very
widely used at the moment, say,
00:10:33
for high frequency trading. So
what we have heard is that, you
00:10:37
know, to venture into new
potential trading activities,
00:10:41
generative AI can be very good
because, you know, it can
00:10:46
generate, say, new trading
strategies very quickly, right?
00:10:49
So it understands— so, like, the
trading strategies that are out
00:10:53
there, or the potential trading
strategies that are out there,
00:10:56
and it can give you trading
ideas, and then you can work
00:10:59
with those ideas, right? It can
literally generate code or— but—
00:11:05
you know, to what extent that is
then actually being used, we
00:11:08
don't fully, fully know. So— so
what we're hearing from market
00:11:13
intelligence is that it's
helpful, but how it's being used
00:11:17
is still somewhat work in
progress. So, you know, when you
00:11:24
think about the financial firm—
so when you think about a risk
00:11:26
manager in a financial firm,
you would certainly have some
00:11:30
sort of concerns about
accountability and decision-
00:11:33
making accuracy, right? Because
in traditional automated
00:11:38
trading, you can go through the
code, right? And you understand
00:11:41
how the code works. And in a
generative AI, right, I mean,
00:11:45
it's not— you know, you can't
trace back the code as to what
00:11:50
decisions are being taken. Now,
if the code— if the model
00:11:54
generates code, then that you
can check. So it is— it is a bit
00:11:59
of a complex undertaking. So
this lack of explainability, the
00:12:05
magnitude of unpredictability
and this lack of— of
00:12:11
accountability and decision
making are certainly issues. Let
00:12:16
me— let me flag two more things.
So you know, there is some
00:12:24
literature that— that is
documenting some bias that's not
00:12:29
necessarily related to financial
activity. But we don't know
00:12:33
whether— so, like, the time series
properties of what is coming out
00:12:37
of the model has good econometric
properties from a trading
00:12:42
perspective, right? So when you
think about trading strategies
00:12:46
or capital market, market making,
right, you really want to make
00:12:50
sure that, you know, your— your
strategy is resilient relative
00:12:57
to extreme events in the future,
relative to structural changes.
00:13:02
And we don't fully understand to
what extent— so, like, the length
00:13:08
of the data to which the model
is calibrated, the kind of data,
00:13:12
you know, to what extent that is
so, like— a good predictor for— for
00:13:16
future performance. And, you know
another— another issue for
00:13:23
financial firms that we've heard
is that they— you know, the
00:13:31
technology firms ultimately
develop and operate the large
00:13:39
models. And the financial firms
are users of the models. But
00:13:47
it's— it's— it's not— you know,
and then they can possibly use
00:13:52
variants of the model or
calibrate the model to specific
00:13:55
data. But there's a certain
amount of reliance on these
00:14:02
outside parties that are
developing models that are not
00:14:05
specifically generated for the
financial industry. So the
00:14:10
reliance on third-party service
providers that may not have the
00:14:15
same challenges in mind when
generating the models as a financial
00:14:21
firm or trading firm would have—
you know, that generates a whole
00:14:25
new set of issues in terms of
third-party dependency that the
00:14:32
financial industry is trying to
tackle.
00:14:36
So basically, what you're saying is
there is potential here for
00:14:39
better information processing
that is going to contribute to
00:14:43
market efficiency, and this is
something that we all want. On
00:14:48
the negative side, on the risk
side, I hear you saying mostly
00:14:51
that it's hard to explain what's
going on. It's kind of a black
00:14:55
box. It is delegated to other
parties, and we don't know
00:15:00
exactly what they're going to do
with it. So it's mostly, I would
00:15:04
say, kind of the risk of the
unknown that— things that we
00:15:09
can't fully think about right
now might happen because of
00:15:13
deploying these new machines and
potentially new agents in the
00:15:17
trading process.
00:15:18
Exactly. So it is a bit of
a black box.
00:15:22
So, you know, we may start to
00:15:26
use it, but we may not fully
understand where results are
00:15:31
coming from and how they're
going to behave in the future.
00:15:34
So the risk management around
using generative AI is certainly
00:15:41
a key priority. I would add
another aspect, which is the
00:15:46
malicious use of— of AI. You
know, it is not easily
00:15:56
verifiable for use of AI, what
kind of information has been fed
00:16:03
into the model, and what is the
relative importance of— so, like,
00:16:09
good data and malicious data. So
it is, in principle, possible
00:16:16
that the models were calibrated
to malicious data where, you
00:16:25
know, some actors may want to
manipulate the outcome of— of the—
00:16:31
of the model. And, you know,
it's difficult to sort of assess
00:16:38
to what extent your model is
robust to such forms of
00:16:44
manipulation. I think that is
another area of concern, and I
00:16:48
think the financial industry is
working on that, but it's
00:16:52
certainly not an easy challenge
to overcome due to the
00:16:55
complexity and the magnitude of
the models.
00:17:00
You know, another
00:17:03
issue, of course, is cyber
vulnerability, right? So we all
00:17:07
know that operational risk
through cyber attacks is a first
00:17:16
order concern for the financial
sector. And, you know, in fact,
00:17:20
regulators are asking financial
services firms to hold capital
00:17:27
and liquidity against cyber
incidents. And you know, IT
00:17:31
departments in financial sector
firms put a huge amount of
00:17:36
resources into protecting firms
against cyber attacks. Now, I
00:17:44
already gave an example where
generative AI is used to detect,
00:17:48
say, malicious or criminal
activity in, say, financial
00:17:51
flows. The challenge is that, of
course, generative AI can also
00:17:56
be used for criminal purposes,
right? So you may use it for
00:18:03
better defense, but you can also—
it can also be used for more
00:18:08
malicious and more effective
attacks. So you know, we worry
00:18:16
that cyber incidents may become
more effective. And let me give
00:18:24
you an example of that. You and
I and the listeners are probably
00:18:30
receiving all kinds of emails
that are trying to phish our— our
00:18:36
confidential information. Are
trying to gain access to our
00:18:40
computer systems. And you know,
in the past, oftentimes you can
00:18:45
sort of fairly easily detect
these phishing attacks, say
00:18:50
through emails or social media.
There are oftentimes spelling
00:18:54
mistakes or things don't look
quite right. But generative AI
00:18:59
could be very good at actually
fixing those issues, and so make
00:19:05
those phishing emails or text
messages, or in whatever other
00:19:12
way you get information, you
know, seem— seem more real. And,
00:19:20
you know, in cyber— in cyber
attacks, it's typically the
00:19:23
interface between technology and
the human element that is the
00:19:27
vulnerability. The cyber
criminals tend to exploit the
00:19:31
weak— the human weaknesses, in
order to gain access to the
00:19:34
technology. And so that could be
much more effective. So on the
00:19:38
one hand, as a sort of, you
know, risk manager or IT
00:19:43
department in the financial
firm, you know, you have better
00:19:46
tools to protect yourself. But
probably the attacks are also
00:19:49
more effective. So, you know
that's another— another first
00:19:52
order concern that we hear from
the industry.
00:19:54
Yeah. So when you're thinking about
regulators, they are in a pretty
00:20:00
tough position right now,
because, on the one hand,
00:20:04
opportunity is coming from AI
that we all recognize. On the
00:20:08
other hand, if you think about
the risks, the risks are pretty
00:20:12
big, but they're not easy to
define. They're not easy to
00:20:17
quantify. A lot of it is just
that, you know, it's hard to
00:20:21
explain. It's a black box. We
kind of don't know exactly what
00:20:26
to— what to expect. So what do
you think regulators should do
00:20:29
in light of all this?
00:20:31
Yeah, so I think the first order
work that regulators have to do
00:20:37
is just to understand. You know,
do outreach to firms. You know,
00:20:43
meet with participants, to
understand what is happening.
00:20:48
And our chapter is trying to do
some of that. We did a lot of
00:20:52
market outreach just to
understand where financial
00:20:58
sector firms are using
generative AI, what they are
00:21:02
exploring, and what they see as
risks and challenges, right? And
00:21:10
you know, in many ways, we're at
the junction time where— so, like,
00:21:15
the perspective of the policy
maker and the perspective of the
00:21:17
financial firms is not that
different, right? In—
00:21:21
traditionally, in regulation,
you often have sort of a big
00:21:24
conflict in between the policy
objective and the individual
00:21:29
firm perspective, because the
policy objectives are not
00:21:32
necessarily closely aligned with
the individual firm perspective.
00:21:37
But in this area, I think
there's actually a fairly close
00:21:40
alignment so that conversations
are fairly straightforward. Now,
00:21:44
having said that, a lot of it is
work in progress, right? I mean,
00:21:47
the technology, we are getting
updates that are first order
00:21:52
improvements. And, you know, the
technology is evolving very
00:21:56
quickly, and the applications
are evolving very quickly. So it
00:22:01
is— it is still working
progress. So, you know, for
00:22:04
example, in the US, the
Securities and Exchange
00:22:08
Commission proposed a rule in
July 2023, last year, on the use
00:22:14
of predictive analytics by
broker dealers and investment
00:22:17
advisors, which would require
regulated entities to take steps
00:22:22
to address conflicts of interest
in the use of predictive data.
00:22:27
And, you know, this is still in
the— in the rule making progress.
00:22:31
But that would be one example
where there's a— there's
00:22:33
sort of like a regulatory step
by a security market regulator that—
00:22:37
that has been undertaken. You know,
we're working closely with the
00:22:42
Financial Stability Board and
the International Organization
00:22:45
of Security Commissions, as well
as the Bank for International
00:22:48
Settlement, in order to, you
know, continue to— to assess, you
00:22:55
know, the developments.
00:22:58
So is there room for more
coordination across regulators,
00:23:04
across different countries,
given that I think the nature
00:23:08
of technology is that the cross
border spillovers are very
00:23:12
significant? So, is that
00:23:14
something that you're trying to
do here?
00:23:16
Yeah. So I would say that for
financial regulators, there's a
00:23:22
very well-developed mode of
cooperation across countries and
00:23:27
across different types of
regulatory bodies. So for
00:23:30
example, the Basel Committee,
which works on regulating banks
00:23:36
and the security market
regulators— you know, which is
00:23:40
this IASCO, they are both
members of the Financial
00:23:44
Stability Board. So the
Financial Stability Board is
00:23:46
bringing together all the
different regulatory bodies,
00:23:49
including also insurance
regulators, for example, in
00:23:52
order to have sort of like an
umbrella of policy framework. So—
00:23:56
so that is very well developed
I think in this area of— of
00:24:01
generative AI and large language
models, there are other
00:24:04
regulatory issues that don't
fall within the financial sector
00:24:11
powers and mandates, right? So,
for example, the use of data,
00:24:16
right? So as a financial sector
firm, you may use a model that
00:24:22
has been calibrated to data
where there could be, potentially,
00:24:28
governance problems around the
data in certain jurisdictions,
00:24:31
right? So, you know, in some
jurisdictions, for example,
00:24:36
individuals have a right to
withdraw their data from being
00:24:40
used in the public domain. Now,
what does that mean for the
00:24:44
calibration of the model? If
there's an individual that was
00:24:48
being used— whose data was being
used for the calibration that is
00:24:51
then withdrawing that data,
right? So these are sort of
00:24:54
complicated model that are
falling to some degree outside
00:24:58
of the policy framework and the
powers of the financial
00:25:01
regulators, but that are highly
relevant for the— for the financial
00:25:07
regulatory sphere. So it's the
interdependence between data
00:25:11
models and the technological
infrastructure that is— that is
00:25:16
at issue here. So— so I think
data, intellectual property, you
00:25:21
know, those are— those are some
of the issues.
00:25:23
So should we worry that the next
financial crisis is going to be
00:25:27
a result of AI, or am I going
too far with that?
00:25:30
It's— it's a little bit too
early to say.
00:25:36
I— I do think we
have made first
00:25:41
order progress in financial
regulation,
00:25:45
you know, particularly in the
00:25:48
regulation of banks and
insurance companies. But also,
00:25:53
you know, important security
market participants such as
00:25:57
broker dealers and— and— and
funds, open end funds, closed
00:26:01
end funds. So I do think we're
in a much better place than we
00:26:05
were ten or 15 years ago. And you
know, for example, with respect
00:26:11
to this operational risk, there
has been a lot of progress. You
00:26:16
know, that is true for banks,
for funds, but also for
00:26:20
infrastructures. I think the
challenge is really around those
00:26:28
issues that are not readily
captured in the existing
00:26:35
regulatory approaches. And I
think regulators themselves are
00:26:43
working on upscaling their own
technology in order to meet
00:26:47
these technological challenges.
So we hear quite a bit about, you
00:26:56
know, regulatory upskilling. So
for example, SupTech and Reg-
00:26:59
Tech. So, where supervisors and
regulators are using, themselves,
00:27:03
technologies to detect risks
better. So, you know, on the one
00:27:10
hand, there could be new risks.
On the other hand, policy makers
00:27:12
could be more effective. But let
me— let me be a little bit more
00:27:17
specific about, you know, some
regulatory initiatives that we
00:27:22
have seen already. So for
example, in Hong Kong, there is
00:27:26
an AI governance framework that
the Hong Kong monetary authority
00:27:31
has put out, which is really
about the risk management and
00:27:37
the reliability of AI systems
used in the banking sector.
00:27:41
That's one concrete example. The
second one in the UK, the
00:27:45
Financial Conduct Authority has
implemented requirements for the
00:27:51
explainability of AI-driven
decision making. So for example,
00:27:56
in credit scoring. So, it's fine
to use those models, but you
00:28:00
need to have sort of a rationale
or an explainability around that.
00:28:07
And that, in turn, helps to
mitigate the risks associated
00:28:11
with the opaqueness of the
models. In Singapore, there are
00:28:18
principles called the Fairness,
Ethics, Accountability and
00:28:22
Transparency principles for AI
in finance. And that really
00:28:27
helps the financial institutions
in Singapore to have a
00:28:31
responsible usage of AI systems.
And that is particularly focused
00:28:37
also on bias and discrimination.
And I would finally mention, in
00:28:43
the European Union, there's the
General Data Protection
00:28:47
Regulation, GDPR, and that has
been adapted to the use of AI
00:28:57
practices in financial services,
particularly for focusing on the
00:29:02
on data protection and privacy
practices. So this is some of
00:29:06
the examples I alluded to
earlier, aimed at reducing the
00:29:11
risk of data breaches and the
misuse of personal information.
00:29:16
Okay. All very fascinating.
Certainly a lot more to hear
00:29:23
about and learn about from this
AI revolution in financial
00:29:27
markets and all the challenges
it poses when thinking about
00:29:31
risks and regulation. Thank you
very much, Tobias, for all the
00:29:36
great work that you're doing at
the IMF and for joining us for
00:29:39
this podcast.
00:29:42
Thanks so much, Itay.
Very good to see you.
00:29:44
And warm regards from
Washington to Pennsylvania.
00:29:51
Thank you. And thank you, everyone,
for listening.
