00:00:00
Dan Loney: Well, there's been concern for
several years now about the
00:00:03
impact that student loans are
having on people's lives once
00:00:06
they're done with their time in
college. New research looks at
00:00:09
what can be done to provide a
quote, unquote "nudge" and help
00:00:13
people on that repayment path so
that they have limited negative
00:00:17
impacts. Pleasure to be joined
by Katy Milkman, Professor of
00:00:20
Operations, Information and
Decisions here at the Wharton
00:00:23
School, and obviously one of the
important people involved in this
00:00:26
research. Katy, always great to
talk with you and talk about
00:00:29
your research. Thanks for
a couple of moments.
00:00:31
Katy Milkman: Always a pleasure to be
here. Thanks for having me.
00:00:34
All right. So, tell us
about this research.
00:00:35
And I— I think
the interesting thing at
00:00:37
the top, 13 million people
involved in this research?
00:00:42
Yeah. Well, I should say, you
know, it was led by an amazing
00:00:45
PhD student named Rob Kuan, and
it was made possible by a
00:00:48
partnership with the Department
of Education, an incredible
00:00:51
group working there in fall of
2023. And they were planning a
00:00:57
communication campaign after the
pause that you might remember
00:01:01
happened during COVID on
requirements that students repay
00:01:05
their loans, the pause on
interest. And so we were
00:01:09
thinking about, how could we be
helpful? They already had a plan
00:01:11
to start communicating with
people who missed their first
00:01:14
payment once that payment pause
ended, and they wanted to pull
00:01:17
in behavioral scientists to
help. So we got this amazing
00:01:19
opportunity to do science at
this incredible scale, with 13
00:01:24
million borrowers who missed a
payment and needed a little
00:01:29
nudge to get them back on track.
And we had the opportunity to be
00:01:32
part of thinking, you know,
what might
00:01:33
be most effective,
and testing to
00:01:35
make sure that then the
Department of Education could
00:01:37
use the very best insights from
our work to move forward and try
00:01:43
to help borrowers who were at
risk of delinquency.
00:01:46
So when you say nudge,
you mean what?
00:01:50
So we helped develop an email campaign.
00:01:51
So if a borrower
had a missed payment,
00:01:53
they'd get a notification from
the Department of Education
00:01:56
that was an alert, essentially
to say, you know, this has
00:02:01
happened, and here's some advice
on next steps. And we varied
00:02:06
what those messages said. And so
by nudge, I mean there's no
00:02:09
change in the incentive
structure they face. You know,
00:02:12
they're not getting different
interest rates or different
00:02:15
rules applied to them or
different periods of
00:02:17
forbearance. Instead, it's just
a different way of communicating
00:02:21
the same information to see
what's most effective, using
00:02:24
insights from behavioral science
to try to make it maximally
00:02:27
helpful to borrowers.
00:02:36
Yes. Yeah, the idea of
behaviorally informed is just
00:02:39
that a group of scientists who
were trying to take the best
00:02:42
insights they could from the
behavioral science literature
00:02:45
about how humans make decisions,
developed a set of messages that
00:02:50
simplified things as much as
possible, made it as clear as
00:02:54
possible what the action steps
should be. And then we— we tested
00:02:58
first of all, was that better
than just sort of sending people
00:03:00
a notification that they should
go check in with their servicer?
00:03:04
Could we do better than that by
using sort of every trick up our
00:03:08
sleeves as behavioral
scientists? That's the first
00:03:10
thing we tested in this project.
And then we also did some other
00:03:14
testing of things we weren't
sure about, where we weren't
00:03:16
positive which way it would go.
For instance, we said, you know,
00:03:20
does it really help if we bug
them again? Should we send a
00:03:23
reminder, not just one message,
but two? And does that create
00:03:26
value? We also looked at whether
or not, when we talked to them
00:03:29
about actions they could take
that would reduce the— like, you
00:03:33
know, that would be helpful,
actions like signing up for
00:03:36
what's called an income-driven
repayment plan, where, if you
00:03:39
have lower income, you can
actually make smaller payments.
00:03:41
So this is really helpful for a
lot of people struggling to make
00:03:44
payments, to switch and sign up
for that kind of a plan. If we
00:03:47
described that and talked about
the savings they could achieve
00:03:50
potentially in percentage terms,
would that be more effective, or
00:03:54
would it be more effective to talk
about the savings they might
00:03:57
receive in terms of what they'd
owe each month in dollar terms?
00:04:00
We weren't sure, so we tested
that. And we also tested whether
00:04:03
or not it's better when we had
multiple pieces of advice for
00:04:06
people, should we give them one
piece of advice at a time? So
00:04:09
each email focused on one piece
of advice, or should we give
00:04:14
them two pieces of advice
repeatedly? So assume you only
00:04:17
can send, say two communications
to someone a month, because
00:04:19
there's some limit on either
nagging or how many times you
00:04:23
can bother them before they
click unsubscribe— so you have
00:04:26
that limit. Are you better off
separating your points or
00:04:29
putting them all together? So
those were the things we tested.
00:04:32
Do the behaviorally-informed
emails work? Do reminders make
00:04:35
them better? Should we talk
about savings in percentage or
00:04:37
dollar terms, and should we give
you one action step at a time,
00:04:41
or two action steps at once
repeatedly.
00:04:43
So two questions for me off of
the scenario you just laid out.
00:04:48
One being, it sounds like giving
them the information, that the
00:04:52
element of having an option or
having a variety of kind of
00:04:56
components that— that they can
refer to, certainly help the
00:05:00
people have a better
understanding of ways that they
00:05:03
can kind of deal with some of
these issues. - Absolutely.
00:05:07
So we we think there's value
add, and we empirically
00:05:12
demonstrate that the sort of
package of not just telling you
00:05:16
you need to check in and figure
out what's going on, but giving
00:05:18
you this guidance about, it might
be a good idea to sign up for
00:05:22
auto debit pay, or automatic
payments, so that you don't have
00:05:24
to think every month to go and
make your payment. It might be a
00:05:27
good idea to sign up for these
income-driven repayment plans.
00:05:29
We do believe that that
information is going to be
00:05:32
valuable and that we can frame
it in a valuable way, and we
00:05:34
find that we can. So that, I
think, is the least interesting
00:05:38
part of the paper in some
respect, although it's always
00:05:40
nice to prove an assumption
holds. So we find that that is
00:05:44
beneficial and reduces 60-day
delinquencies after borrowers
00:05:48
receive those kinds of
communications. But I think
00:05:52
what's more important and more
interesting is also some of the
00:05:54
things we learned on top of that
that were maybe not quite as
00:05:57
intuitive. In fact, more
important than that was actually
00:06:00
just sending a reminder. The
reminder was more impactful than
00:06:03
the behaviorally-informed
initial email itself. So nagging
00:06:06
really works. That's a theme of
a lot of research I've done. We
00:06:09
just want to remind you multiple
times if you haven't taken
00:06:13
action on something important,
and that really boosts the
00:06:16
efficacy even of these
behaviorally informed, sort of
00:06:20
well designed reminders. We also
learned that when we talk about
00:06:24
what you could save in
percentage terms, that was more
00:06:27
effective than talking to you
about the dollar amount you
00:06:30
could save, which I think is
really interesting and
00:06:32
potentially generalizable to
a lot of contexts. And then the
00:06:35
final big takeaway was that we
were better off giving you the
00:06:40
same two pieces of advice twice
across two emails, than sending
00:06:44
one message with one piece of
advice followed by a second
00:06:47
message with just one piece of
advice. So rather than breaking
00:06:51
up two things you want to tell
people so that they only have to
00:06:54
think about one step at a time,
you're better off hammering that
00:06:57
home twice. And that was, I
think, a real question. Like,
00:07:00
simplification is important, but
so is repetition. And we were
00:07:04
able to isolate, at least in
this case,
00:07:07
that repetition beat
simplification.
00:07:09
So let me go back to the component
of the savings in percentage
00:07:13
terms, and why you think
potentially that was more
00:07:17
beneficial to people than
actually seeing what the cold,
00:07:21
hard dollar number was in terms
of what they would be saving.
00:07:25
It's a really great question, and we
truly didn't know what the
00:07:27
answer would be. Past research,
there's reasons to think it
00:07:30
could go either way. Dollars
might be simpler and easier to
00:07:33
interpret, if people aren't
super comfortable thinking in
00:07:36
percentage terms. What we— our
best explanation, and it's still
00:07:40
a guess, and future research is
needed to really look under the
00:07:43
hood, because all we have at
this point is just, we sent
00:07:45
emails. This one worked better.
So we don't have, you know, we
00:07:48
don't have interviews with
people. We— we don't have—
00:07:51
we don't have the kind of data I'd
love to have to explain the why.
00:07:54
But our best guess is that the
percentage numbers were quite
00:07:57
large. So we're talking about
something like thinking about a
00:08:01
46% reduction, or a 40%
reduction in your payment, and
00:08:07
dollar terms were, maybe, you
know, similar in scale. So,
00:08:11
like, that's about a $40
reduction in your monthly
00:08:15
payment, is also roughly 40%. So
these are the kinds of numbers
00:08:19
we're throwing around at people.
They're pretty big percentages,
00:08:21
and maybe the dollar isn't as
huge. We know from some recent
00:08:25
research by other scholars that
when you look at a percentage,
00:08:29
you think of it on a scale from
zero to 100. Of course that's
00:08:32
not true, right? I could give
you 300% increase in your bill
00:08:36
or— -Right.
- Right? But—
00:08:38
but we think of them
as being on
00:08:39
that scale. So when numbers get
sort of in the two-digit range
00:08:43
and get up close anywhere near
100%, people think that's a huge
00:08:47
savings, because 100 is the max.
And these numbers were pretty
00:08:50
big on the percentage scale.
So one possibility is that in
00:08:56
percentage terms, this felt
bigger in magnitude because of
00:09:00
that sense that anything that's
getting close to 100 is, wow,
00:09:03
massive. Which is not fully
accurate, but— but it's a
00:09:06
perception people have. So that
might be why percentages beat
00:09:09
dollars, but we don't know for
sure. And one of the fun things
00:09:12
about an experiment like this is
we can point to what works, and
00:09:15
we can also point to exciting
new directions that researchers
00:09:19
who like to, you know, dig into
the why, and you know— you could
00:09:22
put somebody in an MRI scanner
and try to explore, you know,
00:09:25
how do they react differently
when they see numbers versus
00:09:27
percentages or dollars versus
percentages? Both are numbers,
00:09:30
of course. But— but, you know,
does the— does the brain process
00:09:35
that differently? Or if you ask
people to talk about what they
00:09:38
imagine being able to spend on
now, after showing them a number
00:09:42
in percentage terms that they
might be able to save, or in
00:09:44
dollar terms, would they think
of different things? So there's
00:09:47
a whole host of research you
could do to follow up on this,
00:09:50
and we think that's really
exciting, but it's not the focus
00:09:52
of this project here. We just
show it works better. We
00:09:55
speculate a little bit about
why, and we are excited to see
00:09:58
what future researchers find.
00:09:59
Can you estimate, then, off of this
research— and I guess off of
00:10:02
this group— potentially, how
many delinquencies could have
00:10:06
been prevented if this path had
been followed?
00:10:09
Yeah. Well, what's really exciting
about even small gains at scale
00:10:13
is how massive they can be. So
if— once you sort of imagine,
00:10:17
what if we scaled the best
performer to all 13 million
00:10:20
borrowers who had been in this
experiment, which is, of course,
00:10:24
straightforward for an
organization like the Department
00:10:27
of Education to do, they could
use now the best performer. What
00:10:30
we see is, just over this time
period studied, it would have
00:10:33
averted about 80,000 sixty-day
delinquencies. Just these small
00:10:37
changes to improving messages.
And I think what's exciting
00:10:40
about that is it shows the power
of behavioral science and
00:10:43
testing at scale to make an
impact on lives. These are
00:10:48
essentially free adjustments to
a messaging campaign that have
00:10:52
the potential for large impact.
And again, each individual
00:10:56
effect is pretty small, but when
you scale it to 13 million
00:10:59
people, that's when you start to
see there could be real, real
00:11:03
benefits. And in fact, we know
this campaign did prevent
00:11:06
delinquencies, because people
got these— these behaviorally-
00:11:09
informed messages, and it had
this impact.
00:11:12
So you mentioned about, you know,
hopefully doing more research in
00:11:15
this area. But that being said,
off of the research that you've
00:11:18
done here, what do you think is
your biggest takeaway that
00:11:22
people should really kind of
understand about this process?
00:11:27
I mean, this is one of those
projects where it's hard to boil
00:11:29
it down to one takeaway, because
there's so many. But probably the one
00:11:32
takeaway is, reminders really
work. Reminders really work. And
00:11:38
I guess the second takeaway
would be, you know, when you use
00:11:42
behavioral science and you do AB
testing, you can get— you can
00:11:46
squeeze more juice out of your—
you know, the lemon or the
00:11:49
orange or whatever metaphor you
want to use. We were able to
00:11:52
make those reminders not only
work by using behavioral
00:11:55
science, but we were able to use
AB testing to optimize further
00:11:58
in ways that are pretty potent.
And so I'm a big fan of AB
00:12:02
testing. I'm a big fan of
applying scientific insights to
00:12:06
improve communications. And this
just shows that both of those
00:12:10
things matter and can have huge
positive impact on our wallets.
00:12:15
Katy, great work. Thanks very much for
joining us and giving us all
00:12:19
this insight. - Thank you so much
for having me.
00:12:21
You got it. Katy Milkman,
Professor of Operations,
00:12:23
Information and Decisions here
at the Wharton School.
