00:00:00
Dan Loney: Well, there's been concern for
several years now about the impact that student loans are
having on people's lives once
00:00:06
they're done with their time in
college. New research looks at what can be done to provide a
quote, unquote "nudge" and help
00:00:13
people on that repayment path so
that they have limited negative impacts. Pleasure to be joined
by Katy Milkman, Professor of
00:00:20
Operations, Information and
Decisions here at the Wharton School, and obviously one of the
important people involved in this
00:00:26
research. Katy, always great to
talk with you and talk about your research. Thanks for
a couple of moments.
00:00:31
Katy Milkman: Always a pleasure to be
here. Thanks for having me. All right. So, tell us
about this research.
00:00:35
And I— I think
the interesting thing at the top, 13 million people
involved in this research?
00:00:42
Yeah. Well, I should say, you
know, it was led by an amazing PhD student named Rob Kuan, and
it was made possible by a
00:00:48
partnership with the Department
of Education, an incredible group working there in fall of
2023. And they were planning a
00:00:57
communication campaign after the
pause that you might remember happened during COVID on
requirements that students repay
00:01:05
their loans, the pause on
interest. And so we were thinking about, how could we be
helpful? They already had a plan
00:01:11
to start communicating with
people who missed their first payment once that payment pause
ended, and they wanted to pull
00:01:17
in behavioral scientists to
help. So we got this amazing opportunity to do science at
this incredible scale, with 13
00:01:24
million borrowers who missed a
payment and needed a little nudge to get them back on track.
And we had the opportunity to be
00:01:32
part of thinking, you know,
what might be most effective,
and testing to make sure that then the
Department of Education could
00:01:37
use the very best insights from
our work to move forward and try to help borrowers who were at
risk of delinquency.
00:01:46
So when you say nudge,
you mean what? So we helped develop an email campaign. So if a borrower
had a missed payment,
00:01:53
they'd get a notification from
the Department of Education that was an alert, essentially
to say, you know, this has
00:02:01
happened, and here's some advice
on next steps. And we varied what those messages said. And so
by nudge, I mean there's no
00:02:09
change in the incentive
structure they face. You know, they're not getting different
interest rates or different
00:02:15
rules applied to them or
different periods of forbearance. Instead, it's just
a different way of communicating
00:02:21
the same information to see
what's most effective, using insights from behavioral science
to try to make it maximally
00:02:27
helpful to borrowers. Yes. Yeah, the idea of
behaviorally informed is just that a group of scientists who
were trying to take the best
00:02:42
insights they could from the
behavioral science literature about how humans make decisions,
developed a set of messages that
00:02:50
simplified things as much as
possible, made it as clear as possible what the action steps
should be. And then we— we tested
00:02:58
first of all, was that better
than just sort of sending people a notification that they should
go check in with their servicer?
00:03:04
Could we do better than that by
using sort of every trick up our sleeves as behavioral
scientists? That's the first
00:03:10
thing we tested in this project.
And then we also did some other testing of things we weren't
sure about, where we weren't
00:03:16
positive which way it would go.
For instance, we said, you know, does it really help if we bug
them again? Should we send a
00:03:23
reminder, not just one message,
but two? And does that create value? We also looked at whether
or not, when we talked to them
00:03:29
about actions they could take
that would reduce the— like, you know, that would be helpful,
actions like signing up for
00:03:36
what's called an income-driven
repayment plan, where, if you have lower income, you can
actually make smaller payments.
00:03:41
So this is really helpful for a
lot of people struggling to make payments, to switch and sign up
for that kind of a plan. If we
00:03:47
described that and talked about
the savings they could achieve potentially in percentage terms,
would that be more effective, or
00:03:54
would it be more effective to talk
about the savings they might receive in terms of what they'd
owe each month in dollar terms?
00:04:00
We weren't sure, so we tested
that. And we also tested whether or not it's better when we had
multiple pieces of advice for
00:04:06
people, should we give them one
piece of advice at a time? So each email focused on one piece
of advice, or should we give
00:04:14
them two pieces of advice
repeatedly? So assume you only can send, say two communications
to someone a month, because
00:04:19
there's some limit on either
nagging or how many times you can bother them before they
click unsubscribe— so you have
00:04:26
that limit. Are you better off
separating your points or putting them all together? So
those were the things we tested.
00:04:32
Do the behaviorally-informed
emails work? Do reminders make them better? Should we talk
about savings in percentage or
00:04:37
dollar terms, and should we give
you one action step at a time, or two action steps at once
repeatedly.
00:04:43
So two questions for me off of
the scenario you just laid out. One being, it sounds like giving
them the information, that the
00:04:52
element of having an option or
having a variety of kind of components that— that they can
refer to, certainly help the
00:05:00
people have a better
understanding of ways that they can kind of deal with some of
these issues. - Absolutely.
00:05:07
So we we think there's value
add, and we empirically demonstrate that the sort of
package of not just telling you
00:05:16
you need to check in and figure
out what's going on, but giving you this guidance about, it might
be a good idea to sign up for
00:05:22
auto debit pay, or automatic
payments, so that you don't have to think every month to go and
make your payment. It might be a
00:05:27
good idea to sign up for these
income-driven repayment plans. We do believe that that
information is going to be
00:05:32
valuable and that we can frame
it in a valuable way, and we find that we can. So that, I
think, is the least interesting
00:05:38
part of the paper in some
respect, although it's always nice to prove an assumption
holds. So we find that that is
00:05:44
beneficial and reduces 60-day
delinquencies after borrowers receive those kinds of
communications. But I think
00:05:52
what's more important and more
interesting is also some of the things we learned on top of that
that were maybe not quite as
00:05:57
intuitive. In fact, more
important than that was actually just sending a reminder. The
reminder was more impactful than
00:06:03
the behaviorally-informed
initial email itself. So nagging really works. That's a theme of
a lot of research I've done. We
00:06:09
just want to remind you multiple
times if you haven't taken action on something important,
and that really boosts the
00:06:16
efficacy even of these
behaviorally informed, sort of well designed reminders. We also
learned that when we talk about
00:06:24
what you could save in
percentage terms, that was more effective than talking to you
about the dollar amount you
00:06:30
could save, which I think is
really interesting and potentially generalizable to
a lot of contexts. And then the
00:06:35
final big takeaway was that we
were better off giving you the same two pieces of advice twice
across two emails, than sending
00:06:44
one message with one piece of
advice followed by a second message with just one piece of
advice. So rather than breaking
00:06:51
up two things you want to tell
people so that they only have to think about one step at a time,
you're better off hammering that
00:06:57
home twice. And that was, I
think, a real question. Like, simplification is important, but
so is repetition. And we were
00:07:04
able to isolate, at least in
this case, that repetition beat
simplification. So let me go back to the component
of the savings in percentage
00:07:13
terms, and why you think
potentially that was more beneficial to people than
actually seeing what the cold,
00:07:21
hard dollar number was in terms
of what they would be saving. It's a really great question, and we
truly didn't know what the
00:07:27
answer would be. Past research,
there's reasons to think it could go either way. Dollars
might be simpler and easier to
00:07:33
interpret, if people aren't
super comfortable thinking in percentage terms. What we— our
best explanation, and it's still
00:07:40
a guess, and future research is
needed to really look under the hood, because all we have at
this point is just, we sent
00:07:45
emails. This one worked better.
So we don't have, you know, we don't have interviews with
people. We— we don't have—
00:07:51
we don't have the kind of data I'd
love to have to explain the why. But our best guess is that the
percentage numbers were quite
00:07:57
large. So we're talking about
something like thinking about a 46% reduction, or a 40%
reduction in your payment, and
00:08:07
dollar terms were, maybe, you
know, similar in scale. So, like, that's about a $40
reduction in your monthly
00:08:15
payment, is also roughly 40%. So
these are the kinds of numbers we're throwing around at people.
They're pretty big percentages,
00:08:21
and maybe the dollar isn't as
huge. We know from some recent research by other scholars that
when you look at a percentage,
00:08:29
you think of it on a scale from
zero to 100. Of course that's not true, right? I could give
you 300% increase in your bill
00:08:36
or— -Right.
- Right? But— but we think of them
as being on that scale. So when numbers get
sort of in the two-digit range
00:08:43
and get up close anywhere near
100%, people think that's a huge savings, because 100 is the max.
And these numbers were pretty
00:08:50
big on the percentage scale.
So one possibility is that in percentage terms, this felt
bigger in magnitude because of
00:09:00
that sense that anything that's
getting close to 100 is, wow, massive. Which is not fully
accurate, but— but it's a
00:09:06
perception people have. So that
might be why percentages beat dollars, but we don't know for
sure. And one of the fun things
00:09:12
about an experiment like this is
we can point to what works, and we can also point to exciting
new directions that researchers
00:09:19
who like to, you know, dig into
the why, and you know— you could put somebody in an MRI scanner
and try to explore, you know,
00:09:25
how do they react differently
when they see numbers versus percentages or dollars versus
percentages? Both are numbers,
00:09:30
of course. But— but, you know,
does the— does the brain process that differently? Or if you ask
people to talk about what they
00:09:38
imagine being able to spend on
now, after showing them a number in percentage terms that they
might be able to save, or in
00:09:44
dollar terms, would they think
of different things? So there's a whole host of research you
could do to follow up on this,
00:09:50
and we think that's really
exciting, but it's not the focus of this project here. We just
show it works better. We
00:09:55
speculate a little bit about
why, and we are excited to see what future researchers find.
00:09:59
Can you estimate, then, off of this
research— and I guess off of this group— potentially, how
many delinquencies could have
00:10:06
been prevented if this path had
been followed? Yeah. Well, what's really exciting
about even small gains at scale
00:10:13
is how massive they can be. So
if— once you sort of imagine, what if we scaled the best
performer to all 13 million
00:10:20
borrowers who had been in this
experiment, which is, of course, straightforward for an
organization like the Department
00:10:27
of Education to do, they could
use now the best performer. What we see is, just over this time
period studied, it would have
00:10:33
averted about 80,000 sixty-day
delinquencies. Just these small changes to improving messages.
And I think what's exciting
00:10:40
about that is it shows the power
of behavioral science and testing at scale to make an
impact on lives. These are
00:10:48
essentially free adjustments to
a messaging campaign that have the potential for large impact.
And again, each individual
00:10:56
effect is pretty small, but when
you scale it to 13 million people, that's when you start to
see there could be real, real
00:11:03
benefits. And in fact, we know
this campaign did prevent delinquencies, because people
got these— these behaviorally-
00:11:09
informed messages, and it had
this impact. So you mentioned about, you know,
hopefully doing more research in
00:11:15
this area. But that being said,
off of the research that you've done here, what do you think is
your biggest takeaway that
00:11:22
people should really kind of
understand about this process? I mean, this is one of those
projects where it's hard to boil
00:11:29
it down to one takeaway, because
there's so many. But probably the one takeaway is, reminders really
work. Reminders really work. And
00:11:38
I guess the second takeaway
would be, you know, when you use behavioral science and you do AB
testing, you can get— you can
00:11:46
squeeze more juice out of your—
you know, the lemon or the orange or whatever metaphor you
want to use. We were able to
00:11:52
make those reminders not only
work by using behavioral science, but we were able to use
AB testing to optimize further
00:11:58
in ways that are pretty potent.
And so I'm a big fan of AB testing. I'm a big fan of
applying scientific insights to
00:12:06
improve communications. And this
just shows that both of those things matter and can have huge
positive impact on our wallets.
00:12:15
Katy, great work. Thanks very much for
joining us and giving us all this insight. - Thank you so much
for having me.
00:12:21
You got it. Katy Milkman,
Professor of Operations, Information and Decisions here
at the Wharton School.