00:00:00
Dan Loney: Well, Artificial Intelligence
00:00:01
is going to be one of
the areas to focus on
00:00:03
as we head back into the new
school year. All levels of
00:00:06
education are taking a look at
the potential of AI, and what it
00:00:12
could bring to education. But
there are also some areas of
00:00:15
potential concern. - Welcome to
The Ripple Effect, the podcast
00:00:20
that takes you on a journey
through the minds of Wharton
00:00:22
faculty. I'm your host, Dan
Loney. And in each episode,
00:00:26
we'll be diving deep into the
inspiration behind the
00:00:28
groundbreaking research that
Wharton professors have
00:00:31
conducted, and exploring how
their findings resonate with the
00:00:35
world today. - Pleasure to be
joined here in studio by
00:00:37
Christian Terweisch, who is
professor and Co-Director of the
00:00:41
Mack Institute for Innovation
Management here at the Wharton
00:00:44
School. Great to see you again.
00:00:45
Christian Terweisch: Thanks for having me.
00:00:47
Thank you. And so the
interesting thing is, this is
00:00:49
something that really came into
your mindset in your teaching
00:00:53
here at Wharton. You started to
see AI as a component that you
00:00:56
needed to focus on.
00:00:57
Yeah, absolutely. So by way of
background, many, many years
00:01:01
ago, I have a background in
computer science. Even had a
00:01:04
class on neural network. But
really put this topic to the
00:01:07
side till Thanksgiving about two
years ago. My kids were home
00:01:11
from college and their work. And
like many other families in this
00:01:15
country, we talked about ChatGPT.
And so the topic came up
00:01:18
like, "Hey, Dad, you teach at a
business school. Can ChatGPT
00:01:21
take your final exam?" And so
what we did is we just took my
00:01:25
final exam in Operations
Management for the MBA students,
00:01:28
put it into ChatGPT, and
it did pretty, pretty well. And
00:01:32
so we put the story online, it
went viral, and ever since,
00:01:35
people believe I know something
about AI.
00:01:38
But did it surprise you
that ChatGPT was able to do
00:01:42
well on a Wharton exam?
00:01:44
Absolutely. Not just that it
did well. The way it answered
00:01:48
the question, the level of
sophistication of the language,
00:01:51
the persuasiveness, the
smoothness of the writing. It
00:01:55
just blew my mind.
- So how
00:01:57
has that changed your thought
process in how you teach, how
00:02:01
you give exams? What— how has it
impacted your operation?
00:02:05
Well, the first reaction is,
"Wow, this is big, right? This is
00:02:08
a major productivity boost."
Before we talk more about
00:02:11
teaching, let's digress a little
bit in what it means to have a
00:02:13
productivity boost, right? We're
going to see productivity boosts in
00:02:16
many industries. For doctors,
for nurses, for lawyers, for
00:02:20
accountants, for radio hosts, for
business school teachers,
00:02:23
right? We're all gonna get more
productive. But what does it
00:02:26
mean to be more productive? So
imagine— let's take a
00:02:29
barbershop, right? I have a
technology that makes the barber
00:02:32
twice as productive. What does
it mean? That barber can either
00:02:35
make haircuts that are
twice as good, or they can take
00:02:38
twice as many customers and get
a haircut? Right? So I'm
00:02:41
shifting what we call the cost-
quality frontier, because
00:02:44
quality frontier has shifted
out. And I can do now more with
00:02:47
less. So what does it mean for
teaching? Well, I can either
00:02:51
teach more students, or I can
hold the number of students I'm
00:02:53
teaching constant and provide a
better job. - What
00:02:56
do you think it's going to mean
most for students, in terms of
00:03:00
how they go through their
education process moving forward?
00:03:03
I think you have to distinguish
between two types of use cases,
00:03:06
right? There's a use case for
the students who get the same
00:03:09
education as before, they could
just get it in a more efficient
00:03:14
and thus cheaper way.
00:03:15
I remember many
years ago, you and I talked
00:03:17
about massive open online
learning. - Yeah. Yeah.
00:03:19
I taught one of the first MOOCs
here at Wharton.
00:03:22
So that was all about
taking the
00:03:23
Wharton experience and making it
cheaper. Now, the other use case,
00:03:27
and that's one I think we
should be more bullish about, is
00:03:30
we do a better job, right? The
haircut looks twice as good. So
00:03:33
we have to ask ourselves in
education, what does it mean
00:03:36
to provide education that is
twice as exciting for the same
00:03:39
number of students?
00:03:40
Is there a level of concern
about students maybe not having
00:03:44
the level of learning on specific
topics because they're relying
00:03:48
on the component of AI to help
them through the process?
00:03:52
Absolutely. Right. So there's a
concern, just take a calculator
00:03:55
as an example, when the
calculator came out, when spell
00:03:57
checks came out, we were
concerned— and I think
00:04:00
rightfully concerned— that
students would forget how to do
00:04:03
basic calculations, and that
they would weaken on their
00:04:06
spelling. It's like if you take
a pilot, and the pilot is flying
00:04:10
autopilot all the time, is the
pilot at risk of forgetting how
00:04:13
to fly an airplane? Probably.
00:04:16
The question of, though, the
different levels of education
00:04:19
and how they're going to be
impacted— we're already seeing
00:04:22
those types of discussions form
at this point. Colleges are
00:04:26
obviously thinking about it.
Even high schools and middle
00:04:29
schools are thinking about how
AI can either be a tool to
00:04:33
benefit the students' learning,
but at the same time, being
00:04:37
careful of the component of
exams and tests that they're
00:04:41
providing to the school as well.
- Yeah,
00:04:43
so let's stay with the exams for
just a second. Again, you
00:04:46
mentioned colleges and K12. I
think there the impact is
00:04:49
going to be very similar. The
distinction that matters is,
00:04:52
unfortunately, the rich and the
poor, right? There are
00:04:54
privileged universities and
privileged private schools,
00:04:57
Penn Wharton being one of them.
And there, the question is
00:05:01
all about, like, how can we take
the students that we teach
00:05:03
anyway, and provide them a
better education? Now, I'm
00:05:07
realistic about this. There are
many situations in education
00:05:10
where money is scarce. Today,
think about public schooling in
00:05:13
poorer areas. And there, the
focus will be for— for
00:05:16
the politicians, about how to
make education cheaper. Most of
00:05:20
them boost the frontier. That's
a good thing. But I wish there
00:05:23
would be more people who would
benefit in the way that we could
00:05:25
benefit at Wharton.
- But obviously,
00:05:26
just when you think about K
through 12, just having the
00:05:30
access to computers, and then
obviously having the access to
00:05:34
AI or ChatGPT as an assistant in,
you know, doing research or
00:05:38
doing a paper, that's a
significant hindrance in many
00:05:41
cases in different parts of the
country. - I think most
00:05:44
K12 teachers I've talked to
will have the first reaction
00:05:47
like, "This is going to be
problematic." Right. And I think
00:05:49
it's— sure I mean, if you have,
basically, your brother-in-law
00:05:52
taking the exam for you, I'm
sure that is hindering the
00:05:56
education process. In an
institution like ours, we're
00:05:59
certifying a specific skill for
somebody. And if that
00:06:03
certification is now weakened
by the fact that we're not sure
00:06:06
whether the person has the skill
or the technology has the skill,
00:06:09
we have a problem.
- But I guess
00:06:11
we shouldn't be surprised
that that is somewhat the
00:06:13
reaction that we're seeing right
now, is because in many cases,
00:06:16
when we see a new technology
come on the board with great
00:06:20
potential, there is a concern.
00:06:21
There's a hesitancy
that's out there.
00:06:23
And it should be, right? I
think— again, let's go back to
00:06:25
calculators. On calculators, you
might still remember how to do
00:06:29
seven times seven. Let me just
get the math right. It's 49, if
00:06:33
I'm not mistaken. So you
remember that. I remember that.
00:06:36
But somebody who grew up in the age
of the calculator, they might
00:06:39
say, like, "I don't have to know
seven times seven, I just ask
00:06:41
the calculator." So you can now
make the same argument for
00:06:44
ChatGPT, where you say, like,
"I don't know how to write an
00:06:46
English essay, but ChatGPT can
do it for me." So I think it
00:06:50
really boils down to the
question, like, what skills do we
00:06:53
want to teach our kids? And what
is the learning outcome that I
00:06:56
want to have for my students
in my course?
00:06:57
And there's also the component
of the development of the Chat
00:07:00
GPT itself, with the different
versions we've seen come out
00:07:03
even in the short period of
time, and how in the technical
00:07:07
side can improve, you know,
what's available for use in and
00:07:11
around ChatGPT.
00:07:12
Yeah, so I started using 3.5,
version 3.5, about a year and a
00:07:16
half, two years ago, for my
initial exam. That did pretty
00:07:20
well already. And then it was 4
coming out, and 4 Turbo. And now 4
00:07:24
Omnichannel. It just got better
and better.
00:07:27
What are the discussions you've
had with your students about the
00:07:30
use of ChatGPT? What are the
parameters and barriers that
00:07:33
they can and cannot use?
- I'm
00:07:35
like a harsh German guy, right?
So the rule is very simple.
00:07:38
You're not allowed to use
ChatGPT on the final exam.
00:07:41
So that's the first and most
important rule. The second one
00:07:44
is, you can use it for case
preparation and discussion. I've
00:07:48
seen not too many students take
advantage of that. But I expect
00:07:51
that number to be growing. So
I'm totally fine if you
00:07:55
take the case, upload into
ChatGPT and ask for advice, I
00:07:58
think that's fair game. And then
in class, we're typically not
00:08:02
using it at all. I mean, I think
it works best when students
00:08:06
prepare for the case, for an
assignment, in advance of class.
00:08:10
So then seemingly we're looking
at a time where we're going to
00:08:13
go through a period where
leadership of the education
00:08:17
sector, whether it be colleges,
universities or school districts,
00:08:21
are going to have to have long
conversations about what is
00:08:24
allowable and what is not allowable
around AI. - Yeah,
00:08:27
I've wrote an Op Ed on the
question of what is allowable
00:08:30
and not. And I think it's helpful
to distinguish between three
00:08:32
types of tests. The tests that
you do to certify a skill— and
00:08:36
for those, don't use ChatGPT.
Then the second thing is
00:08:39
what I oftentimes do as a
teacher in class, I ask my
00:08:42
students questions to just get a
sense of how they progress on
00:08:45
their learning journey. And
again, using ChatGPT would be
00:08:48
misleading. The third type of
test is really more like a
00:08:51
homework that is hopefully
engaging the students in the
00:08:54
learning. And for that, if I can
use ChatGPT to increase that
00:08:58
engagement, that would be a good
thing. But so always ask
00:09:01
yourself, why are you testing?
Tests are not for the sake of
00:09:04
testing. As an educator, you
have to always think, like, Why
00:09:07
do I test?
- How much do you think
00:09:10
ChatGPT has impacted
your profession right now?
00:09:15
I think we're just seeing the
implications for the academic
00:09:19
profession. I use it for idea
generation oftentimes. So I've
00:09:23
wrote a paper that shows how
ChatGPT outperforms human
00:09:27
beings at coming up with ideas for
new products or services. That
00:09:30
is something that somebody who
teaches innovation cares a lot
00:09:33
about. I use it for writing.
When I'm stuck with something, I
00:09:36
say let ChatGPT continue that
page for me. I think the key
00:09:40
thing that you have to realize
about ChatGPT is it still makes
00:09:43
mistakes. But there are
situations where mistakes are
00:09:46
not consequential. So if I'm
asking you for 10 ideas, and
00:09:49
five of them are bad, I still
get five good ideas. If I asked
00:09:53
you to do heart surgery for me
and five of those heart surgeries go
00:09:57
wrong, that would be a disaster.
So you want to use it in
00:10:00
relatively low stakes
environments, where you have a
00:10:03
human in the loop so that you
make sure— make sure that the
00:10:05
quality is ultimately where you
want it to be. Because
00:10:08
ultimately, it's your name on the
paper.
00:10:10
And again, if you have that
understanding that things like
00:10:15
ChatGPT can be an assistant, be
a guide, in the process of doing
00:10:20
a test exam, or whatever it is,
but still that you have to, you
00:10:24
know, do the double check, but
you know, fall back and make
00:10:26
sure that everything is proper
and in working order, then it
00:10:31
can be a benefit for good, moving
forward. - Absolutely.
00:10:35
Right. I think if all we
ended up is a world where we
00:10:38
were three years or four years
ago, we would have wasted a huge
00:10:42
opportunity.
- Where do you think
00:10:44
we're gonna go with
education, then, longer term, with
00:10:48
things like AI and ChatGPT in
the process?
00:10:51
So, I think it's going to
increase access, first of all.
00:10:54
There are many people living in
the world— that goes back to our
00:10:56
MOOC discussion, when we started
the MOOCs— so many people in the
00:10:59
world who don't have access to
an education like here at
00:11:02
Wharton. And we can basically
share and open the gates to
00:11:06
those people. I think that's a
good thing. That is all about
00:11:09
making things more efficient.
For the people who are at
00:11:11
Wharton, I hope we can do a
better job educating them. We
00:11:14
can create some really awesome
learning experiences. I know
00:11:17
Ethan Mollick and the folks
building simulations are already
00:11:20
actively working on that. We can
create real kind of increases in
00:11:24
the quality of education, just
like the person, the
00:11:27
barbershop is providing better
looks to their clients.
00:11:30
Christian, great to talk with
you as always. Thanks very much
00:11:33
for your time.
00:11:33
Thanks for having me here.
00:11:34
You got it. Christian Terweisch,
professor and Co-Director of the
00:11:37
Mack Institute for Innovation
Management here at the Wharton
00:11:40
School. - Thank you for listening
to The Ripple Effect. We hope
00:11:43
you found this episode
informative and engaging. Don't
00:11:46
forget to subscribe and leave us
a review so that we can continue
00:11:50
to bring you the best insight
from the Wharton School.
