00:00:00
Dan Loney: Well, Artificial Intelligence is going to be one of
the areas to focus on
00:00:03
as we head back into the new
school year. All levels of education are taking a look at
the potential of AI, and what it
00:00:12
could bring to education. But
there are also some areas of potential concern. - Welcome to
The Ripple Effect, the podcast
00:00:20
that takes you on a journey
through the minds of Wharton faculty. I'm your host, Dan
Loney. And in each episode,
00:00:26
we'll be diving deep into the
inspiration behind the groundbreaking research that
Wharton professors have
00:00:31
conducted, and exploring how
their findings resonate with the world today. - Pleasure to be
joined here in studio by
00:00:37
Christian Terweisch, who is
professor and Co-Director of the Mack Institute for Innovation
Management here at the Wharton
00:00:44
School. Great to see you again. Christian Terweisch: Thanks for having me. Thank you. And so the
interesting thing is, this is
00:00:49
something that really came into
your mindset in your teaching here at Wharton. You started to
see AI as a component that you
00:00:56
needed to focus on. Yeah, absolutely. So by way of
background, many, many years ago, I have a background in
computer science. Even had a
00:01:04
class on neural network. But
really put this topic to the side till Thanksgiving about two
years ago. My kids were home
00:01:11
from college and their work. And
like many other families in this country, we talked about ChatGPT.
And so the topic came up
00:01:18
like, "Hey, Dad, you teach at a
business school. Can ChatGPT take your final exam?" And so
what we did is we just took my
00:01:25
final exam in Operations
Management for the MBA students, put it into ChatGPT, and
it did pretty, pretty well. And
00:01:32
so we put the story online, it
went viral, and ever since, people believe I know something
about AI.
00:01:38
But did it surprise you
that ChatGPT was able to do well on a Wharton exam? Absolutely. Not just that it
did well. The way it answered
00:01:48
the question, the level of
sophistication of the language, the persuasiveness, the
smoothness of the writing. It
00:01:55
just blew my mind.
- So how has that changed your thought
process in how you teach, how
00:02:01
you give exams? What— how has it
impacted your operation? Well, the first reaction is,
"Wow, this is big, right? This is
00:02:08
a major productivity boost."
Before we talk more about teaching, let's digress a little
bit in what it means to have a
00:02:13
productivity boost, right? We're
going to see productivity boosts in many industries. For doctors,
for nurses, for lawyers, for
00:02:20
accountants, for radio hosts, for
business school teachers, right? We're all gonna get more
productive. But what does it
00:02:26
mean to be more productive? So
imagine— let's take a barbershop, right? I have a
technology that makes the barber
00:02:32
twice as productive. What does
it mean? That barber can either make haircuts that are
twice as good, or they can take
00:02:38
twice as many customers and get
a haircut? Right? So I'm shifting what we call the cost-
quality frontier, because
00:02:44
quality frontier has shifted
out. And I can do now more with less. So what does it mean for
teaching? Well, I can either
00:02:51
teach more students, or I can
hold the number of students I'm teaching constant and provide a
better job. - What
00:02:56
do you think it's going to mean
most for students, in terms of how they go through their
education process moving forward?
00:03:03
I think you have to distinguish
between two types of use cases, right? There's a use case for
the students who get the same
00:03:09
education as before, they could
just get it in a more efficient and thus cheaper way.
00:03:15
I remember many
years ago, you and I talked about massive open online
learning. - Yeah. Yeah.
00:03:19
I taught one of the first MOOCs
here at Wharton. So that was all about
taking the
00:03:23
Wharton experience and making it
cheaper. Now, the other use case, and that's one I think we
should be more bullish about, is
00:03:30
we do a better job, right? The
haircut looks twice as good. So we have to ask ourselves in
education, what does it mean
00:03:36
to provide education that is
twice as exciting for the same number of students? Is there a level of concern
about students maybe not having
00:03:44
the level of learning on specific
topics because they're relying on the component of AI to help
them through the process?
00:03:52
Absolutely. Right. So there's a
concern, just take a calculator as an example, when the
calculator came out, when spell
00:03:57
checks came out, we were
concerned— and I think rightfully concerned— that
students would forget how to do
00:04:03
basic calculations, and that
they would weaken on their spelling. It's like if you take
a pilot, and the pilot is flying
00:04:10
autopilot all the time, is the
pilot at risk of forgetting how to fly an airplane? Probably.
00:04:16
The question of, though, the
different levels of education and how they're going to be
impacted— we're already seeing
00:04:22
those types of discussions form
at this point. Colleges are obviously thinking about it.
Even high schools and middle
00:04:29
schools are thinking about how
AI can either be a tool to benefit the students' learning,
but at the same time, being
00:04:37
careful of the component of
exams and tests that they're providing to the school as well.
- Yeah,
00:04:43
so let's stay with the exams for
just a second. Again, you mentioned colleges and K12. I
think there the impact is
00:04:49
going to be very similar. The
distinction that matters is, unfortunately, the rich and the
poor, right? There are
00:04:54
privileged universities and
privileged private schools, Penn Wharton being one of them.
And there, the question is
00:05:01
all about, like, how can we take
the students that we teach anyway, and provide them a
better education? Now, I'm
00:05:07
realistic about this. There are
many situations in education where money is scarce. Today,
think about public schooling in
00:05:13
poorer areas. And there, the
focus will be for— for the politicians, about how to
make education cheaper. Most of
00:05:20
them boost the frontier. That's
a good thing. But I wish there would be more people who would
benefit in the way that we could
00:05:25
benefit at Wharton.
- But obviously, just when you think about K
through 12, just having the
00:05:30
access to computers, and then
obviously having the access to AI or ChatGPT as an assistant in,
you know, doing research or
00:05:38
doing a paper, that's a
significant hindrance in many cases in different parts of the
country. - I think most
00:05:44
K12 teachers I've talked to
will have the first reaction like, "This is going to be
problematic." Right. And I think
00:05:49
it's— sure I mean, if you have,
basically, your brother-in-law taking the exam for you, I'm
sure that is hindering the
00:05:56
education process. In an
institution like ours, we're certifying a specific skill for
somebody. And if that
00:06:03
certification is now weakened
by the fact that we're not sure whether the person has the skill
or the technology has the skill,
00:06:09
we have a problem.
- But I guess we shouldn't be surprised
that that is somewhat the
00:06:13
reaction that we're seeing right
now, is because in many cases, when we see a new technology
come on the board with great
00:06:20
potential, there is a concern. There's a hesitancy
that's out there. And it should be, right? I
think— again, let's go back to
00:06:25
calculators. On calculators, you
might still remember how to do seven times seven. Let me just
get the math right. It's 49, if
00:06:33
I'm not mistaken. So you
remember that. I remember that. But somebody who grew up in the age
of the calculator, they might
00:06:39
say, like, "I don't have to know
seven times seven, I just ask the calculator." So you can now
make the same argument for
00:06:44
ChatGPT, where you say, like,
"I don't know how to write an English essay, but ChatGPT can
do it for me." So I think it
00:06:50
really boils down to the
question, like, what skills do we want to teach our kids? And what
is the learning outcome that I
00:06:56
want to have for my students
in my course? And there's also the component
of the development of the Chat
00:07:00
GPT itself, with the different
versions we've seen come out even in the short period of
time, and how in the technical
00:07:07
side can improve, you know,
what's available for use in and around ChatGPT. Yeah, so I started using 3.5,
version 3.5, about a year and a
00:07:16
half, two years ago, for my
initial exam. That did pretty well already. And then it was 4
coming out, and 4 Turbo. And now 4
00:07:24
Omnichannel. It just got better
and better. What are the discussions you've
had with your students about the
00:07:30
use of ChatGPT? What are the
parameters and barriers that they can and cannot use?
- I'm
00:07:35
like a harsh German guy, right?
So the rule is very simple. You're not allowed to use
ChatGPT on the final exam.
00:07:41
So that's the first and most
important rule. The second one is, you can use it for case
preparation and discussion. I've
00:07:48
seen not too many students take
advantage of that. But I expect that number to be growing. So
I'm totally fine if you
00:07:55
take the case, upload into
ChatGPT and ask for advice, I think that's fair game. And then
in class, we're typically not
00:08:02
using it at all. I mean, I think
it works best when students prepare for the case, for an
assignment, in advance of class.
00:08:10
So then seemingly we're looking
at a time where we're going to go through a period where
leadership of the education
00:08:17
sector, whether it be colleges,
universities or school districts, are going to have to have long
conversations about what is
00:08:24
allowable and what is not allowable
around AI. - Yeah, I've wrote an Op Ed on the
question of what is allowable
00:08:30
and not. And I think it's helpful
to distinguish between three types of tests. The tests that
you do to certify a skill— and
00:08:36
for those, don't use ChatGPT.
Then the second thing is what I oftentimes do as a
teacher in class, I ask my
00:08:42
students questions to just get a
sense of how they progress on their learning journey. And
again, using ChatGPT would be
00:08:48
misleading. The third type of
test is really more like a homework that is hopefully
engaging the students in the
00:08:54
learning. And for that, if I can
use ChatGPT to increase that engagement, that would be a good
thing. But so always ask
00:09:01
yourself, why are you testing?
Tests are not for the sake of testing. As an educator, you
have to always think, like, Why
00:09:07
do I test?
- How much do you think ChatGPT has impacted
your profession right now?
00:09:15
I think we're just seeing the
implications for the academic profession. I use it for idea
generation oftentimes. So I've
00:09:23
wrote a paper that shows how
ChatGPT outperforms human beings at coming up with ideas for
new products or services. That
00:09:30
is something that somebody who
teaches innovation cares a lot about. I use it for writing.
When I'm stuck with something, I
00:09:36
say let ChatGPT continue that
page for me. I think the key thing that you have to realize
about ChatGPT is it still makes
00:09:43
mistakes. But there are
situations where mistakes are not consequential. So if I'm
asking you for 10 ideas, and
00:09:49
five of them are bad, I still
get five good ideas. If I asked you to do heart surgery for me
and five of those heart surgeries go
00:09:57
wrong, that would be a disaster.
So you want to use it in relatively low stakes
environments, where you have a
00:10:03
human in the loop so that you
make sure— make sure that the quality is ultimately where you
want it to be. Because
00:10:08
ultimately, it's your name on the
paper. And again, if you have that
understanding that things like
00:10:15
ChatGPT can be an assistant, be
a guide, in the process of doing a test exam, or whatever it is,
but still that you have to, you
00:10:24
know, do the double check, but
you know, fall back and make sure that everything is proper
and in working order, then it
00:10:31
can be a benefit for good, moving
forward. - Absolutely. Right. I think if all we
ended up is a world where we
00:10:38
were three years or four years
ago, we would have wasted a huge opportunity.
- Where do you think
00:10:44
we're gonna go with
education, then, longer term, with things like AI and ChatGPT in
the process?
00:10:51
So, I think it's going to
increase access, first of all. There are many people living in
the world— that goes back to our
00:10:56
MOOC discussion, when we started
the MOOCs— so many people in the world who don't have access to
an education like here at
00:11:02
Wharton. And we can basically
share and open the gates to those people. I think that's a
good thing. That is all about
00:11:09
making things more efficient.
For the people who are at Wharton, I hope we can do a
better job educating them. We
00:11:14
can create some really awesome
learning experiences. I know Ethan Mollick and the folks
building simulations are already
00:11:20
actively working on that. We can
create real kind of increases in the quality of education, just
like the person, the
00:11:27
barbershop is providing better
looks to their clients. Christian, great to talk with
you as always. Thanks very much
00:11:33
for your time. Thanks for having me here. You got it. Christian Terweisch,
professor and Co-Director of the
00:11:37
Mack Institute for Innovation
Management here at the Wharton School. - Thank you for listening
to The Ripple Effect. We hope
00:11:43
you found this episode
informative and engaging. Don't forget to subscribe and leave us
a review so that we can continue
00:11:50
to bring you the best insight
from the Wharton School.