00:00:00
welcome welcome to this edition of the AI at Wharton and analytics at Wharton podcast series on artificial
00:00:06
intelligence today's episode will actually have a dual role while it says Ai and education here our guest actually
00:00:12
has lots of expertise in Ai and education and the workforce and a lot more General topics as well I'm joined
00:00:18
by my faculty colleague uh Ethan mik Ethan is the Ralph J Roberts distinguished faculty scholar he's an
00:00:24
associate professor in our Management Department he's also the academic director of Wharton interactive so Ethan
00:00:30
Welcome to our podcast series I'm thrilled to be here thank you well I don't even know where to start because
00:00:36
I'll just say and I this is my first question to you most of what I've learned on AI and education will start
00:00:43
there is by watching the five-part series that you and your wife created um so could you tell our listeners here on
00:00:49
our Ai and education and Workforce uh episode what was in those five episodes like what do all of us as professors
00:00:57
need to know about Ai and education well I mean there's at least three different
00:01:01
things that matter right the first thing that matters is disruption uh homework is over right there's not a homework
00:01:07
assignment basically anywhere that a well- prompted AI can solve at this point so that's a big deal and just to
00:01:12
be clear let me take them one at a time there are explain to our listeners there
00:01:16
are multiple versions of even let's even say chat GPT which is just one of the
00:01:21
open AI sources like some versions can ingest documents some versions cannot some have just a text prompt so which
00:01:29
version or versions are you referring to when you say kind of home homework as we
00:01:34
know it is over okay so when you think about AI you want to think about um sort of all the what's called Foundation
00:01:40
models which are llama and Chachi all these kind of different models but you also want to think about what are called
00:01:45
Frontier models not to create more confusing vocabulary but there's really only three Frontier models right now
00:01:51
which are open AI chat gp4 uh which is the paid version but you can also get it for free through Microsoft Bing in
00:01:58
creative mode which turns out to be really important for education for reasons we'll talk about which is the
00:02:01
way I do right now and it's a little Limited in some ways it's weird it's has
00:02:05
a personality we can talk about that then there is um Google's Bard which right now is powered by a underpowered
00:02:10
model called pal 2 but all the rumors are that it will be upgraded to a model that probably will be the first model to
00:02:15
beat gp4 in the next couple of months and then finally there's a company called anthropic that has a product
00:02:21
called Claude 2 so when we talk when I talk about AI can do something I'm almost always talking about the frontier
00:02:26
models so gp4 um currently has like a separate mode for vision and pictures that's all
00:02:31
being united there's already been they're already rolling that out so they'll be able to take in documents
00:02:35
take in images read PDFs it already can it just does it a little bit uh jenily right now well let me ask you a few
00:02:40
things um well all of this right now there are paid and unpaid versions what's your vision since you also teach
00:02:47
Innovation you teach entrepreneurship are all these things going to stay free if they are what's
00:02:53
their revenue model is it adverti like how do you see this playing out from just a from our point of view and from
00:02:59
the company point of view so right now open aai is has announced that they're
00:03:04
on a run rate of $1.2 billion in revenue for after less than a year after releasing chat GPT most of that money
00:03:09
probably comes through their use of their API which is their their um businessto Business Solution right uh
00:03:15
less of it is the $20 a month we pay right if you pay for gbd plus which by the way if you can you should um there's
00:03:22
is almost the difference between gbd4 and gbt 3.5 the paid version and the free version is so large as it may not
00:03:28
appear that way at first but it's big enough that it is 100% worth it um Microsoft releases a bunch of gp4
00:03:35
products through Bing and they're planning on doing it free as far as I know for the near future Google bard is
00:03:39
also releasing theirs for free both because it's part of the search engine fight going on so we are the benefits
00:03:45
but it also means that people in 169 countries around the world have access to gp4 and will probably have barred
00:03:50
access as well which means the same model that you get access to if you go to Goldman Sachs or you go to you know
00:03:55
McKenzie or you go to Nike it's no better than the model that every kid in ug Gand and Sri Lanka has access to
00:04:01
which I think is really exciting and interesting also really a big deal for Education well you've brought me five
00:04:06
questions but let me go one at a time so I'm getting excited here um obviously
00:04:10
you spent time thinking about Wharton interactive so this idea of democratizing Education through
00:04:16
something like this has to be really thrilling and exciting to you as an educator and a scholar because I know
00:04:22
that was part of your mission and still is with Wharton interactive yeah so for those who don't know Wharton interactive
00:04:27
is our attempt to build games and simulations to teach entrepreneurship at scale and Wharton's been incredibly
00:04:32
supportive we have you've been incredibly supportive we had we've have built these very large games we have a
00:04:36
team of people we have writers and coders and you know interactive fiction experts and you know once gbd4 came out
00:04:42
we just tried the little experiment we tried saying like what if we just write a paragraph create a simulation of a
00:04:48
negotiation give me grading on it make it realistic and 80% of the way there with a paragraph like gbd4 just runs a
00:04:54
simulation so we have pivoted now all of our simulations are basically AIS powering every we have ai watching ing
00:04:59
Ai and AIS that are instructors and AIS that are mentors all interacting with each other that are actually doing
00:05:04
teaching right so so is it writing the code doesn't it doesn't even need to
00:05:08
write the code like it is writing the code but that turns out to be secondary it's writing the code it's creating the
00:05:12
images it's doing all that stuff but what it really is is also the brain's
00:05:15
the operation if we do good prompting we can tell it here's your goal make sure
00:05:19
that you're keeping students engaged change Tone If you need to here's your
00:05:23
overall and it just does it all I'm just in shock because wow so let me ask you a
00:05:30
question how does one become I know this is part of your video series how does one be I don't know if called an expert
00:05:36
but how does one become sophisticated is a good word in prompt engineering like could I do this be do I need subject
00:05:43
matter expertise to create specific enough prompts or is it just by as you and your wife talked about in the video
00:05:51
series you'll just learn by doing yeah so it's a really good question and sort
00:05:56
of all of the above so a few things one is for those who don't know prompt engineering right is the idea of writing
00:06:01
really good prompts to AI it is going to go away uh there's not a I talk to open
00:06:04
AI regularly talk to Microsoft talk to Google nobody who's insiders thinks this
00:06:08
going to last because the AI is really good at intent if you say I want to write a novel fairly soon it'll just be
00:06:13
able to say like okay here's let's go through the steps together it already
00:06:15
kind of does that right so it's you can get 80% of the way there by just interacting with the AI now there is an
00:06:21
exception if you want to encode your expertise as a subject maner expert you want this to do a really good you know
00:06:25
marketing analysis it will do a perfectly fine job but if you en code your expertise into it by saying here's
00:06:31
the angle you should take here's the approach do a little bit of prompt engineering you could give that prompt
00:06:35
to anyone and they'll get the benefits almost of your experience or wisdom so
00:06:38
there is some value in doing that uh it's pretty straightforward most of it's
00:06:42
using it 10 hours of the frontier model is my minimum rule of thumb but then beyond that piece there's a couple
00:06:48
simple tricks so one is you tell the AI who it is you give it context so the more context the better you are an
00:06:53
expert marketer and weirdly by the way there is a research now suggesting that when you tell the AI is an expert under
00:06:57
some circumstances it works better um again it's there's a lot of strange
00:07:01
stuff about prompting the second thing you want to do is provide a lot it's called fuse shot you want to provide a
00:07:05
lot of examples so when you're sending that you want to say here's some examples of the kind of report that
00:07:09
you'll produce and the third thing you want to do is have it do step-by-step
00:07:13
thinking so you want to say first do this then do that then do this because it kind of only knows what it writes so
00:07:18
you want to write stuff out and then go back to it and then build a plan from there those three things will make you a
00:07:23
better prompt engineer but it's not going to be that important in the long term all right so let's go back to the
00:07:28
focus of AI and education so let's talk about the roles that we have as Educators you already said the
00:07:34
traditional way of doing homework let's start with homework is in Jeopardy so
00:07:38
given that what can we do or in my view I've already I'm teaching next semester
00:07:44
I'm like use chat GPT as a matter of fact if you know how to do this to solve
00:07:49
the problems I'm asking you to do that's a skill set or should I be thinking
00:07:53
about this differently then I want to go to standing in the classroom then I want
00:07:56
to go to other forms of assessment and things that we do yeah well so the problem is what I feel is everybody's
00:08:03
rushing to do a chat GPD class right and then the answer is like yes chat can do
00:08:06
that I mean there are very few things that um at a sort of medium level that chat like at the 80th percentile that
00:08:13
chat doesn't do reasonably well right now so the question is do we want all of
00:08:16
our classes to be can you use chat CP to solve this problem I think we still want
00:08:20
to teach the subjects that are really good at teaching we still think people need to learn these things which means
00:08:24
we have to adjust right uh you can't use by the way everybody should know do not
00:08:28
use any kind of um AI detector they do not work they're biased against people
00:08:33
use English and second language they all whole five you know that that ship is sailed we cannot detect AI okay so just
00:08:38
to be clear just like when we we have canvas at the University of Pennsylvania there's a turn it in which is so
00:08:45
whatever version of that for AI you might as well just forget it no it doesn't mean but well this isn't look I
00:08:51
think you can see where my next question is going let's say Eric bradow and Ethan
00:08:54
mik are both in a class and they both use chat GPT for to solve some problem and let's say by chance they happen to
00:09:02
both put in the same prompt will they get the same exact text back and if the answer is no if both of those were
00:09:10
turned in could canvases turn it in not say it was AI generated but would it say
00:09:15
hey wait a second there's cheating going on there because their responses are so
00:09:18
similar so really good question a few things first of all they wouldn't get the same answer because there's
00:09:22
Randomness built into this a temperature right so there's a r there's a random
00:09:26
seed initially and then the words are a little bit randomly different which IES over time you're saying large language
00:09:31
models are prob I'm a statistician are are they're probabilistic models which
00:09:35
means even if we put in the exact same prompt we're going to get out things because there's a probability of the
00:09:41
next word or the next phrase and they're autoaggressive so once they head into
00:09:44
one direction or another they sort of spin off in that direction further very interesting so that's the first thing
00:09:49
right the second so the the second thing is I have had an assignment even before
00:09:53
chat GPD came out using gpt3 where I had my students cheat in class so I had them
00:09:57
write the best essay they could part part of the assignment is you have to prompt it at least five times by the
00:10:01
time youve prompted AI two or three or four times there's no way that they seem
00:10:04
similar anymore right if you give it so yes if people just pasted in the question right they're not going to be
00:10:10
the same answer but they might have some similarities if they do any work like make this more Vivid or here's my
00:10:15
writing style that's all they need to make it very different turn in will not
00:10:20
detect those things and by the way I really think it's unethical to use turn
00:10:23
in right now you should be turning it off right like it is it is it it will falsely has a high f accusation rate and
00:10:29
also by the way even worse is to ask GPD 4 or chat GPD free the 3.5 free version
00:10:34
whether cly was created by AI um new studies showed GPD 4 has a 95% rate of just telling you that something's made
00:10:41
by AI if you paste it in and ask if it's made by Ai and GPD 3.5 has like a 5%
00:10:45
rate of telling you they just randomly say this is made by AI or not they have no way of telling so what kind of things
00:10:51
can be you know what when you presented to the Wharton faculty which was one of the best most informative presentations
00:10:58
I've seen the a long time when you presented to the Wharton faculty I was like okay maybe at the time this was
00:11:04
true like all right so maybe it's not text data but maybe I'll give exam
00:11:09
questions that have video or maybe I'll give stuff that has voice because you
00:11:13
know what chat GPT can't possibly do as well with that am I off base or well let
00:11:19
me just say I might be correct but it's better than you think no it's it's
00:11:24
better at voice than humans are right now so whisper which is the free uh G built in the chat GPT app uh probably
00:11:32
illegally trained not illegally I don't I don't know who's watching but trained
00:11:35
on uh on YouTube videos probably as far as we can tell has better than human hearing so like accents um you know
00:11:42
mixes of languages I use it all the time when actually my students pitch to it um
00:11:46
I have real venture capitalist and then I have the AI playing a VC the VCS think
00:11:50
that the AI does a better job than they do in giving feedback um so yes it can listen um they and now can see things so
00:11:56
any visual problem you just upload and it will address a video or anything so video still has a little bit of trouble
00:12:01
with so that you can just pull off video right now um but give it give it give it
00:12:06
a few days no I mean a couple months probably well you actually wrote up another topic and we'll we've been
00:12:11
talking about Ai and education again I'm joined by my friend and colleague Ethan
00:12:15
mik he's the Ralph J Roberts distinguished faculty scholar a professor in our Management Department
00:12:19
also the academic director of Wharton interactive and clearly I think it's fair to say one of the leading Scholars
00:12:24
on artificial intelligence both in education and the workforce um could you talk to us about the work that you've
00:12:30
done on AI and the workforce cuz I know you're extremely proud of the work you're doing and I just want to see
00:12:36
because a lot of our you know uh experts have talked about it's can't be AI or
00:12:41
humans it's got to be Ai and humans I'm just interested in the angle in which
00:12:44
you've studied AI in the workforce yeah s okay so um one example we have a I
00:12:49
have a paper with um a whole bunch of great people at Harvard including kareim Lani uh frbo dequa um and people at MIT
00:12:57
Kate Kate Kellogg a whole bunch of people on this project but what we did was we went to BCG right one of the
00:13:02
three big Elite Consulting companies um and a lot of our Boston Consulting Group
00:13:06
a lot of our students want of work there a lot of alumni work there and we did an
00:13:09
experiment we uh created 20 tasks they were all realistic tasks with BCG they're actual tasks they use uh and we
00:13:16
gave some eight we used 8% of the global Workforce which is a lot and Al So when
00:13:20
you say an experiment you mean an actual experiment I mean an actual experiment 8% of their Global Workforce and some of
00:13:24
them got the help of gbt 4 and some did not and there were a bunch of other conditions the people who were given GPT
00:13:29
40 use in business house had a 40% Improvement in quality no training no specialization in the mod just the same
00:13:35
chat GPT all of us have access to 40% increase in quality across 108 regressions how was quality measured
00:13:42
every way we could so we did okay analytical tasks and marketing task and persuasion T and all of them were graded
00:13:47
by human phds human NBAs and then we also used gp4 which by the way grades just as well as humans do it just is a
00:13:53
little nicer on the scores but the relative scores were exactly the same and then they completed tasks 26% faster
00:13:58
got 12 point Sorry 26% more tasks done 12.5% faster no training nothing training like we only had like 5 minutes
00:14:05
of training for sub conditions others for none just to put that in context when steam power was put into a factory
00:14:11
in the early 1800s and improved performance by 18 to 22% we've never seen a 40% Improvement this is not tuned
00:14:17
this is not train this is the chat interface that you're used to using um so huge huge performance impacts on on
00:14:24
from just a little experiment all right so usually in academic papers we have some thesis hypothesis and in your case
00:14:29
you have an experiment in results what did the back end of that paper look like so let's imagine you're now Consulting
00:14:36
for a company BCG or you're Consulting to our students undergrads mbas like this should be how you think about your
00:14:44
training what did the back end of the paper like what conclusions did you come to as a result of this well we barely
00:14:51
there's so much else we could talk about here too that are you know interesting
00:14:53
caveats on creativity and who who uses what answers but let let's and and also
00:14:58
how people people work with AI right that's another thing I've been doing a
00:15:00
lot of work on uh effectively but um I mean the back end of the paper is really the idea that like look this is a big
00:15:07
enough impact that this should be a red alert everywhere in every organization right you don't see these kind of
00:15:12
performance improvements a lot of people are taking their time on AI they're
00:15:15
assisting that they do something like you know integrate their own data with the AI system we didn't have to do that
00:15:20
here like the AI the p and GPT stands for pre-trained it knows a lot of stuff already it's not clear that you should
00:15:26
be waiting to build a large data integration use r and all these other techniques when you should just probably
00:15:30
be using this and it should be a red alert to figure out how to use this because you know as much as we say oh
00:15:36
it's people using AI but also I mean another side of this paper was there's
00:15:40
appendix C to the paper that I don't always talk about because it's I don't
00:15:43
quite know what to do with it but it measures uh what's called retainment how
00:15:46
much of gp4 is answer that you just use as your answer and there's almost a direct correlation between how much of
00:15:51
the answer you use and how successful your results are I mean there's a direct
00:15:54
correlation it's almost perfect correlation so basically the only way to mess up was to changed chat gpt's answer
00:16:00
H and not only that the performance boost was at the largest for everyone in the bottom half of performance so we
00:16:06
measured prior and after performance 42% boost and improvement from the bottom half 18% for the top half performance it
00:16:12
leveled everybody up to like the 8th percentile of BCG Consultants like I don't even know what to do with that
00:16:17
that's such a big number so given as you said this is more impactful than the
00:16:22
steam engine what what did BCG now do with this like what's their planned I mean is there any let me just say Do
00:16:29
they have any doubt that what you found is generalizable like maybe here's an
00:16:34
argument I'll be a statistician for a moment maybe this wasn't Al you said 8%
00:16:38
of the workforce maybe this wasn't a massive sample size maybe it works for these 20 tasks but not for these tasks
00:16:44
or maybe there's some sort of you know maybe it helps in the short run but you
00:16:49
know what we can also train humans so maybe the effectiveness is going to decrease over time I'm just playing a
00:16:54
like if I were a reviewer on a paper I'm just playing The Devil's Advocate what
00:16:57
would be the respon to all of this so a few things there we did manage to create
00:17:01
one task the AI couldn't do and right so one of the things to know about our
00:17:05
listeners here how want to hear what that is so well it was hard right it was a task where we had to hide data in
00:17:10
interviews and some was in spreadsheets and this was before Ada the Advanced Data analysis module came out but we
00:17:15
managed to find something right um and it took some work and then on that task you what happened was people who used AI
00:17:21
did worse because they were mistaken because they took in what they was do so there so part of what people are saying
00:17:26
is like well what's the border of what AI does and doesn't we call that the
00:17:29
jagged Frontier like for example if you ask gbd4 to write a 25-word paragraph it'll have trouble doing that because it
00:17:35
doesn't see words it sees tokens but if you ask it to write a sonnet it'll do an
00:17:38
amazing job of that sonnets are harder for humans than 25 words you have to learn the frontiers of AI going back to
00:17:44
the point we mentioned earlier if you use it a lot that's how you start to understand it's going to be good at this
00:17:48
task bad another task so to go back to the overall kind of question right about what do you do with this right is it
00:17:53
generalizable this is just one piece of result there's another study out MIT
00:17:57
that got published in science that shows similar size improvements in business writing tasks in a completely different
00:18:02
sample there's a study um out of GitHub showing the same kind of improvement for
00:18:06
programmers like the 30 to 70% number just keeps coming up over and over again in different samples and different
00:18:13
there's a piece on Creative work there's another paper U out of Harvard looking
00:18:17
at um at you know business implications uh sorry business um proposal writing there's our own colleague Christian turt
00:18:24
Carl erck and and their colleagues work showing Innovation so this is not like a
00:18:28
onetime thing you know this is a pretty broad-based set of findings so where do you think the I I like the word Jagged
00:18:35
Edge yeah where is Jagged Frontier where is the jagged Frontier here I mean do you even I mean you probably know more
00:18:42
than let's say you're in the one 1000 upper percentile of people in knowledge
00:18:46
about GPT or AI in general right now where do you think that Jagged Frontier is so it's hard to explain right which
00:18:53
is part of why this is as much doesn't it always move out that is that's the
00:18:57
main thing is the frer is moving out like I guarantee in the next month the frontier is going to move out right like
00:19:03
some for some reasons I know I can't talk about some reasons that are already
00:19:05
public elsewhere but like this is not stopping there's no indication to me that the jagged Frontier is not going to
00:19:11
keep moving forward in the next few months right next year or two and I think ultimately the big question is how
00:19:17
far how fast and when does it stop and I will tell you the people training the AI
00:19:20
models I don't think have an answer to that question so let's start with a few
00:19:23
things um how much I I saw I think it was yesterday President Biden sign signed a bill or something an executive
00:19:30
order on artificial intelligence and some sort of safety security protection could you tell our listeners what is
00:19:37
that about like what is what is the policy trying to do so there there's a few things in the policy right what I I
00:19:44
I I have not spent a huge amount of time in the executive order but what it does
00:19:47
that I like about is there's there's sort of two stages of threats from AI
00:19:50
that people are worried about one is you may have hear if you read the press a lot you may hear about Extinction risk
00:19:55
right what if we make AGI artificial intelligence smarter than a human and what does it do to us does it save us
00:20:00
does it kill us what happens if we build a machine God and by the way that's like
00:20:02
the stated goal of open a AI is to build AGI right that's their plan by the way
00:20:07
just so I know I I just want to be sure since I'm a big movie guy I don't know
00:20:10
how much you watch movies like wasn't that some part of Terminator like in other words these robots became so smart
00:20:17
that in some sense they ended up launching Wars and I mean the schwarzeneger movie ter that is it's not
00:20:23
unrelated right no that that is one example of AGI right um the people who are in favor of AGI think that this will
00:20:30
save us all and you know and redeem humanity and give us all eternal life the people who uh don't like it think
00:20:35
it'll murder us all so I think it's worth worrying about like it's something
00:20:38
that enough serious people in computer science are worried about that we should I'm glad that we're addressing that but
00:20:42
I think the bigger policy implications in the near term for me as somebody at the Wharton School is look we've got
00:20:48
something that's doing high-end creative work high-end managerial work it's going
00:20:52
to get better at this like we're improvising a chat interface and using that to do Consulting work like that's
00:20:57
pretty crazy so that means there's going to be widespread implications for work
00:21:00
widespread implications for Education as we've were talking about and also one of
00:21:04
the things that really is important like the part of the reason we catch criminals and like Bad actors so easily
00:21:09
especially when they're not connected to like a state intelligence agency is most
00:21:11
of them aren't that great right now what happens if the AI makes brings everybody
00:21:15
up to 80th percentile in biological engineering 80th percentile in building chemical weapons that's also a concern
00:21:22
and then there's privacy concerns deep fakes are perfect from this thing right
00:21:24
like so can you define what a deep fake is like what does that term mean sure I mean AI um content is basically
00:21:31
undetectable I have made videos of my fake me talking I I my presentations I always have one real picture everything
00:21:36
else I generate on my own no one could tell what the real picture is right so you cannot tell if I can create an actor
00:21:42
with their voice I can do this right now for $150 with software that anyone can use it's not even like dark web software
00:21:47
it's a company that's VC backed uh and I can use 11 labs in did and create a fake
00:21:52
video of you talking right now and you know it'd be pretty realistic so we have
00:21:56
this issue with this kind of deep fake email it's also perfect fishing engine
00:22:01
like you shouldn't trust anything you see online anymore and that's not a joke
00:22:03
like there literally is no way to know like I've already talked to Banks who've
00:22:06
gotten calls uh in from the voice of people who weren't actually calling them
00:22:09
demanding money from ransoms like stuff is going crazy already that is ship is already staled so part of the Biden
00:22:15
agenda is like how do we Watermark these things not going to be possible like it's just that's that's not going to
00:22:20
happen really because even though these Frontier models are based in the US there's a whole bunch of Open Source
00:22:24
models worldwide models that are not going to have this kind of protection so the the the attempt of this executive
00:22:30
order for the way I see it is both to do worrying about this sort of future Ai and training it but also trying to think
00:22:35
about how do we restore privacy how do we restore you know and I don't know how
00:22:39
much of that's possible but the regulation is probably needed so a lot of our listeners are probably sitting
00:22:44
here saying I need to get started like I I I haven't even started but now clearly
00:22:49
if I'm listening to Professor mik I've got to start now where do you suggest
00:22:53
that someone starts like you mentioned 10 hours let's say that's good what what
00:22:58
should they spend their 10 hours doing like where should they go you know we're
00:23:02
at Wharton so we're fortunate we have the video series you sent around but I
00:23:06
mean whether it's you have materials others have materials can you go I'm
00:23:09
making it up can you go to KH Academy I I don't know where did where does someone go to get started so a few
00:23:14
things those videos are on YouTube anyone can see them if you search the name of ethol you'll find it I also have
00:23:19
a substack with a whole bunch of getting started guides called one useful thing all which I just which I just started
00:23:24
following today excellent hopefully you'll enjoy that but I mean I think the
00:23:27
B BS of this are my my principles of AI my first principle of using AI is just invited to everything you morally and
00:23:33
legally can use it for everything use it for your job like literally you want to
00:23:36
send an email see how good the AI is writing the email you have to do ideation you know have it do ideation
00:23:41
you going into a meeting bring it to the meeting have it record the meeting and give you advice and feedback on what you
00:23:45
should do better next time just use it for everything that's the only way to
00:23:48
figure out what the jagged Frontier is in your field there nobody knows anything right now right like as I said
00:23:53
I talk to all of the major AI companies on a regular basis and like no one has an instruction manual for this thing no
00:23:58
one knows whether it's going to be good or bad in your subfield whoever you're
00:24:01
listening at right now so you can be the world expert by just using it and seeing
00:24:05
what that is so just try it for your work and then there's a bunch of techniques you'll start to learn but the
00:24:09
first thing is to try it it's really important though to use a Frontier Model
00:24:12
to use the most advanced model available to you right now now you mentioned something else about you think that I
00:24:17
think let me see if I got this right that I think it was Google you I think you use the term like they're coming out
00:24:23
with something new that might be better than chat GPT what would better mean in like when you use the word better I was
00:24:31
just intrigued by that what does it mean to be better than one you know uh large
00:24:36
language model Etc being better than another okay so there's a lot of interesting angles to that right now all
00:24:41
the major front the two major Frontier models which are open ai's model which
00:24:45
sort of Microsoft Powers it as well and Google's model both are have added a
00:24:50
whole bunch of capabilities that if you're not paying attention you may have
00:24:52
missed so they're all fully multimodal you can ask them to create pictures they
00:24:56
they also can see the world right so like not doing an image search but like you literally could show a picture and
00:25:00
say how does this dress fit and it will give you reasonable feedback on that or how do you know what how do I undo this
00:25:05
lock or what's this passcode whatever you want to do um so they're all multimodal they all are going to be
00:25:10
doing voice back and forth they all connect to other do can read documents connect to other materials so that's
00:25:15
kind of the basics all of that is connected to the large language model itself which you can kind of think of as
00:25:19
the brain and so large language models basically get smarter over time so if we think about uh GPT 3.5 the free version
00:25:25
you're using maybe High School sophomore I would say GPD 4 at its best moments is
00:25:30
is a first year grad student so part of the question yeah so part of the question is what is two or four times
00:25:35
better than that look like we don't know yet so we could just do raw test scores
00:25:38
right we go from scoring at the you know at the 5ifth perc of the bar exam beating 5% of humans for free chat GPD
00:25:45
to beating 95% of humans for GPD 4 what happens with the next one we we don't
00:25:50
really know but so smarts are sort of the smarts of the brains behind the whole thing so this is the question I've
00:25:55
asked everyone in this series uh to kind of end the episode if we're sitting here
00:26:00
10 years from now and we'll make a date I'm going to interview this I hope
00:26:03
interview the same group of people 10 years from now what are we talking about do you think that has happened over the
00:26:09
previous 10 years I can only think it's scenarios at this point right I mean
00:26:13
because the the only question that kind of matters for this is how fast will these models improve and when will they
00:26:20
hit their limits right and nobody knows the answer to those questions right so it's not going to be static if you think
00:26:25
you have time to wait you don't because these model are advancing very rapidly
00:26:29
the question is do they stop at the 95th percentile of the best humans in one area like so everybody's got something
00:26:34
they're really good at that they definitely beat the AI in 99th percentile better than human we don't
00:26:39
know right and so to me that's the only relevant question right and so nobody
00:26:44
has the answer to that so we have to prepare for a scenario where okay we're
00:26:48
getting close to the top I don't I haven't seen evidence of this but it's
00:26:50
entirely possible that it starts to slow down then we still have at least 10 or 15 years of absorbing what gbd4 can do
00:26:56
because it's barely connected to anything anything in the world right we've got 10 years of disruption ahead
00:27:00
of us that's going to roll ahead anyway if they keep getting better then we start to think seriously about if not
00:27:05
AGI what does it mean that it beats every human at you know writing marketing copy like what do we do with
00:27:10
that right do we you know and so there's a lot of open questions so I don't have
00:27:14
the easy answers but I think that there is I think we're you're more likely to
00:27:17
see a transformed World in 10 years than in five even if the technology stops because it takes a while for systems to
00:27:22
absorb change right the the the futurist rule is that everyone overestimates short-term change it underestimates
00:27:28
long-term change I think 10 years we're going to see a transformed World in a
00:27:31
lot of ways that are some are good some are bad well I'd like to thank uh Professor Ethan mik for joining me today
00:27:37
on the podcast series on AI in education and the workforce uh Ethan's an associate professor of management he's
00:27:42
also an academic director of whorton interactive and as he said you can go to your favorite engine and type in Ethan
00:27:47
mik and you can see about his substack and about his videos on YouTube Ethan thank you for joining me thank you for
00:27:52
having me