00:00:01
[Music]
00:00:08
[Music]
00:00:20
uh dhan thank you so much for joining us
00:00:22
today thank you well how does a massive
00:00:26
open online course or a Muk as it is
00:00:28
called like those thought through
00:00:30
corsera differ from the traditional
00:00:32
educational experience and what are the
00:00:35
pros and cons so if by the traditional
00:00:38
education experience you mean the
00:00:40
face-to-face teaching um I think that
00:00:43
there are differences that um vary based
00:00:47
on what audience you're talking about so
00:00:49
these mukes as they're called have been
00:00:51
offered separately both to the general
00:00:54
public at large but also as a way of
00:00:56
transforming the educational experience
00:00:58
of oncampus students so in terms of the
00:01:00
students in the general public there is
00:01:03
this um tremendous opportunity for them
00:01:05
to be exposed to the kind of knowledge
00:01:08
that has until now only been available
00:01:09
to a tiny sliver of the world's
00:01:11
population and they don't get the same
00:01:13
experience necessarily as the on-campus
00:01:15
students because they don't get that you
00:01:17
know one-on-one connection with the
00:01:18
instructor but on the other hand um the
00:01:21
availability of the content is just a
00:01:23
tremendously enabling experience for
00:01:25
those students and there's still a
00:01:27
hugely interactive experience in terms
00:01:29
terms of um interaction with the
00:01:31
material which is not just video there's
00:01:33
also a lot of exercises and assessments
00:01:35
as well as this Rich community that is
00:01:38
based on um that is based around the
00:01:40
material where students interact with
00:01:41
each other in a kind of peer teaching
00:01:43
format at the same time students on
00:01:46
campus can benefit from that same
00:01:48
content and have the experience of going
00:01:52
and watching the content separately
00:01:54
outside the classroom and then coming
00:01:56
into class and having a lot more time
00:01:58
for an active learning interactive
00:02:00
experience with her instructor let just
00:02:02
just take a step back how would U corera
00:02:05
fit into the higher education landscape
00:02:08
uh does a Muk supplement uh traditional
00:02:11
learning as you just described or do you
00:02:14
see it as a substitute as well I think
00:02:16
it depends on the population again for
00:02:19
those students who are privileged enough
00:02:20
to be enrolled in an academic
00:02:22
institution such as Wharton or Princeton
00:02:24
or Stanford um there is uh a tremendous
00:02:28
opportunity to supplement the stand
00:02:30
instruction and then again repurpose the
00:02:33
classroom time for a much more
00:02:34
meaningful and interactive experience
00:02:37
for a lot of students out there however
00:02:39
they will never have access to this
00:02:41
quality of education and so you can view
00:02:44
this as a substitute for the education
00:02:46
that they would have liked to have um
00:02:49
for many students they that are taking
00:02:51
our courses they are in fact already
00:02:53
educated professionals they have a
00:02:54
degree sometimes they have even an
00:02:56
advanced degree but they would like to
00:02:58
expand their minds and learn new things
00:02:59
things and they will never go back to
00:03:01
school um whether for financial or
00:03:04
geographical constraints and this is a
00:03:05
tremendous opportunity for them to learn
00:03:07
something great so now I understand that
00:03:10
uh do 33 institutions offer muks through
00:03:13
corsera that's right uh and how many
00:03:15
students have they served So currently
00:03:18
enrolled on the platform we have over
00:03:20
1.5 million students um that uh are
00:03:24
currently enrolled in there's 5 million
00:03:26
distinct course enrollments because
00:03:28
students often enroll from more than one
00:03:30
class um a lot of these students have
00:03:32
not yet been served because a lot of our
00:03:34
close to 200 classes have not yet
00:03:36
started to run and so um uh you know I
00:03:40
think we've served hundreds of thousands
00:03:41
of students uh so far but you know not
00:03:44
the full 1.5 million that have been
00:03:46
enrolled and what have you learned so
00:03:48
far about um participation completion
00:03:52
Dropout rates of this online population
00:03:56
so one of the things that we see is that
00:03:58
you know because enrolling is so easy
00:04:00
it's a matter of just clicking a button
00:04:01
and it's free a lot of students do that
00:04:04
um and then when it comes time for the
00:04:05
course to actually begin about 70% of
00:04:08
them show up because maybe their life
00:04:10
has moved on and they're now busy doing
00:04:11
something else of the ones that show up
00:04:14
we see a bifurcation of the population
00:04:16
there's students who come in and they
00:04:19
really are there primarily to just get
00:04:21
the benefit of the content they're just
00:04:22
there to watch videos and of those who
00:04:25
start watching videos about 30 to 40%
00:04:28
continue watching the entire rest of the
00:04:30
course to its completion uh with a sort
00:04:33
of Fairly constant drop off rate week on
00:04:35
week as students get busy and their life
00:04:38
uh imposes some challenges on them
00:04:40
similarly there is the second population
00:04:42
uh those students who really do intend
00:04:44
to take the course for real and we kind
00:04:45
of gauge that right now by seeing who
00:04:48
submits the first assignment as an
00:04:49
indicator that if you submit the first
00:04:51
assignment you maybe really do intend to
00:04:53
take the class for real and again the
00:04:54
retention rate for that population is
00:04:58
comparable it's about 30% % of the
00:05:00
people who start the first assignment
00:05:02
complete the last one now in terms of
00:05:05
sort of overall what you might
00:05:07
traditionally call retention that is the
00:05:10
number of students who submit the last
00:05:12
assignment uh relative to the population
00:05:15
who initially enroll for the course that
00:05:17
translates into a completion of 7% or so
00:05:21
9% depends on the course but that's
00:05:22
really the wrong way of looking at it
00:05:24
because many of these students never
00:05:25
really intended to take the class in the
00:05:27
first place right not for real right so
00:05:29
for example I I understand that the
00:05:32
gamification course that was offered
00:05:34
through Wharton uh had about 70,000
00:05:37
people who enrolled initially but about
00:05:39
9,000 uh completed it now now that
00:05:42
that's quite a big gap what have you
00:05:44
learned about the different kinds of
00:05:46
courses where uh you've seen greater
00:05:49
success than others I don't view 9,000
00:05:52
out of 70,000 to be a uh to be a bad
00:05:55
statistic because I didn't say so yeah
00:05:58
it was just a description
00:06:00
criticism um you know I think we're
00:06:03
still starting I mean we don't have a
00:06:05
sufficiently large sample size right now
00:06:07
to gauge what um makes for success and
00:06:10
what doesn't and in fact one of the
00:06:12
things that we've started to do is
00:06:14
research that is intended specifically
00:06:16
to try and determine what increases
00:06:18
retention what increases engagement so
00:06:20
we're now running a test in which
00:06:23
students are getting encouraging emails
00:06:25
once a week saying oh you know you did
00:06:27
so well in the last couple of
00:06:29
assignments and the next one is due
00:06:31
tomorrow don't you think it's good if
00:06:32
you log in and try and complete the next
00:06:33
one you don't want to break your streak
00:06:35
and we're doing an AB test in the same
00:06:37
way that Google does AB test in in
00:06:39
search quality to see whether that
00:06:40
actually increases student engagement
00:06:42
and one of the really beautiful things
00:06:44
about having such a large po base of
00:06:47
population that you can run these
00:06:49
experiments and get statistically
00:06:50
significant results so that we can learn
00:06:52
how to improve the learning experience
00:06:54
for our students right now some of the
00:06:57
institutions that offer these courses
00:06:59
off for credit and some don't is is why
00:07:02
is the that difference exist well you
00:07:05
know as of now only one institution
00:07:08
offers credit uh that's the University
00:07:11
of Washington and they're offering it
00:07:12
for an enhanced version of the course
00:07:14
where students have uh interaction with
00:07:16
the instructor and and um you know some
00:07:19
additional homework besides what's
00:07:21
available on the corsera platform I
00:07:23
think that you know the granting of
00:07:26
credit is a very sensitive issue for um
00:07:29
institutions and they need to think
00:07:31
carefully about what makes a credit
00:07:32
bearing class and what doesn't and this
00:07:34
is such a new area and this whole
00:07:37
transformational change that we're
00:07:39
seeing is so new I mean a year ago today
00:07:43
approximately September or October of
00:07:45
2011 was when the first real mukes
00:07:48
launched and so so much has happened
00:07:51
since that we are still I think all of
00:07:54
us as a community are just trying to
00:07:56
assimilate the change and figure out
00:07:57
where we want to take this and I think
00:07:59
over the course of time more and more
00:08:00
institutions will move into the credit
00:08:03
arena with this but um it's too early I
00:08:05
think to have that step taken and do uh
00:08:08
any institutions offer certification
00:08:10
also so most of the institutions that
00:08:13
we're working with have agreed to offer
00:08:17
some kind of unofficial certification um
00:08:20
as I mentioned earlier most of our
00:08:22
population right now at least are people
00:08:25
who already have a degree and will never
00:08:28
go back to school to get another one so
00:08:29
for them the question of whether the
00:08:31
course carries credit is really of less
00:08:34
relevant compared to whether they have a
00:08:36
meaningful certificate that they can for
00:08:37
example present to an employer to
00:08:39
demonstrate competence uh in a
00:08:41
particular topic and so I'm I'm very
00:08:43
pleased that most of the institutions
00:08:45
have decided to offer that benefit to
00:08:47
students which allows them to really
00:08:48
gain something substantial for the work
00:08:51
that they did and is it corera that
00:08:53
offers the certification or the
00:08:54
institution and is there any charge for
00:08:56
it so um what what we have been able to
00:09:00
offer up to now is a what is often
00:09:04
called a statement of accomplishment or
00:09:07
um something along those lines and it is
00:09:09
an unofficial letter from the instructor
00:09:12
with the instructor signature that does
00:09:13
not carry any kind of University
00:09:15
authorization in the coming months we uh
00:09:19
plan to be able to offer for some of the
00:09:22
courses and not from all institutions a
00:09:25
basically University branded certificate
00:09:27
that would carry the university brand
00:09:28
and it would say that this is an online
00:09:31
non-credit bearing course so that to
00:09:33
distinguish from the courses that the
00:09:34
university offers its own enrolled
00:09:36
students um and so it'll be a university
00:09:40
brand certificate it'll also carry the
00:09:41
corsera brand somewhere but in a clearly
00:09:43
different role because the two roles are
00:09:45
different we don't we do not produce the
00:09:46
content we do not determine academic
00:09:48
standards uh when we begin to offer
00:09:51
those then yes we um we believe there
00:09:53
will be a charge for something that has
00:09:55
I think such a substantial value to the
00:09:56
students but it won't be a huge charge
00:09:59
so when you have U certification the
00:10:02
question of assessment becomes very
00:10:04
critical uh now given the large volumes
00:10:07
we have in these mukes uh I I would
00:10:10
imagine that'll be quite a challenge how
00:10:11
have you dealt with that so the question
00:10:14
of assessment is one to which we have
00:10:16
devoted extensive attention in our work
00:10:20
because uh we not even primarily for the
00:10:23
purpose of certification at the end but
00:10:26
because we believe that exercises for
00:10:28
which one gets meaningful feedback the
00:10:30
so-called formative assessments are a
00:10:33
really critical part of the learning
00:10:35
experience for the students if you don't
00:10:37
get feedback on how well you're doing
00:10:38
you really either don't do the work or
00:10:40
you don't learn the material well
00:10:41
because you don't know that you're not
00:10:42
getting it and so we've developed a
00:10:45
fairly extensive set of exercises that
00:10:48
can be graded automatically by the
00:10:50
computer and that ranges from The Fairly
00:10:53
mundane multiple choice or fill-in the
00:10:55
blank or numerical answers to some
00:10:57
fairly in-depth assessments like like uh
00:10:59
grading of programming assignments of
00:11:02
Excel spreadsheets for example for
00:11:04
marketing or financial models um grading
00:11:06
of mathematical expressions and most
00:11:09
recently one of our biggest efforts uh
00:11:13
which we started to put out in the
00:11:15
spring and has since gone out I think
00:11:16
nine classes or maybe even more is the
00:11:19
whole notion of peer grading where
00:11:22
students critically assess each other's
00:11:25
work providing both a quantitative grade
00:11:27
as well as qualitative feedback back and
00:11:30
by combining ideas from the kind of peer
00:11:33
review process that has been used in the
00:11:34
education Community with the
00:11:36
crowdsourcing ideas that have been used
00:11:38
in in websites like Wikipedia um uh
00:11:41
where wisdom of the crowds is used to
00:11:42
sort of uh compensate for lack of
00:11:44
expertise among uh the people doing the
00:11:47
work we believe that we can actually put
00:11:49
together something that provides
00:11:50
students with meaningful feedback for
00:11:52
the kind of open-ended work that is so
00:11:54
critical for example in Humanities and
00:11:56
in business and in other disciplines
00:11:58
yeah I mean I can certainly understand
00:12:00
that peer reviewed um uh assessments uh
00:12:04
help to address the scale question right
00:12:07
uh but uh isn't the absence of uh
00:12:10
feedback from a qualified instructor a
00:12:13
limitation so I think that clearly the
00:12:16
answer is that in certain um disciplines
00:12:18
in certain contexts the it is a
00:12:20
limitation so for example I would not
00:12:24
use peer riew to grade 10,000w essays
00:12:27
where an instructor would be able to
00:12:29
provide a real in-depth assessment of
00:12:32
the quality of the work and give the
00:12:34
kind of substantial feedback that only
00:12:36
an expert can give so I think um it
00:12:39
comes back to an important component of
00:12:42
this that we I think U discussed a
00:12:44
little bit earlier which is that this is
00:12:46
not going to be an experience that is
00:12:49
comparable to the experience that a
00:12:51
student who attends an institution like
00:12:53
the University of Pennsylvania and
00:12:54
person gets so there's going to be
00:12:57
differences and the question that we
00:12:58
should ask ask ourselves isn't whether
00:13:01
we are going to achieve equality between
00:13:03
the students at the University of
00:13:05
Pennsylvania and the students out in the
00:13:06
general public but whether by the use of
00:13:09
Technology we have improved the quality
00:13:11
of the experience for each of them
00:13:13
separately so both of them are better
00:13:14
than they were before even if they're
00:13:16
still not quite the
00:13:18
same that's very interesting and well
00:13:20
you you said before that you know a lot
00:13:22
of the people who take these courses are
00:13:24
already professionals who have degrees
00:13:27
and and that's fine but there may be
00:13:28
some
00:13:30
uh uh young people as well who are
00:13:31
trying to become professionals um how do
00:13:35
you address issues in the case of such
00:13:37
populations about things like verifying
00:13:39
the students Identity or plagiarism how
00:13:43
how how have you dealt with those things
00:13:45
so first of all um to speak to just the
00:13:48
the um assumption behind your question I
00:13:50
do think that as the mix of courses that
00:13:52
we're offering changes to include more
00:13:54
and more of these um introductory or
00:13:57
Gateway classes we're seeing uh increase
00:13:59
in the population of you know 18 to 25
00:14:01
year olds and often even younger
00:14:03
students from high schools who would
00:14:05
like to as you said become professionals
00:14:07
and and get their degree and and so I
00:14:09
think that's a really exciting Direction
00:14:12
because it is an important segment of
00:14:14
the population to which you want to
00:14:15
offer high quality education um you know
00:14:18
I think that there are a range of
00:14:21
technological solutions that one can
00:14:23
adopt in order to address issues of
00:14:26
plagiarism detection and um identity
00:14:30
verification and reducing the amount of
00:14:31
cheating um keeping in mind of course
00:14:34
that you will never be able to stop this
00:14:36
entirely and we have heard recently and
00:14:39
in the past of incidents of uh you know
00:14:41
cheating and plagiarism on our own
00:14:44
college campuses including at the
00:14:45
highest ranked institutions um we have
00:14:48
seen in uh incidents of cheating at the
00:14:51
SAT exams which are supposedly the most
00:14:53
proctored environment that one could
00:14:55
have so it's always going to be an arms
00:14:57
race and you're going to prevent some
00:14:58
form ter of cheating and others will um
00:15:00
emerge to replace them um so one has to
00:15:03
keep one's expectations realistic I do
00:15:06
think and there's been a bunch of uh you
00:15:08
know discussion about you know cheating
00:15:09
in online courses um on the web um I
00:15:13
think that to some extent those have
00:15:15
been a little bit overinflated um
00:15:19
relative to the actual amount that takes
00:15:21
place so uh for example our um in the
00:15:25
Princeton Sociology class the TA graded
00:15:29
every single one of the 2500 or so
00:15:30
midterms they estimate the proportion of
00:15:32
plagiarism at about 5% now even if
00:15:34
that's an underestimate because they
00:15:36
didn't actually plug this through a
00:15:38
plagiarism texure so maybe it's double
00:15:40
that 10% is that really that much higher
00:15:43
than what we see in our oncampus classes
00:15:45
not clear to
00:15:47
me love to uh switch to some questions
00:15:50
about the business model if you don't
00:15:51
mind um csair is not the only moo
00:15:55
initiative I wonder how you compare your
00:15:57
approach to that of edex or
00:16:00
Udacity so let me start from uh the
00:16:03
Udacity comparison because I think
00:16:05
that's a fairly easy comparison um we
00:16:11
are a technology platform that enables
00:16:15
the best institutions to offer their
00:16:17
education to the rest of the world and
00:16:21
we're not intending to become an
00:16:23
educational institution on in our own
00:16:25
rights we do not produce our own content
00:16:28
we rely on the content experts which are
00:16:30
some of the world's best instructors to
00:16:32
provide the content and set the academic
00:16:34
standards um and we do that by
00:16:36
partnering with the institutions um so
00:16:38
that faculty can basically teach under
00:16:42
their um under the brand of their
00:16:44
institution with the support of their
00:16:46
institution and I think we get some
00:16:48
really amazing um material by having
00:16:51
that relationship with some of the
00:16:53
world's best
00:16:54
universities um so um so that's one
00:16:58
comparison
00:16:59
um the the edex initiative in some
00:17:02
respects is more similar to what we're
00:17:05
doing because it's also an Institutional
00:17:08
level effort um in this case it's a you
00:17:12
know Consortium of two of you know the
00:17:14
world's outstanding institutions most
00:17:16
recently joined by a third um and I
00:17:20
think that it's uh wonderful for
00:17:23
students around the world to have access
00:17:25
to content from those universities as
00:17:27
well uh we personally are very uh fond
00:17:33
of this sort of notion of a broad
00:17:36
Consortium of peer institutions because
00:17:39
not only do we think that it provides
00:17:41
economies of scale because a single
00:17:44
platform that is a fairly expensive and
00:17:47
complicated thing to develop is now the
00:17:49
cost of that is advertised over a much
00:17:50
larger set of courses and and
00:17:52
institutions we also see tremendous
00:17:54
benefit to the hub of having um you know
00:17:59
a single place where people come for a
00:18:01
very large amount of content we have
00:18:02
almost 200 courses right now and more
00:18:04
coming up and that's why we have 1.5
00:18:07
million students in Rising is because of
00:18:09
that Hub in the same way that you know
00:18:11
Amazon is a hub or iTunes is a hub and
00:18:13
then finally I think an advantage that
00:18:15
maybe not quite as Apparent from the
00:18:17
outside but you can clearly see it from
00:18:19
the inside is having this collaboration
00:18:22
across institutions allows institutions
00:18:24
to learn a tremendous amount from each
00:18:26
other because this is a completely new
00:18:29
Paradigm in teaching this is not
00:18:30
standard instruction sort of videotaped
00:18:32
and placed online as used to be the case
00:18:35
this is a um a real shift in how we
00:18:39
teach and we're all kind of learning
00:18:41
together and by having a bunch of peer
00:18:43
institutions that are all communicating
00:18:45
with each other and learning I think
00:18:47
we're going to develop good pedagogy and
00:18:50
good ways of exploiting this uh this new
00:18:54
paradigm this new medium much much
00:18:56
faster since knowledge at has been
00:19:00
giving away free content and free
00:19:02
knowledge for the past 14 years uh you
00:19:05
know this is a question that is very
00:19:07
dear to my heart uh there is clearly a
00:19:10
cost to producing the content uh how do
00:19:12
you plan to monetize your service
00:19:15
presumably your investors are also
00:19:17
interested in that yeah let me say that
00:19:19
there's not only a cost to producing the
00:19:21
content there's also a cost to hosting
00:19:23
and disseminating the content there's
00:19:24
also cost of developing the platform
00:19:26
that does all that so yes there's costs
00:19:28
all around and and um one of the
00:19:31
decisions that we made was that
00:19:33
recognizing these costs and recognizing
00:19:35
that they are ongoing we realized that
00:19:37
we couldn't count on some miraculous uh
00:19:40
influx of funds from philanthropic
00:19:42
foundations to keep us going
00:19:44
indefinitely which is why we decided we
00:19:45
had to make it sustainable and so we
00:19:48
have several ideas on Revenue models all
00:19:51
of which at this point remain to be
00:19:53
tested because we're very very new to
00:19:55
this um but one of them is when we
00:19:58
touched on earlier which is the
00:19:59
potential to charge for certification so
00:20:02
even though we're committed to keeping
00:20:03
the content free so that even a kid in
00:20:06
Africa that doesn't have a credit card
00:20:08
can still learn something to make their
00:20:09
life better if you get a certificate at
00:20:12
the end that allows you to go and apply
00:20:13
for a job and gives you some kind of
00:20:15
tangible benefit then that I think is a
00:20:17
reasonable thing to charge for um we
00:20:20
think that we can get employers to put
00:20:24
in money into the effort by getting
00:20:27
access to the records of students who
00:20:30
have taken some of our courses only
00:20:31
those who have opted in of course um and
00:20:35
because there's some really amazing
00:20:37
things about our student base first of
00:20:38
all not only are they often educated
00:20:41
they're ones who have demonstrated the
00:20:43
interest in in in continuing their
00:20:45
education so they're self- selected for
00:20:47
some really good properties um
00:20:49
furthermore employers would get much
00:20:52
more quantitative and detailed
00:20:53
information about those students than
00:20:55
just the resume which is what they say
00:20:57
about themselves but rather did they
00:20:58
actually do well both in hard skills
00:21:01
like on the exercises but also in soft
00:21:04
skills like in helping each other on the
00:21:06
discussion form and in study groups and
00:21:08
so that's really important type of
00:21:11
information in the 21st century
00:21:12
workplace for which I think employers
00:21:14
would be willing to pay and we've
00:21:16
already had some interest from employers
00:21:17
about that will the institutions who are
00:21:19
part of corsera also monetize their
00:21:21
content in some way so when we um when
00:21:25
we bring in Revenue the university gets
00:21:27
part of it we share the revenue back
00:21:28
with the institution specifically to
00:21:30
support content production let me ask
00:21:32
end with one last question uh what's
00:21:35
next for corsera and when I say next
00:21:39
let's look next five years 10 years so I
00:21:43
think that um in five years uh we will
00:21:47
be able to offer most of the curriculum
00:21:50
in most disciplines we're we're now um
00:21:53
what eight months after starting off at
00:21:55
200 courses um typical instit tion even
00:21:59
a large one like pen has something like
00:22:02
3,000 courses um you know I think in 5
00:22:05
years we'll have 3,000 courses so that
00:22:07
if you want to be educated as uh you
00:22:09
know as a mechanical engineer or as a
00:22:12
business person or as a lawyer you will
00:22:14
be able to get access to some of the
00:22:16
world's best courses and gain an
00:22:18
education along those lines I also think
00:22:22
that in 5 to 10 years from the
00:22:25
perspective of the higher education
00:22:27
ecosystem
00:22:29
people will look back on 20th century
00:22:33
and say that I can't believe that we
00:22:36
spent so much of our students time
00:22:38
shoveling them into auditoria and having
00:22:40
them sit there for 75 minutes while
00:22:43
somebody lectured at them when we all so
00:22:45
clearly recognize and even recognize
00:22:48
today that that's not the best form for
00:22:51
getting people to really learn on the
00:22:54
material and and use it effectively and
00:22:56
so I think that our no of what makes for
00:22:59
a good education will shift drastically
00:23:01
I hope in five but maybe it'll take 10
00:23:03
years the old model has been good for
00:23:06
teaching but perhaps not so good for
00:23:08
learning so I would say that the old
00:23:11
model has been good for scaling because
00:23:14
up until now it was the only way that we
00:23:16
had as a society to really convey that
00:23:19
material to the larger number of
00:23:22
students who came in through our
00:23:23
institutional doors but I don't think
00:23:26
it's the right way to teach
00:23:29
and um and what we would hope to be able
00:23:32
to see is that by placing the content in
00:23:34
a way that can be assimilated easily by
00:23:36
students in an online format we can go
00:23:38
back to teaching the way it should have
00:23:40
been okay well well good luck to you and
00:23:43
thank you so much for joining us today
00:23:44
thank you
00:23:48
[Music]
