00:00:00
welcome to the next installment of the analytics at Wharton series focused on artificial intelligence I'm Eric bradow
00:00:07
professor of marketing and statistics here at the Wharton School and I'm also
00:00:11
Vice dean of analytics at Wharton we think one of the important applications and areas that we as a business school
00:00:18
should focus on is what we're calling today's session AI in action and I can
00:00:23
think of no two better people to speak to us about this than number one my colleague Bob Meyer who will be joining
00:00:30
us Bob is the Frederick H eer bet life insurance Professor he's also the co-director of the Wharton impact of
00:00:37
technology initiative so first Bob uh Welcome to our podcast welcome it's good
00:00:42
good to be here and then second um is Mr Roger Goo uh Roger is the co-founder and
00:00:48
president of Wai a prominent independent mobile based platform for comprehensive
00:00:53
Wealth Management in China uh Rogers got a career spanning multiple decades both
00:00:58
in the United States and in China he also serves and I'm very proud to have Roger both as a friend but as a valued
00:01:05
member of the analytics at Wharton Advisory board so Roger Welcome to our podcast well thanks for having pleasure
00:01:13
so Bob let me start with you um you do a lot of work on the impact of technology
00:01:19
let's call it broadly defined on human behavior so could you all and of course
00:01:24
employees are humans too so could you talk about in your perspective some interesting let's say uses of artificial
00:01:31
intelligence today in companies either by employees themselves or by firms and kind of the impact that you think it's
00:01:38
having uh well in many respects you know when when did it all start or how long have we've been using it and and I think
00:01:45
you actually have to go back to through decades where basically different kinds of artificial intelligence have
00:01:51
basically been an integral part of companies forever I I remember I I like to say I was on the ground floor uh when
00:01:57
I first started my career I was at car melon University and I used to play poker with some people from the computer
00:02:03
science department and one of their complaints was that they had to go walk all the way down the hallway to get find
00:02:09
out what was in the Coke machine because they would go down there and find out it
00:02:13
was empty so basically they they they programmed one of the very first uh uses of FID technology to to program uh their
00:02:21
uh the Coke machine so they could sit their computers and find out where uh uh whether it was empty or not it was worth
00:02:28
the trip and so of course today there's you know how many Internet connected
00:02:32
devices there are is like you know three times the world's population so it's
00:02:36
basically been integrated throughout every single function of a business particularly in manufacturing consumer
00:02:42
use and so forth and so forth um and so I I think some of the things that are happening today is we're shifting from
00:02:49
uh artificial intelligence as a way of looking up information processing data to actually generating new knowledge and
00:02:57
um uh and that's sort of like for and so one of the challenges I think for a lot
00:03:01
of employees if you're saying advertising it I no longer going to be needed to generate um advertising copy
00:03:08
when I can just throw it into chat GPT and it will generate the advertising so yeah I think one of the big areas that
00:03:14
we've talked about on this series is what I say and I agree ai's been around
00:03:19
for a long time as I was a PhD student almost 30 years ago is that predictive Ai and the use of AI to is as a data
00:03:27
source whether it's computer vision sound Etc that's been around as you pointed out it's the generative AI part
00:03:34
that's got people really excited today so Roger you actually have a company an
00:03:38
actual company with i that does work in this area so could you elaborate on some
00:03:43
specific areas where your organiz or your organization is implementing AI today yeah sure yeah initially we also
00:03:54
started with like you said in predictive AI because in our platform people par their financial accounts information
00:04:01
like their bank accounts credit cards Insurance retirement plans so we have lot of C structured data so we would use
00:04:09
the data to improve our you experience and also help help our partners to sell their products But as time moves out you
00:04:17
know we got unstructured data like voice image as you mentioned and the last few
00:04:23
years it is the generative AI that came along it's quite interesting we launched
00:04:29
a business called online financial literacy program about three years ago when people were locked down during the
00:04:37
coid period when this live streaming e-commerce went on like crazy you know so we launched this uh online financial
00:04:44
literacy and so far we have about three million people paid for our uh wealth management courses and it's very
00:04:51
interesting uh the large language model and the GPD 3 I think they emerged about
00:04:58
two years ago and we took notice it's not until last November when this chat
00:05:04
GPT GPT 3.5 came along it hit us at the right time because back then we were Hing the right bottleneck of high we
00:05:13
cannot hide our Shing assistant fast enough and train them fast enough then we turn on to the models you know uh
00:05:20
generally AI helped quite a bit one it is the uh kind of help enhance the interactions by doing more C
00:05:30
more uh kind of uh deeper and far reaching with automatic content generation so we develop an Robo ta and
00:05:41
what it does is uh this traditional kind of instruction exercise quiz cycle it take
00:05:50
aot with each individual student and when the test is done it will not tell the people that you know what you did
00:05:58
wrong and help them to review the content the content is tor made individually on each knowledge point
00:06:05
that they have missed so this PowerPoint is generated on the spot and the people
00:06:11
can do it interactively until they click I understand button so it been very helpful and also we had daily financial
00:06:19
news kind of analysis program and today I think more than 90% of the content is generated by this
00:06:27
uh uh large length model generative AI our research analyst only to do a very brief review and then click on the
00:06:35
publish button so it does help us quite a lot I think over 80% of interactions are performed by AI today rather than
00:06:44
human being so Bob could you talk uh given you know the center you you run now well you're one of the co-directors
00:06:51
of AI at Wharton um but it was also the Wharton impact of technology initiative um how do humans whether it's Learners
00:07:00
as in the case Roger was talking about employees respondents in studies how do they tend to respond when they know
00:07:08
something maybe they know we'll get to ro back to Roger in a second whether they know it's generated by uh an AI
00:07:14
engine how do consumers tend to respond to the difference between the two yeah that's very interesting there's been uh
00:07:19
increasing amount of work naturally on that I think one of the challenges of course right now there was a time when
00:07:25
uh you you could tell whether or not something was you were interacting for for example in a a text interaction with
00:07:32
a service person where you knew you were dealing with a robot and people didn't
00:07:36
like that okay whereas now it's very very difficult to tell and sort of one of the issues in uh in like online
00:07:42
advertising is deep fakes where basically you cannot tell the difference between it and so that sort of
00:07:47
represents sort of an ethical issue and certainly as you might expect people don't like it when they think that
00:07:53
they're being fooled and so that there's some evidence of that another area that
00:07:58
we're looking into I have a colleague that's looking into is one of the things
00:08:02
that the generative AI is doing is that it's synthesizing information and offering summaries and advice in task
00:08:11
domains where you used to do it manually so for example um if you needed to know
00:08:16
something about a topic you would go to Google and you would go through Bunches of sites and you were left to do the
00:08:21
synthesis yourself and form your own conclusion now you could just go to whether it's Bing or chat GPT ask a
00:08:28
question you know how do you do this what's your best advice for this and it'll take it will eventually do all
00:08:34
that work for you synthesize it and give you an answer and what we're finding and
00:08:39
it's it's sort of early in the process but basically people don't necessarily
00:08:43
like that all that much in some sense it's a it's a thing where you feel that
00:08:46
you have less ownership over it and so right now one of the biggest outstanding question is how much intelligence do
00:08:54
people want okay and the the reality is is I guess just the same way that you don't necessarily necessarily um you
00:09:00
know want to order all your meals out sometimes you want to cook them yourself in a lot of cases it could be there are
00:09:06
going to be some domain where people are going to be more trusting and feel more
00:09:09
ownership if they're Gathering the information themselves rather than having a computer do it even if it's the
00:09:15
case with the computer uh advice is actually maybe a little bit better so Roger Bob's response is a perfect segue
00:09:22
to my next question to you um as the president of a large company that's impacting millions of not only learners
00:09:28
but also so investors people doing their private wealth management how do you decide what to kind of assign to the an
00:09:37
AI engine whether it's an AI chatbot or an AI automatic grader or an AI person
00:09:43
that gives or an AI engine that gives feedback and what do you leave to humans is it purely one of scale is it one of
00:09:49
Finance H how do you think about what to assign to whom oh I think it's a in practice it's
00:09:58
an emerging process process it's a gray scale kind of segmentation and we do
00:10:03
have a lot of content uh generated by Ai and the people would know it's AI for
00:10:10
example it's a Q&A session when they ask general question or AI ask question they
00:10:16
answer they know they're dealing with AI that's kind of a level level one but
00:10:21
level two we have people people would think they are interacting with a live person
00:10:29
but actually this life person is largely assisted by the AI uh for the for example a piece of news analysis uh
00:10:39
actually our research ANS are not that powerful of gathering so much information on time you know they are
00:10:45
really empowered by the AI and and the highest level and we have people kind of upgrade to the ultimate level they
00:10:54
become a a member they pay the membership fee for our advisory services and they would appreciate uh not only
00:11:02
the assistant with AI so uh it definitely helps a lot but I don't think at this stage AI can completely replace
00:11:11
human being even for know I the uh I have three followers and my company a digital version of R and didn't do that
00:11:24
well I don't like it yet so I still try to publish my V uh in person so I think it's a it's a
00:11:32
process emerging ongoing but you know looking back in less than year it has made tremendous
00:11:40
progress yeah so Bob um building on Roger point and the point you made earlier how do you see you know we
00:11:46
always try to say it's not humans or AI it's humans and AI how do you see that
00:11:53
partnership evolving and also do you know you could imagine if I was an employee and I was being strategic if in
00:12:00
some sense I prove to the company that AI can replace me that may not be great either so how do you see those two
00:12:06
interacting well I think Roger spot on and saying the real challenge is is how do you figure what's their optimal blend
00:12:12
like what are the things and uh we lastly in the um um a couple months ago we ran a uh generative AI conference out
00:12:20
in San Francisco and uh some of one of the big topics there was trying to figure out uh such how good is BET The
00:12:29
generative AI in generating creative Sol solutions to problems and um and it seems to be the case that the the in
00:12:37
virging consensus is that what it does do is um is if people if you let uh say chat GPT work on a creative solution to
00:12:45
a problem what it does is it it's much better at bringing up the low end of of
00:12:50
human ability so basically if people who are not creative not good problem solvers you definitely want the machine
00:12:55
stepping in because they do a much better job on the other hand what it does do do is it tends to um make
00:13:01
Solutions seem very similar that it comes up with and basically there's the highend tale of people who there are
00:13:08
people who are particularly skilled of problem solving and uh and in that case that what happen is you'll compare the
00:13:15
the best of the human judgments and those tend to be better as judged by outside people uh than than Chachi PT
00:13:22
solution so so the task is is that that you I think consistent with what Roger was talking about um you want to
00:13:28
identify people within the organization who really do have these very special creativity skills and so forth and you
00:13:35
want to let them free you know and maybe work work you know with tools but basically you don't want to replace that
00:13:41
it's actually an interesting Theory which I'm sure people will test over
00:13:45
time which is does this if you'd like AI engine actually inhibit creativity on
00:13:51
the part of the humans it's why I always say you know the last thing I always
00:13:54
tell this to PhD students and then Roger I have another question for you the last
00:13:58
thing I do when I try to come up with a creative idea is when to read someone else's paper or synopsis paper because
00:14:03
it does not help me come up with a creative idea at all um so Roger let me ask you um Robo advisory Services seems
00:14:13
maybe to me a much higher Stakes type of decision than someone trying to learn financial literacy so how do you think
00:14:22
I'll just use the language that we use in Academia all the time there's a very
00:14:25
different loss function between an AI engine making some portfolio allocation or recommendation to me and then I go
00:14:33
bankrupt as a result or you know I take some sort of literacy test and the AI engine kind of gets it wrong and I'm
00:14:39
like all right well that's not great but it's not the end of the world how do you
00:14:43
think about Bob was talking about let's called employee heterogenity how do you
00:14:47
think about task importance heterogenity and the role of AI given high stakes versus low stakes types of decisions
00:14:55
well this is a very important question you know uh I take the Global advisory is that high stake uh it's interesting
00:15:02
this uh this uh this new phenomenon called generative AI tend to be sometimes very creative especially GPT
00:15:10
you know sometimes it's imaginary uh well llama to tends to be more specific
00:15:17
so in this uh Robo advisory has two sides oftion uh one side is the customer profiling custom the other side are the
00:15:25
asset understanding asset Char Char istics so on the right hand side asset characteristics we want to be very
00:15:34
careful with in terms of AI it's still classical AI you know classical uh statistics uh partial differential
00:15:45
equation uh some sort of uh recursive neural network just understand uh understand the
00:15:53
sigas the betas uh the gamas you know make sure that part differ frer are still there but the left hand side the
00:16:03
customer needs uh the the life you know uh objectives can be very can be very suggestive because in the past the
00:16:13
classical way uh doing this advisory Services is used to ask people about you know your personal asset liability
00:16:22
income how many kids you have when are they going to school when they want to retire then come up with tailor Med
00:16:28
Solutions but now with generative AI this can go a lot deeper uh you can having conversations about life
00:16:36
objectives one is of course retirement you also have example your wedding anniversary but then in the The Advisory
00:16:45
who are wedding anniversary the risk profile appetite could be very different from a kind of 20-year retirement plan
00:16:53
so for wedding anniversary if you take on risk like play with derivatives or on Leverage trading if it works well you
00:17:01
know you go on skiing in Switzerland if it does not work you always have you know Disneyland in Florida so I think
00:17:08
with the help of AI this can go a lot deeper a lot more creative inactive and also gim finish me meanwhile still holds
00:17:18
the seriousness of financial advisory so I think it definitely add value you need
00:17:23
to understand where to use them kind of uh correctly in the in the Prudence fashion yeah Roger I have a question for
00:17:32
you uh do you ever worry a little bit that as um particularly in the context of financial advising that the uh the
00:17:40
tool becomes so good that P maybe put too much trust in it um so for example if you're dealing with a human advisor
00:17:48
you know you're dealing with a human and you know that humans are are fallible
00:17:52
and so if the human advisor basically tells you this is what you should be doing with your money this is what you
00:17:57
should retirement you'll follow that advice but basically with u um which you
00:18:02
know it's potentially somewhat fallible but on the other hand if if you have a u
00:18:07
a very Advanced um computer tool which is uh and you you sell it as being you know optimized based on whatever uh is
00:18:17
there a worry that people would become then go the other extreme of being too trusting of it well but I'm not worried
00:18:25
this part actually people believe what they believe they can believe a real person they can believe into an
00:18:32
algorithm which is also humanized have content generation if the results is good and they believe into it so the
00:18:39
beauty here is you don't have one key opinion leader you have dozens you know
00:18:45
because even different AI have different right characteristics people have my value based I'm my gross based so and
00:18:54
it's fun what I do worry is actually on the regular regulatory pieces because
00:18:59
you know generally like I said before tend to be more creative so but there are rules and regulations what you can
00:19:07
say what you cannot say for example uh with certain uh kind of uh licenses you're not allowed to recommend
00:19:15
individual stocks or maybe you are not allowed to recommend something to people not at this risk rate so this part you
00:19:23
got to be very careful if you ever cross a line it is actually the companies the
00:19:29
people behind those this AI are holding held responsible so I want to be very careful about that but unfortunately uh
00:19:38
the lines are not very clear given from The Regulators because it's a new thing
00:19:43
for them as well so Bob you and I are both obviously marketing professors um you had mentioned about let's call it ad
00:19:51
creation as one area what do you see and then I'm going to ask Roger about his
00:19:55
area in fintech but what do you see as the major application areas that you see not just us as academic studying but
00:20:03
that you see AI is going to be used in our home field of marketing well I think you had mentioned the case of usually
00:20:09
when people say how there's two different cases of it okay certainly s of predictive AI that's been around
00:20:14
forever and product design so forth um uh and and certainly in the case of every time you go to Amazon you're
00:20:21
seeing AI okay you're basically seeing products recommended for you and that's
00:20:26
been around for for for for a while so presumably they'll see improvements on
00:20:30
that you're going to be seeing um uh better customizations when you go there
00:20:34
you go wow how did it know that's exactly what I wanted and some people might find that scary but other people
00:20:40
might find that's exactly sort of the product I want uh the other side in terms of the creativity part that's sort
00:20:47
of a little bit we don't quite know yet um in terms of whether or not if we Fally U fully turn over all of um the
00:20:56
creative process and advertising design creative process and strategy formulation uh po the potential is
00:21:02
certainly there for example with u large language models and general generative AI at whole to basically generate all of
00:21:09
this now whether or not that's end up going to be the type of thing which is
00:21:12
going to generate uh that outof the-box type of AD Campaign which really makes the difference for a company as opposed
00:21:20
to whether or not it makes it generates a whole bunch of advertising which is all the same okay so effectively it's
00:21:26
just sort of you know govering the Mass rather than the tail and usually we like
00:21:30
to focus more on that that breakthrough creative idea and I'm not so sure they've conver very much out whether or
00:21:37
not um generative AI can produce that kind of a breakthrough and Roger how about in fintech what do you see as the
00:21:45
biggest uses of AI today well we just follow on what you guys were discussing about in marketing I think uh I'm
00:21:53
actually very optimistic just like help to discover new you know DNA patterns new drugs and also in ALA go you know AI
00:22:02
discovered new ways of playing go and in this marketing especially in the fintech
00:22:08
area every months we spend the tens of millions uh into those the we call you know uh information on Tik Tok or
00:22:19
equivalent to kind of Google app kind of on on those sides so every week or every
00:22:26
point of time usually we have hands if we call marketing plan don't and which
00:22:33
one's are going to stick you don't know but with generative AI you can create
00:22:38
those creatives uh very easily it's like it disperse different patterns you the way
00:22:46
figure out the dnas from those marketing plans maybe it is you know the way you bring up a a pet versus a young boy you
00:22:57
know maybe maybe it is the feeling of a retirement age something like that but the underlying I call DNA of marketing
00:23:07
materials now can be S of tried and Ed and discovered very wild and in this whole Space of digital marketing I think
00:23:16
it takes to the next level so I think that's on the one hand because every company needs toire customers and manage
00:23:24
customer experience I think AI could help lot uh I think what furthermore because this chat gbt uh this Genera AI
00:23:34
a large language model usually it's not as good as sort of doing new numerical
00:23:40
calculations but now with this plugins you can actually combine you know those tools with the traditional Ai and
00:23:48
actually even on the asset management side I can see new kind of AI powered kind of trading algorith is very
00:23:58
different from the traditional ones and competing on part with the uh uh the kind of traditional hedge funds uh
00:24:08
programs so I think uh it's a big thing it's coming and only we right now only
00:24:15
scratching the surface so we only have about a minute left so Bob maybe in 30 seconds or so um if we're sitting here
00:24:21
10 years from now what have we been talking about what are we going to be talking about that's happened over the
00:24:26
last 10 years that's an awesome question I have no idea whatsoever and and I
00:24:31
think that in it's it's from my perspective as a researcher this is like
00:24:35
uh the most exciting time to be alive because basically what we're in the precipice of is just really very
00:24:41
fundamental Transformations as to uh how people get information how people generate information and we're just
00:24:48
beginning to understand how this is affecting society and so to me there's just so much to that we have to learn as
00:24:55
researchers going forward so it's an awesome time to be here and Roger from your point of view um how do you think
00:25:01
about the business World investing what do you think are going to be the big breakthrough issues in AI in the next
00:25:06
few years well I think companies have to figure out their position this big AI game you know it's like a big tree right
00:25:13
you have open AI Google those guys are the roots right and you have maybe uh F Tech players like us there are this
00:25:23
industry specific applications they are like a Trunks and there are many many Le
00:25:28
applications so I think uh either you are a young entrepreneur right into the game or whether you are established
00:25:35
company I think it's very important understand that Financial uh the technology Trends and figuring out your
00:25:41
position in your field I think know keeping an open mind would be very very helpful and uh things change and we got
00:25:50
adap well this is Eric bradow professor of marketing and statistics here at the Wharton School and also Vice Jean of
00:25:56
analytics I'd like to thank my two guests my colleague and friend Bob Meyer the Frederick H Ecker Met Life Insurance
00:26:01
professor and also one of our co-directors of our Center on artificial intelligence and and I'd also like
00:26:07
to thank Roger goo who's the co-founder and president of aai a prominent independent mobile based platform for
00:26:12
personal Wealth Management in China and as I also mentioned one of our valued board members at analytics at Wharton so
00:26:18
Bob and Roger thank you for joining us today thank you very much