00:00:00
it is the center of the technology world right now it's not what Mark did it's when he did
00:00:08
it and the king of the cloud is Salesforce please welcome Mark Ben one of the things that really
00:00:18
matters me is having a positive Global impact technology is not good or bad it's what you do with it that matters in
00:00:25
your quest to change the world don't forget to do something for other people and that was a moment in time when I
00:00:33
said wow when I start a company I'm going to make sure that philanthropy and giving and generosity and these values
00:00:41
are in the culture of the company from day one you want to stay here are you want the couch you want to
00:00:53
sit on the couch where you want to sit I'll give you the couch I'll sit here you deser you deserve the couch you
00:01:00
deserve it the big couch okay it's little little too close it's nice to see you also I I warned you that
00:01:09
I I'm not the the interviewer in the group you but this is you chose me so I'm honored but you're the nice one oh okay
00:01:18
thank you well it is am I right is he the nice one and you're the one that all the
00:01:26
women really like like I'll talk to my friends at dinner there like you know sax what's he like he's amazing let's
00:01:33
just say we're honored to have Mark Benny off here and uh truly who's a Visionary in the world of software and I
00:01:40
would say you know there's probably a lot of and I thank my mother for writing that video by the way is well Mom thank
00:01:47
you for writing that for me great you know in the in the world of business offer in particular we don't have that
00:01:54
many people who you can describe as Visionaries but you consistently have been one you really it's true
00:02:01
you got the I think the whole we're sitting now on the edge of the couch okay here we go maybe we'll end up on
00:02:06
the floor I don't know what's going to happen I'm trying to keep it engag around a lot oh okay all
00:02:15
right is this how it's going to be the whole time way worse way worse okay all right
00:02:22
let me finish the this little intro here um I forgot where I was so can I just before we start you know
00:02:31
listen so I want to just do something I would not normally do and this is like going to be a little bit of a thing but
00:02:37
I just have to do a little Riff on this but we just heard some an extraordinary presentation on an extraordinary man and
00:02:45
there's somebody who's amazing that most people don't get to hear of and we just
00:02:48
heard his name quite a few times his name is shin yamanaka Yaman nakasan he is um based in Kyoto Japan but he works
00:02:58
halime at UCSF and it's amazing what his vision for the world is that he thinks basically that
00:03:05
we're salamanders and we're going to be able to regenerate ourselves and that's amazing and so I've been friends with
00:03:12
him maybe for a decade but I fund his research and so a lot of these things to watch him have these breakthroughs and
00:03:18
you heard about the yamanaka factors the yamanaka factors which are basically this idea that
00:03:24
yamanaka had this breakthrough in Kyoto you know basically hanging out there in his lab eating the sushi the whole thing
00:03:32
and then boom and he goes if I take these four things I can take an ordinary skin
00:03:40
cell just any little skin cell and turn it into a stem cell which is like the the heart of human existence and he did
00:03:49
it and he was able to repeat it and repeat it and repeat it and he won the Nobel Prize for it pretty cool and then
00:03:57
he and I'm going to get the pronunciation of this wrong but but he then was able to take that stem cell put
00:04:03
it into your eye if you have tacular generation and boom healed the eye because the eye
00:04:12
regenerated then he worked with a buddy of his in the lab next door and he took the same thing took the stem cells and
00:04:20
he turned it into on a cookie sheet and it looked like it was like a plastic thing on the cookie sheet it was
00:04:27
really cool and then he's like took out somebody's cornea that was all screwed up cut the material out of the cookie
00:04:35
sheet popped it in the eye and the guy could see it was like amazing then he's like listen this is
00:04:44
amazing I bet I can grow a brain so he took the step cells and he started growing brains called
00:04:52
organoids and he's like got a cookie sheet of brains and I'm like really he's like this is amazing look at all the
00:04:59
brain brains and I's like and then I went and saw him and had lunch with him and like I'm like what's happening with
00:05:05
the brains he's like I stopped the brains I'm like why did you stop the brains I think they can feel the pain
00:05:12
I'm like oh scary then he's like then I said to him now what are you doing oh I'm growing
00:05:21
intestines I'm like whoa intestin is that good he's like huge idea I can now grow intestines on the cookie sheet and
00:05:30
taking the you know stem cells I've got a whole intestine here and then like he can do like turn it into a lab for all
00:05:36
the horrible things that people get in their gut and all these diseases that have never been cured but now you have a
00:05:42
real simulated environment this is an incredible person anyway where do you want to go with that I'm going I'm going
00:05:50
with this you got to stay with me trying to help bring the energy up in here okay
00:05:54
listen follow just hold on hold on hold on hold on hold wait wait wait this is going to get good okay so then I'm like
00:06:02
you heard the story like at the end they said listen how do I get these regenerative factors going inside myself
00:06:09
so UCSF just published research based on funding grant that I and others have given them and they had a breakthrough
00:06:16
that the regenerative Factor inside your own blood is called pf4 and the way you get pf4 and I'm not
00:06:23
going to get this exactly right because you know I'm in software I'm not a doctor so just follow with me I thought
00:06:28
that's what we're going to talk about today that I know but I got to tell you this cuz I'm got so jacked watching that
00:06:33
one is it was either that or those crazy shots you have backstage I don't know okay number one is P4 you get more
00:06:43
regenerative factors in your body like what you know calorie restriction and if you know David and I that does not sound
00:06:50
very good two working out with weights also not exactly our top thing parabiosis do you know what that is so
00:07:04
parabiosis kind of came out of research published a decade ago in New York Times
00:07:07
and others which is came from salval another person I work with at UCSF where they took the blood of a young Mouse and
00:07:16
put it into an old mouse and then the old mouse got young again and that was moving the pf4 into that old mouse so
00:07:26
that's and the fourth thing is cloth theapy which is a genetic therapy that I don't really understand and these four
00:07:31
things can start to generate more of these things inside your body so then I'm like getting excited I'm like God I
00:07:37
have these problems maybe I can regenerate different parts of myself whatever and so I'm talking to my doctor
00:07:44
at UCSF because I'm going through my own serious problem where I'm like my left leg is like a half an inch shorter than
00:07:50
my right leg and I'm running on the treadmill and I'm always ripping my achilles ripping ripping and all of a
00:07:56
sudden my achilles looks like it has a dut and in fact I went to UCSF and there's like an MRI you know well how
00:08:03
many of you have had an MRI raise your hand so you know what as horrible it is anyway you get in this big machine
00:08:07
they're looking at my Killers they're come out they're all like this oh sorry about this really horrible and I'm like
00:08:15
so I kind of took this thought and I'm like talking to my doctor I'm like why can't we like do use some of this figure
00:08:21
out what we can do so he's like all right I'm come back on Wednesday so I come back on Wednesday at 5: CL you know
00:08:30
I'm in Mission Bay at UCSF and I'm like hey Anthony where is everybody we're going to talk about it come into my lab
00:08:39
so I come into the lab they've got like a centrifuge there all this stuff going on I'm like well this is interesting
00:08:46
it's like did you work out today yes I'm work out do you're following the P4 I'm
00:08:50
doing it okay this is what we're going to do it's going to be very straightforward because we have two we
00:08:55
have two things we can do with you mark number one we can just take your kill and we bring you into surgery right now
00:09:01
we'll just shave off half your Achilles and then put you in a boot and see where
00:09:04
you are in 6 months I go doesn't sound great second idea what we're going to do is we're going
00:09:13
to we're going to take a scalpel B right here we're going to cut into your Achilles like 20 times and into your
00:09:21
ankle I'm going to take your butt I'm going to spin it I'm going to try to find the pf4 in your plasma I'm going to
00:09:26
inject it into your Achilles and into your plasma slice into it with my scalpel I go sounds great goes one
00:09:34
problem I go with that we don't use anesthesia to do that why because it destabilizes the PRP and the plasma and
00:09:41
all the pf4 and all that I'm like let's rock let's rock so he did the whole thing and then boom like an I'm like a
00:09:52
salamander they grew me a new Achilles right in place so that thing that you just heard that is real
00:10:00
and you know it's pretty awesome what can what and uh it's yeah I have a question for you um so if he can sell
00:10:10
three billion into his startup I should probably start I'm ready to go got the pitch did do you ever consider that you
00:10:16
missed your calling as a scientific researcher definitely not definitely not you're happy with the choices you made
00:10:22
well no that's an incredible story so you are one of the first to actually try using the yamanaka factors on yourself I
00:10:29
wouldn't think I'm one of the first but I think that it's very real and it's going to going to have a huge impact on
00:10:34
our whole on our lives and I think that we should be supporting these medical researchers I think it's it's one of the
00:10:40
reasons that I've you know put almost a billion dollars into UCSF of philanthropy because I believe in these
00:10:45
people who have absolutely yeah they've dedicated their lives you know to basic science and doing and meeting them they
00:10:55
so inspiring to me and like I just had lunch with yamanaka and salval and Anthony Luke and another incredible
00:11:02
researcher Mark moiser at my house and like we're talking about the intersection between oncology and
00:11:08
regenerate medicine which is like two completely different worlds that don't talk to each other and it's what
00:11:13
inspires me that you know we can you know work with others to kind of get give them the entrepreneurial push to go
00:11:20
do something incredible and these people are just awesome each one is amazing that is that is incredible so let's
00:11:26
shift gears and talk about something else nice coincidence with the yeah no it's incredible it's a great story I
00:11:31
know you you're very philanthropic and do a lot with UCSF so uh kudos to you for encouraging that type of research uh
00:11:38
let's shift to to another thing that's having a huge impact in our lives which is the cloud and software where you were
00:11:43
a Pioneer you started Salesforce back in 1999 25 years ago 25 years ago and how long have you been a public company for
00:11:51
at this point 24 to 20 years 20 years and one of the things I noticed count 80 earnings calls
00:12:01
yeah well actually speaking of earnings let's here let's see if we have this slide do we have earnings well first oh
00:12:07
yeah oh boy what's what slide is that this is your stock chart over 25 I think 25 years 20 years yeah you're almostly
00:12:15
your all I guess there's no linear success exactly right yeah really good point yeah we had a keep basically had a
00:12:21
bubble we had a bubble inate 21 we had a huge correction in 22 there was I need to make a note of that we should talk
00:12:27
about that but you're your basic back to where you were this is one of your tweets actually this is one of the
00:12:32
things I appreciate about the way you do earnings calls is you just put out this
00:12:36
really simple tweet and it just and it shows a progression and if you know if you like
00:12:42
looking at numbers the way I do and seeing patterns in them one of the things I noticed a while ago was that if
00:12:48
you start at the bottom and work your way to the top that Salesforce is growing by about 20% a year and if you
00:12:56
look at it over 3 years that's roughly a dou so every 3 years Salesforce was doubling
00:13:02
and that means that over a decade it's growing 10x and so every decade is basically an exponential if you can
00:13:09
stick with it long enough that was one of the patterns I noticed with sales look I I think that you know that the
00:13:15
growth obviously is incredible to 38 billion and obviously the cash flow is incredible you know it's more than C
00:13:22
Coca-Cola did I think last quarter and the margin is incredible but let me just say probably the best decision we made
00:13:28
in not on the slide which is the day we started the company um we put 1% of our Equity 1% of
00:13:37
our profit 1% of our product 1% of all of our employees time into a 50613 foundation now at the time it was very
00:13:46
easy because we had no employees we had no equity we had no profit we had no so wasn't very complicated but that idea
00:13:55
though really kind of created the foundation of the company because we're able to do now and I think you know the
00:14:01
numbers right where you know almost 10 million hours of volunteerism we've been able to give away almost a billion in
00:14:07
Grants we run almost 100,000 nonprofits and NOS for free on our service and I think it really set the stage that
00:14:14
business could be the greatest platform for change when it came for Salesforce it un it gave it that philanthropic
00:14:19
platform so is there two billion in equity sitting in that 501c3 at this point a lot well there's more I think
00:14:27
there's about a half a billion in the foundation and a lot of has been already given out and then we give out more
00:14:32
every year and every month every day whatever but like on Monday we'll give another $25 million approximately to the
00:14:40
San Francisco and Oakland Public Schools and that is you know we've given them about $150 million I mean it's it's
00:14:48
obviously I went to Public Schools it was very important to me but all my mother was a teacher in the San
00:14:54
Francisco Public Schools but also our employees you know have 75,000 employees their kids are in the public schools and
00:15:03
so it's a key part of our Mantra and our culture that we're trying to support public education I adopt a public school
00:15:12
I really think that each one of us can needs to focus more on the public educ education system in the United States
00:15:18
it's something I encourage in not all my employees but whenever I do a presentation I'm like you know my public
00:15:25
school is like a block for my house procedo middle school and I just went down there and knocked on the door and
00:15:32
they're like who are you and I'm like what do you how can I help you and what can I do to support you they need a
00:15:39
new playground they need this they need that and maybe they just need some me some support moral support um but uh
00:15:47
it's been a great thing to really anchor the company in those values and I think
00:15:52
it's an important thing uh for every company so what did you think when you saw that open AI started with a non
00:15:59
profit not as 1% but as 100% but then it became a for-profit what did you think of that Innovation confusing I you know
00:16:08
I mean 18,000 companies have now followed our 111 model you can find out about it at pledge
00:16:14
1.org that other model I don't really understand I think we've proven our model this is important you know we came
00:16:20
out with three models the cloud model which you also have been part of that the subscription model you've also been
00:16:27
part of that and the philanthropic model and you've been part of that and those ideas that we're doing three models
00:16:33
that's continues to be the fuel for the company and extremely important and I think that for a lot of these companies
00:16:40
that have followed us that have gone onto scale and have had huge IPOs and whether it was slack or whether it was
00:16:46
at lasan or whether it was eilo or whatever they have these huge foundations and have had huge impact and
00:16:54
business can be the greatest platform for change and you can do a lot with your business and you you know we are
00:16:59
all building great products okay that's great and we're selling them that's great too but we can also do a little
00:17:07
more with our business and we can use it in a positive way and try to move the world maybe a little bit more in the
00:17:14
right direction okay so let's talk about the cloud part of that Innovation where
00:17:19
do you think we're at right now I mean is it it's is it all AI all the time how how are you thinking about it we're at
00:17:26
the precipice of the greatest moment in the history of enterprise software and of cloud computing there there's no
00:17:34
question we you know I had a moment I would say more than a decade ago which I call my kind of AI freakout moment where
00:17:41
I really felt I mean maybe it's you know obviously we've all spent how many of you watched Minority Report all right we
00:17:49
saw that movie and what about war games War Games anybody remember that from okay uh
00:17:54
her um yeah we all saw these movies Terminator Okay that one's a little scary but we all seen the movies and you
00:18:03
know like Peter Schwarz who wrote or was a key part of writing Minority Report and um also war games uh you know as our
00:18:11
chief futurist at Salesforce and a decade more than a decade ago I had this moment where I was like okay this is
00:18:17
really happening here we go and bought a bunch of companies and put together Einstein and Einstein has done amazing
00:18:23
you know it's doing trillion transactions trillion and a half transactions a week predictive
00:18:28
generative I really thought okay this is was going to be the moment but now I'm really convinced that we are now really
00:18:36
at the moment right now where enterprise software is going to be completely transformed with artificial intelligence
00:18:44
and we're going to see it and obviously I'm getting tuned up for dreamforce which is going to be Tuesday of next
00:18:50
week how many of you are coming to dreamforce not enough anyway sad these aren't my people I'm
00:18:58
leaving now well it's good you well they look but no let me just tell like since you're not
00:19:03
going to be there let me tell you what's going to happen thanks for being part of my team
00:19:09
anyway number one is you know we're going to you know we really see a moment right now where we
00:19:17
are 100% focused on one thing and one idea and I can tell you why that is if you're interested but it's agent force
00:19:24
and agent force is the most exciting thing I have ever worked on in my career um it's the culmination really of
00:19:32
everything that we've done at Salesforce because to make agent force really deliver we had to have all of our
00:19:38
customer touch points wired up which we do we have to have an Amalgamated data Cloud because we need the data
00:19:44
especially to achieve the AI accuracy and the metadata as well and we have to have the agents it's these three layers
00:19:53
that are really going to deliver this next generation of capability and I was just with Disney last night and Disney
00:19:59
has agent force they have the newest version which we call Atlas which is our most accurate not just model but we have
00:20:05
an extremely unusual technique that we'll talk about and Atlas delivers for for Disney for their cast members which
00:20:12
are their employees through extremely complex uh problems that it's solving for them more than 90% accuracy and
00:20:21
almost no hallucinations and in some cases 95% accuracy and almost no hallucinations and that idea that we can
00:20:29
kind of come into a very difficult and complex and sophisticated data set now with now with Disney if you go to
00:20:37
disneystore.com that's Salesforce if you go to the Disney parks do you still go to Disneyland sometimes yeah okay you
00:20:42
ever get a Disney guide sometimes yeah it's great because you get to cut around the lines and all that how many of you
00:20:48
have done the Disney guides thing we got like a lot of poor people here actually sad anyway she get these Disney
00:20:57
guides cuz they're like get you around the lines you got do 30 rides a day and it's much better than having to wait
00:21:04
okay but anyway Disney guides run on sales force they just SL they have slack too they've
00:21:10
got we do disneystore.com we have Disney plus because you know the service now like fell over and we had to like
00:21:18
replace that inside the Disney plus call center we have we're do the Disney Cruises and the Disney real estate and
00:21:25
we have every Disney customer test Point all wired up so The Amalgamated data set
00:21:30
that we have around Disney is awesome so when we can take that Disney data set and then we apply Atlas and agent force
00:21:37
okay so how do you define we are able to deliver a level of accuracy that has been incredible and I've got a couple
00:21:43
more examples I can tell you that are just blowing my mind and I never thought it was really possible but now it really
00:21:49
is go ahead yes well I just wanted to you want me to ask you a question also no no no um what do you mean by
00:21:59
agent because we're starting to hear this term a lot but I think a lot of people here may not know what that means
00:22:03
in the context of AI did you see the movie The Matrix yes I did so are we talking about agent Smith or what are we
00:22:08
talking about well we're at some level I mean I think like I'll give you an example that you know um we're working
00:22:14
with a large medical company not so far away from her Kaiser they've got 20 million patients they have a super
00:22:21
complex data set they have all of the data from epic they are the largest epic customer in the world and more than 90%
00:22:30
of all patient inquiries and scheduling requests and schedule my doctor schedule
00:22:35
my CT scan my MRI my this my that are being resolved by agent force and Atlas that idea that we can resolve through a
00:22:44
autonomous agent a deep and complex customer interaction is a breakthrough thought obviously we have to do a few
00:22:52
things to make it really work for our customers number one is it's got to be trusted because our customers Trust
00:22:58
we're running the largest banks insurance companies media companies cpg companies blah blah blah blah blah in
00:23:04
the world number two is it's got to be easy for them it can't be some separate team that they're going to spin up it's
00:23:10
their existing Salesforce team it's happening within the Salesforce platform it's got to be open it has to have be
00:23:17
able to work with and interoperate with other systems it's going to have to be multimodal so it's going to have to
00:23:22
speak to them and have voice and video and do all of those kind of incredible capabilities and one other key thing
00:23:29
because evidently the humans have not gone away the doctors have not gone away from Kaiser and the cast members have
00:23:37
not gone away from Disney and on and on so we're going to have to handshake seamlessly with our apps so even though
00:23:44
we have all these apps and we've wired up all these customer touch points the agents are autonomously interacting with
00:23:51
and building the data and metadata and extending it and by the end of this month we'll have more than a thousand
00:23:57
customers on our agent force platform the efficiency and productivity that we've been had with agent force is like
00:24:05
nothing I have ever seen with any of our customers or technology in the history of software but there's a second point
00:24:12
it isn't just about this kind of ease of use it's that that they have the ability
00:24:16
to do things that are truly astonishing and that is also generate Revenue so they can go out and like on a day like
00:24:25
today like it's 10 something degrees outside or if you been out there it's pretty
00:24:29
hot and Disneyland may not be as full today as it's going to be and they knew that was going to be true two days ago
00:24:36
that a heat wave was coming Disney can proactively go out to their consumers and their customers and say hey come
00:24:43
enjoy the heat with us all you know Disneyland and we're going to give you a special promotion or Price or contest or
00:24:49
whatever it is to come to Disneyland so we want to be able to proactively go out
00:24:53
and generate revenue and we also want to be able to kind of bring that customer service in I think last night I had
00:25:00
dinner at Beverly Hills at the grill have you been there great right cream spinach well I did something different B
00:25:06
what do you want what do you prefer um well I like like potatoes you know potatoes okay po you know any kind of
00:25:13
potato you like any kind of potato baked potato steak fries all so I'm on Open Table Right anybody here use Open Table
00:25:21
not it's very weird group anyway so oh I'm using anyway you can use Open Table to make restaurant
00:25:31
reservations and there's 160 million consumers on Open Table they're not in this room but they're somewhere and
00:25:39
they've got also 60,000 restaurants and they've got a lot of complex issues you know in regard you know I didn't get my
00:25:46
table or my food wasn't right my potato didn't get cooked whatever it is these things are going to get worked out but
00:25:52
also all of a sudden the restaurant's like oh look we're not as full tonight as we want to be and we're willing to do
00:25:57
let's go out to our customer base and bring them in but let's do it through a complex conversation you know an
00:26:04
empathic conversation as an agent with our customers I think it's going to be a rocket ship okay so so how long will it
00:26:12
be until when you call a customer support center you're talking to an AI that sounds like a human and you can't
00:26:19
tell the difference are we there yet or we're there yet we are already at that point we already have that live and we
00:26:27
will have that scaled for thousands of customers before the end of um live for with thousands of customers live before
00:26:36
the end of this year and we just I just demoed it I was just just at a conference and spoke uh mile couple
00:26:42
miles away from here at KPMG and we showed them that exact situation where you know through you know we used to
00:26:49
call you know this kind of voice response system whatever but you would kind of hit a wall pretty quickly with
00:26:56
your Bot you know but these aren't Bots these are not the Bots you're looking for these are like we're really getting
00:27:04
to like another level capability and I think that it's pretty impressive and I think in the example of Disney you know
00:27:10
Google has some great products I know Serj was here yesterday and they've done a great job with AI as you know but in a
00:27:16
head-to-head Benchmark of sales forces agent force against Google's AI uh we twox them on accuracy and the reason why
00:27:25
is we'll explain it next week um you know it's a couple of things not only is there our NextGen models but it's also
00:27:32
new techniques involving Next Generation retrieval augmented generation rag techniques that no one has seen before
00:27:39
and it's really incredible what's possible so you're kicking Google's ass I'm cool with that well they're good
00:27:45
partner also customer and I love them but yeah it's it's competitive just yeah let me keep let me keep holding on this
00:27:51
we're trying to all make AI a little more accurate and a lot of less a little few less hallucinations along the way
00:27:56
let me give the audience a little update about something we just heard at open AI
00:28:00
they just did a a a day where they brought in relatively small number of investors and kind of give gave us all a
00:28:07
update on their product road map and it sounds kind of similar because everyone's moving in the same direction
00:28:12
so there are three big takeaways number one was that they said that llms would soon be at PhD level reasoning right now
00:28:20
it's more like a smart high school or college student in terms of the answers we're going to be at the next level
00:28:26
shortly behind that is agents like you're talking about and then third and closely related is that agents will have
00:28:33
the ability to use tools and a tool can be a website so if you think about it now you've got this llm it's really
00:28:41
smart it's got you know it's like a PhD it you can give it an objective it will break that objective into a list of
00:28:48
tasks and those tasks can include using other pieces of software and thanks to things like open AI just launched the
00:28:58
audio API which developers can use it's in private beta we have some companies using it the llm can now basically
00:29:06
pretend to be a human and you know there's it won't be hard to find a piece of software to enable a phone call so
00:29:12
you can imagine telling a a personal assistant agent that and it could be you know it could be open table that hey
00:29:21
book me book me a a dinner reservation at the grill and it could place a phone call on your behalf and actually talk to
00:29:27
the grill it could also go on open table and just use open table and book it but
00:29:31
if for some reason that didn't work it could literally place a phone call on your behalf and the person picking up at
00:29:37
Open Table wouldn't even know that your agent actually isn't a human it's an AI but here's where I think it gets really
00:29:44
crazy is when the phone gets picked up on the other end that could be an AI too pretending to be a human so you could
00:29:51
have two AIS pretend to be humans talking to each other and resolving tasks on your behalf and I I literally I
00:29:57
think that's where it's at it we're definitely moving in this direction but there's a cautionary tale here and I
00:30:03
think that I'll just tell you the real world experience with my customers and what I'm the problems that I'm trying to
00:30:08
solve for them I I think in the last few years we've kind of heard and you know some of it has come from open AI but
00:30:15
especially from Microsoft that we're in this co-pilot world and these co-pilots have universally failed the level of
00:30:22
accuracy the spillage of information the lack of trusted environment co-pilot has
00:30:26
been a complete disaster and that idea that this kind of amount of Technology got you know released and
00:30:34
sold into these very large customers telling them that the all promise of AI is here but didn't do it in a trusted
00:30:41
way didn't do it with the level of accuracy didn't do it with the level of security needed and one of the things
00:30:46
that was interesting because I was with one of the customers would trying to do this exact technique that you're talking
00:30:51
about which is a large telecommunications company in Seattle and what this company did is
00:30:57
take model of nope going to just tell you training a model retraining a model building their own model Mark we have to
00:31:06
have our own models we're going to DIY it we're going to DIY our Ai and it's going to be awesome then we're going to
00:31:13
write our own agents and we're going to do this we're going to do that the other
00:31:15
thing and I'm sitting there and I'm going through it and whatever and I'm then I finally I'm like now show me your
00:31:19
benchmarks and show me all these different pieces and and you know for them it's a bit of a science project and
00:31:24
I've seen this now with a number of our customers that they're kind of DIY and their Ai and you know DIY I think it's
00:31:32
fine if you're like Neil Young and it's homegrown and it's Canada and it's you know Ontario but this is not what you
00:31:39
should be doing with your artificial intelligence but what are you guys using as your foundation model is it llama 2
00:31:45
like what do you guys use we have a lot of our own models our own techniques our
00:31:49
own and then we let you bring in the model that you want but we are all about achieving your accuracy because what
00:31:55
I've seen with these kind of approaches especially the one that you just outlined is that yeah you can get maybe
00:32:01
30 or 40% accuracy you know in this case this customer is 25% you had somebody on
00:32:07
the stage yesterday I won't tell it is who's a common friend of both of ours who tried to take this approach for a
00:32:13
large telecommunications company that he owns and he said he was getting about a
00:32:16
25% accuracy with this homegrown model and I'm like why are you doing that instead in our platform the platform is
00:32:23
building the model for you you're not having to train and retrain your own models you're building your own models
00:32:29
in our platform and we're going to deliver much higher levels of accuracy for you and we're going to deliver AI
00:32:36
this is the AI that you want this is this next generation of AI and I think that we'll have to prove that with
00:32:43
benchmarks and with bake offs and to show customers because the promise is amazing but at a very deep level
00:32:50
customers are going to need you know what you and I have done for the last you know 20 years of our life which is
00:32:55
build professional Enterprise software and delivered to them and a capability And in regards to an agent running
00:33:01
enterprise software I mean you just saw like that was the fundamental business model of adept
00:33:06
which was David Lewan's company you know and that's he built gpt3 then he left open AI to start Adept and this idea to
00:33:13
build agents that are going to drive apps I'm sure that all of those things are going to happen but again you have
00:33:20
to get to a level of accuracy because everyone in this room and you and I we've all had this experience where on
00:33:26
these models and it's like this is not really more than hallucinations and that's no good or as we say here in Los
00:33:34
Angeles no es bueno when it comes to okay Kaiser and you're dealing with health care you know when you're dealing
00:33:42
with health care and you've got a patient and you're reading their medical records you better be delivering more
00:33:47
than 90 or 95% accuracy because the 50% accuracy thing is no good well I can see
00:33:54
you're ready for dream Force I'm trying to go find it I'm testing mat out here a
00:33:58
little bit what do you guys think I'm like are you are you guys excited for the rise of Agents yeah it's going to be
00:34:10
a really big deal I think that everything we've seen so far with llms has been again about reasoning and and
00:34:17
generating but with agents the AI is going to be able to take actions and and they're going to know how to use tools
00:34:24
which until now it's been some the only human I got to tell you a really good story because you're like inspiring me
00:34:28
around uh you know Steve Jobs had a huge impact on my life and at I worked at Apple in 1984 when I was a I was uh in
00:34:37
high school and coming into college and I was an assembly language program I wrote the first Native Assembly Language
00:34:44
um on this Macintosh know the 68,000 assembler and sitting there in the cubes and Steve is running whatever it is and
00:34:54
thank God you know I have this relationship and influenced me so much in my life and then called me on a
00:34:59
series of times and after I started Salesforce gave me really key advice anyway it was 2010 and he calls me come
00:35:07
down here I need to talk to you I'm like what the hell what did I do this time so I go down there to his office
00:35:13
and I always bring a few Salesforce employees with me and I've got some great folks with me and we're sitting
00:35:19
there and he's like I'm going to show you this and I'm like all right let's go and he brings out the iPad and he's got
00:35:25
two of them he's got the big one and the small one and he's like yeah Mark here it is you
00:35:30
know but I don't like the small one I'm only going to have one size you know that I'm like yes sir and he's like uh
00:35:37
listen you know I've been working on this concept for a long time and you know in
00:35:42
2007 um I introduced a iPhone and you know I said thank you for sending me one I love it it's great like but do you
00:35:49
know why now we're doing the iPad I'm like no because I know you had that too in 2007 oh yeah but you know what the
00:35:56
real situation here is that couple I'm like what is it Steve he's like we only have one a team here one a team so we're
00:36:03
only focused on one thing at a time and then he lays out like five or six products on his coffee table and he goes
00:36:10
and we will never have more products than can fit on my coffee table and I'm like well that's really awesome and he's
00:36:16
like I've been focused on 2007 on the iPhone and now I'm gonna zero in and I'm only going to do iPad one focus at a
00:36:24
time remember that Mark that's the way you need to rent Salesforce and I'm like okay is that why
00:36:31
you brought me down here yes you may go and and that's how I feel right now about agent force this is all I am doing
00:36:40
just try to take our company you know we have a great company 38 billion Revenue
00:36:46
75,000 employees hundreds of thousands of customers and one Focus agent force this is the because of what you're
00:36:53
saying this is the moment this is the greatest opport opportunity in the history of enterprise software and it
00:37:00
must be executed with absolute Acuity and excellence and that is what I think we all need to
00:37:08
do you know so so I agree with you I mean I think the agents are going to be huge and Elon say something kind of
00:37:16
similar to the other day um he said we was we got him talking about Optimus you know his his robot heard about the farm
00:37:22
animals I didn't know about the what was was there another part of the presentation he said well he was talking
00:37:26
about he's talking about Optimist and um oh you the the thing thing about these jokes are all each one is kind of dying
00:37:34
very fast it's sad took me a second to to realize that you were talking about his
00:37:40
um but now I got it okay um no he was referring to um it's great how you bring this humor into uh the Allin yes it's
00:37:49
very subtle I understand um so what Elon mentioned that really stuck with me is he said that humanoid
00:37:58
robots the creation of these humanoid robots are the biggest Economic Opportunity in the history of the world
00:38:05
the average person is he making some of them by any chance he is but well it's kind of like you saying that uh agents
00:38:11
are the biggest opportunity in the history of Enterprise sofware let me write that down thank you for letting me
00:38:16
know that it's it strikes me that there's something similar here which isan is a good salesman is that your
00:38:20
point well I'm I'm saying there's an analogy here between he didn't give you the regenerative pitch is that my
00:38:28
well no what here me the point is this is that is that where we're going with AI is it's going to be able to take real
00:38:35
actions and in the case of Optimus is in the physical world and it's going to be
00:38:38
the brain for these humanoid robots in the Enterprise it's basically the brain for these agents I think these things
00:38:45
are actually pretty they're on Parallel tracks I wouldn't say they're competing and these are the droids you're looking
00:38:53
for so I think anyway I think that uh I I think you're right about this opportunity and what I'm saying is I
00:38:59
think it's analogous to what Elon is seeing with robotics I think there's no question and I think that for our
00:39:05
customers they're going to augment their employees they're going to make things lower cost they're going to increase
00:39:09
their revenues they're going to increase their margins we're going to take some customers and just turn them into margin
00:39:14
machines and I think that the opportunity in the Enterprise is unbelievable he's also directly
00:39:20
addressing the consumer Market which I think is very exciting obviously he's an expert in that area and yeah we're about
00:39:27
to move into this new world of AI of droids of all these things and you know it's a bunch of waves of you know where
00:39:34
you know look technology is getting lower cost and easier to use it's a Continuum and we're all rioting that
00:39:41
Continuum this is extremely important but also what's very important is especially as we move into this we all
00:39:48
have to think about what are the values that are going to guide this technology because each of us have seen the movies
00:39:54
we all watched the movies that was the one place where I got the hands to go up right so we know how it can go really
00:40:00
wrong right everybody saw that part of the movie so what are the values what's going to be really important to us will
00:40:07
it be trust is it customer success is it Innovation is it equality is it sustainability is what are the values as
00:40:14
we kind of guide into the next level of the future because those core values that we need to manifest and really
00:40:20
focus on that is I think still out there as a major discussion item it's got to be figured out and that is why we're
00:40:27
very lucky that you are one of the great Visionaries of our industry because you're not just a great entrepreneur and
00:40:33
CEO but you're a great human being so thank you Mark thank you