Search Captions & Ask AI

OpenAI Battles Safety Concerns and High-Profile Exits | Pivot

May 21, 2024 / 08:32

This episode discusses the recent resignation of Yan LeCun from OpenAI, the company's focus on product development over safety, and the implications of high-profile exits.

Yan LeCun, the head of the super alignment team, left OpenAI citing a shift in the company's safety culture. He expressed concerns that safety processes have been overshadowed by the push for new products.

The episode includes commentary on OpenAI's decision to dissolve the super alignment team and integrate its efforts into broader research. Sam Altman and co-founder Greg Brockman shared their views on the future of AI and the risks associated with AGI.

Discussion also touches on the company's stringent offboarding agreements, which included non-disclosure and non-disparagement clauses. The hosts reflect on the implications of these policies for departing employees.

The conversation highlights the tension between profit motives and safety considerations within OpenAI, questioning whether the company can balance these competing interests.

TLDR

OpenAI faces turmoil as Yan LeCun resigns, raising concerns about safety culture amid rapid product development.

Episode

8:32
00:00:00
things are getting messy at open AI again oh man is this the most like telen Nolla of a Company the company had other
00:00:06
high-profile departure again this week with the resignation of yan Leica uh the head of super alignment which is the
00:00:14
team focused on AI safety that's the words we're going to be all super aligned uh Leica explained his departure
00:00:20
in a series of social media posts saying in part that open AI safety culture and
00:00:24
processes have taken a backseat to shiny products and there's been a bunch of shiny products they showed off last week
00:00:30
uh open it's not the things are not unrelated um open AI has dissolved that super alignment team the company told
00:00:37
Bloomberg the group will be integrated across research efforts to help achieve safety goals uh Sam Alman put out a
00:00:44
statement and also did open a co-founder Greg Brockman uh sharing their view of the future they said the company has
00:00:50
quote raised awareness of the risks and opportunities of AGI so the world can better prepare for it whether they're
00:00:55
preparing for it or not is a big question there's been at least 11 high-profile exits in the last few
00:01:00
months um you know this is an issue again of speed versus safety they have been rolling out the products because
00:01:06
they're deathly terrified of getting rolled over by the big companies I can feel it I can feel such a Netscape
00:01:12
moment for them um you know uh we'll see uh we'll see what'll happen here but it's definitely a company still shaking
00:01:20
off or dealing with these issues that they've had these two different types of people who are um involved in this
00:01:27
company which is some that are think this is a a risk to humanity others who are like calm the [ __ ] down let's make
00:01:33
some stuff and we'll figure it out later um uh one of the things that got a lot of reporting was open aa's offboarding
00:01:41
agreements that have non-disclosure and non-disparagement Provisions not uncommon but theirs were particularly
00:01:46
stringent if a departy employee violated these Provisions they were in danger of
00:01:49
losing all their vested Equity according to Vox Sam mman confirmed in a tweet there was a provision about potential
00:01:55
Equity cancellation for departing employees but it was never en forced the company is currently changing that
00:02:00
language uh it sounds like they're just like just tough customers on that thing um it is further than other people do
00:02:07
it's usually more Talent friendly in general in Silicon Valley Scott what are your thoughts on all this when Ilia was
00:02:14
part of the board yeah when when he was part of the board that fired Sam Alman if you're going to stab the prince you
00:02:21
better kill him when he came back Ilia became the information age equivalent of prosan he was dead man walking he just
00:02:27
wasn't going to survive oh no feelings for firing me come on water under the bridge that just wasn't going to happen
00:02:34
and this is similar to meta the fastest way to get a severance check is to go to
00:02:38
work for the trust and safety team because every once in a while in response to real heat they'll pretend to
00:02:43
give a good godamn and they'll create a trust and safety team I doubt Mark listens to them or cares about them and
00:02:48
then under the cover of dark fires most of them I like the fact that open AI is becoming more like what they really are
00:02:54
and that is they are a for-profit company and they're not pretending tough mothers and they're not pretending to be
00:02:59
any anything I'd rather them be like New Yorkers that's that's one of the reasons
00:03:02
I love New York versus doing business in California is they don't pretend to be something they aren't
00:03:10
and this is a four profit companies and open AI are going to be so good at making profit they shouldn't be trusted
00:03:17
to do anything else and the fact and first off and also well except they were founded with a slightly different idea
00:03:23
but go ahead yeah and then when they took 11 billion those people wanted their money back so they should have
00:03:29
never taken that money but the one thing the one compensatory thing here is that
00:03:34
any group of people that decides to call themselves super alignment should be fired that is I thought you'd like that
00:03:40
is those people should endure a certain amount of ridicule and pain what are your thoughts well you know AI is going
00:03:46
to kill us Scott I mean it's a really interesting thing because these people are you know there's a group over in
00:03:51
anthropic that are much more concerned in that regard and I think they have a right to be absolutely I think listen
00:03:58
I've always been a safety let why are you not paying attention to any bit of safety from the very get-go and so I
00:04:04
would naturally be affiliated with the super alignment people at the same time for them to think this is anything other
00:04:10
than a for-profit institution is kind of well then go on a board go on a you know
00:04:16
go to Stanford and and become a high-profile naysayer of these things or write a book like burnbook right um
00:04:22
because they're inside these companies I think that because of the amount of money here there's just no way people
00:04:27
aren't going to be aligned around the money making and you know some people are like oh you like Sam mman I said I
00:04:33
said yes but he's a tough [ __ ] I was like are you kidding he's so aggressive he's so much like all the
00:04:38
people he you know he's just like he's not like Elon because elon's a toxic piece of [ __ ] sometimes and most of the
00:04:45
time uh I don't consider Sam like that but he is interested in not having this company die you know he is interested in
00:04:52
making it the most dominant I have no question he's hyper aggressive and just as feral as the rest of them and so um
00:05:01
you know they're going to have these things because what they want to do is pretend that they care about this safety
00:05:06
stuff which they do peripherally you know more than other people I guess they bring it up more but they don't they
00:05:12
just don't care about that issue and they're not they're not they're not going to be around to see the machines
00:05:17
eating Us Alive or something like you know what I mean like it's it's not their job it's it's well but it this is
00:05:23
the problem its birth was like it was like two it was like a hippie parent and a a non- hippie parent right and they're
00:05:32
fighting forever this group these groups of people are fighting forever and so they are never going to be this whole
00:05:37
company isn't going to be in alignment it started out out of alignment and they're never going to I know that I'm
00:05:43
using a term about the idea of making sure it's safe but they will never this company will never it might be what
00:05:49
kills them right it might be what kills them because they're going to get unusual it's like Google saying don't be
00:05:55
evil that was this one big mistake like why did they do that right because they're evil not evil that's too far but
00:06:01
you know what I mean like that that that whole cosplaying about being heroic has
00:06:06
always been a problem for that's not these companies always figure out they're like okay unless we're re Ben
00:06:11
and Jerry's let's be honest folks we all want our own big home we all want to take care of our kids we all want a
00:06:17
broader selection set of mates than we deserve so let's let's stop let's stop pretending where it's dangerous is that
00:06:24
we keep thinking that Sam Alman is actually a better generation of leaders we don't need to be as worried about AI
00:06:31
because Sam's in charge and he'll speak in hush tones and say how concerned he is and what it does is it dampens the
00:06:36
urgency in the need to elect people who can craft legislation to regulate these guys that's right it's not Sam's job
00:06:42
that's exactly right his Sam is doing his job he's firing people who get in the way of him making being the no if
00:06:51
Sam were to lose to to gemini or to llama or whatever or xai they would say yeah but he was more ethical and more
00:07:01
concerned and we love him for that he's not going to get a statue for that let's
00:07:06
just say there have to be ethical things if you're working for this kind like it's working for Palante here you're
00:07:12
going to make defense department stuff don't work there like that you know or you know the Google employees that's
00:07:17
different because Google sort of gave them an in to to complain because they were like we're better than this but
00:07:22
they're really not better than this right so I think when you you sort of you know you you per have a performative
00:07:28
nature of being being heroic you're going to be slapped later cuz you're going to let people down period and it's
00:07:34
going to be very clear and again the reason my first line in my book was and so it was capitalism after all sticks
00:07:41
that's what it is that's what's happening here and I think you all should go off and form a group of people
00:07:46
that's that scares the [ __ ] B Jesus out of of the potential and you go up to Congress and you march in those offices
00:07:52
and you explain to them why they need to make legislation that's what that's what
00:07:55
you need to do I think when people all backed Sam a lot of people are like oh they love them I'm like no they love the
00:08:01
money and they want to make money here and they you know and and they want to make they want to be not just make money
00:08:07
they want to be at the coolest company making the [ __ ] they want to be at the winning company that makes them Rich
00:08:12
yeah and also is the coolest company right that's more than that more than that anyway we'll see what happens I
00:08:18
thought Yan's um I thought his his series of tweets was interesting but it doesn't really I'm sorry Yan do
00:08:26
something else

Badges

This episode stands out for the following:

  • 70
    Most shocking
  • 70
    Biggest twist
  • 60
    Most intense
  • 60
    Best concept / idea

Episode Highlights

  • High-Profile Departures at OpenAI
    OpenAI faces turmoil with multiple high-profile exits, raising concerns about its safety culture.
    “There's been at least 11 high-profile exits in the last few months.”
    @ 00m 57s
    May 21, 2024
  • Conflict of Speed vs. Safety
    The tension between rapid product rollout and safety concerns is palpable at OpenAI.
    “This is an issue again of speed versus safety.”
    @ 01m 03s
    May 21, 2024
  • Sam Altman's Leadership Style
    Sam Altman's aggressive approach to leadership raises questions about ethics in AI development.
    “He's hyper aggressive and just as feral as the rest of them.”
    @ 04m 55s
    May 21, 2024

Episode Quotes

  • It's definitely a company still shaking off or dealing with these issues.
    OpenAI Battles Safety Concerns and High-Profile Exits | Pivot
  • AI is going to kill us.
    OpenAI Battles Safety Concerns and High-Profile Exits | Pivot
  • This company will never be in alignment.
    OpenAI Battles Safety Concerns and High-Profile Exits | Pivot
  • Capitalism after all sticks, that's what's happening here.
    OpenAI Battles Safety Concerns and High-Profile Exits | Pivot

Key Moments

  • High-Profile Exits00:06
  • Safety Culture Concerns00:22
  • Speed vs. Safety01:03
  • Aggressive Leadership04:55

Tension Over Time

Words per Minute Over Time

Vibes Breakdown