00:02Chris Anderson: Sam, welcome to TED

00:04Thank you so much for coming

00:06Sam Altman: Thank you

00:09It's an honor

00:11CA: Your company has been releasing crazy, insane new models pretty much every other week, it feels like

00:14I've been playing with a couple of them

00:18I'd like to show you what I've been playing

00:25So, Sora, this is the image and video generator

00:32I asked Sora this: What will it look like when you share some shocking revelations here at TED? You want to see how it imagined it, you know? (Laughter) I mean, not bad, right? How would you grade that? Five fingers on all hands

00:37SA: Very close to what I'm wearing, you

00:41You know, it's good

00:47CA: I've never seen you quite that animated

00:52SA: No, I'm not that animated of a person

00:58CA: So maybe a B-plus, but this one genuinely astounded me when I asked it to come up with a diagram that shows the difference between intelligence and consciousness, like how would you do that? This is what it did

01:04I mean, this is so simple, but it's incredible

01:10What is the kind of process that would allow -- like this is clearly not just image generation, it's linking into the core intelligences that your overall model has

01:21SA: Yeah, the new image generation model is part of GPT-4o, so it's got all of the intelligence in there, and I think that's one of the reasons it's been able to do these things that people really love

01:28CA: I mean, if I'm a management consultant and I'm playing with some of this stuff, I'm thinking, uh oh, what does my future look like? SA: I mean, I think there are sort of two views you can take

01:45You can say, oh, man, it's doing everything I do, what's going to happen to me? Or you can say, like through every other technological revolution in history, OK, now there's this new tool, I can do a lot more

01:56What am I going to be able to do? It is true that the expectation of what we’ll have for Someone in a particular job increases, but the capabilities will increase so dramatically that I think it will be easy to rise to that occasion

02:03CA: So this impressed me too

02:10I asked it to imagine Charlie Brown as thinking of himself as an AI

02:15It came up with this

02:19I thought this was actually rather profound

02:23What do you think? (Laughs) I mean, the writing quality of some of the new models, not just here, but in detail, is really going to a new level

02:36SA: I mean, this is an incredible meta answer, but there's really no way to know if it is thinking that or it just saw that a lot of times in the training set, and Of course, like if you can’t tell the difference, how much do you care? CA: So that's really interesting

02:47We don't know

02:53Isn't there though, like at first glance this looks like IP theft? Like you guys don’t have a deal with the “Peanuts” estate? (Applause) You can clap about that all you want, enjoy

03:04(Laughter and murmuring) I will say that I think the creative spirit of humanity is an incredibly important thing, and we want to build tools that lift that up, that make it so that new people can create better art, better content, write better novels that we all enjoy

03:12I believe very deeply that humans

03:21Will be at the center of that

03:29I also believe that we probably do need to figure out some sort of new model around the economics of creative output

03:39I think people have been building on the creativity of others for a long time

03:50People take inspiration for a long time, but as the access to creativity gets incredibly democratized and people are building off of each other's ideas all the time, I think there are incredible new business models that we and others are excited to explore

03:55Exactly what that's going to look like, I'm not sure

04:00Clearly, there’s some cut and dry stuff, like you can’t copy someone

04:06else's work

04:12But how much inspiration can you take? If you say, "I want to generate art in the style of these seven people, all of whom have consented to that," how do you, like, divvy up how much money goes to each one? These are, like, big questions

04:19But every time throughout history we have put better and more powerful technology in the hands of creators, I think we collectively get better creative output and people do just more amazing stuff

04:24CA: An even bigger question is when they haven't consented to it

04:33In our opening session, Carole Cadwalladr showed, you know, "ChatGPT give a talk in the style of..." Carole Cadwalladr," and sure enough, it gave a talk that wasn't quite as good as the talk she gave, but it was pretty impressive

04:45And she said, "OK, it's great, but I did not consent to this." How are we going to navigate this? Like, isn’t there a way, should it just be people who’ve consented? Or shouldn’t there be a model that somehow says that any named individual in a prompt whose work is then used, they should get something for that? SA: So right now, if you use our image-gen thing and say, I want something in the style of a living artist, it won't do that

04:57But if you say I want it in the style of this

05:10particular like kind of vibe, or this studio or this art movement or whatever, it will

05:22And obviously if you’re like, you know, output a song that is like a copy of a song, it won't do that

05:28The question of like where that line should be and how people say like, this is too much, we sorted that out before with copyright law and kind of what fair use looks like

05:41Again, I think in the world of AI, there will be a new model that we figure out

05:48CA: From the point of view, I mean, creative people are some of the angriest people right now or the most scared people about AI

05:54And the difference between feeling your work is being stolen from you and your future is being stolen from you, and feeling your work is being amplified and can be amplified, those are such different feelings

06:06And if we could shift to the other one, to the second one, I think that really changes how much humanity as a whole embraces all this

06:13SA: Well, again, I would say some creative people are very upset

06:20Some creatives are like, "This is the most amazing tool ever, I'm doing incredible new work." But you know, like it’s definitely a change

06:27And I have a lot of like empathy to people who are just like, "I wish this change weren't happening

06:34I liked the way things were before." CA: But in principle, you can calculate from any given prompt how much..

06:41there should be some way of being able to calculate what percentage of a subscription, revenue, or whatever, goes towards each answer

06:48In principle, it should be possible if one could get the rest of the rules figured out

06:56It's obviously complicated

07:04You could calculate some kind of revenue share, no? SA: If you're a musician and you spend your whole life, your whole childhood, listening to music, and then you get an idea and you go compose a song that is inspired by what you've heard before, but a new Direction, it'd be very hard for you to say, like, this much was from this song I heard when I was 11

07:09CA: That's right

07:13But we're talking here about the situation where someone specifically in a prompt names someone

07:18SA: Well, again, right now, if you try to, like, go generate an image in a named style, we just say that artist is living, we don't do it

07:25But I think it would be cool to figure out a new model where if you say, I want to do it in the name of this artist and they opt in, there's a revenue model there

07:28I think that's a good thing to explore

07:32CA: So, I think the world should help you figure out that model

07:35quickly, and I think it will make a huge difference actually

07:41I want to switch topics quickly

07:47(Applause) The battle between your model and open source, how much were you shaken up by the arrival of DeepSeek? SA: I think open source has an important place

07:53We actually, just last night, hosted our first community session to kind of decide the parameters of our open-source model and how we want to shape it

07:59We're going to do a very powerful open-source model

08:05I think this is important

08:12We're going to do something near the frontier, I think, better than any current open-source model out there

08:17This Will not be all, there will be people who will use this in ways that some people in this room, maybe you or I, don’t like

08:22But there is going to be an important place for open-source models as part of the constellation here

08:27And, you know, I think we were late to act on that, but we're going to do it really well now

08:34CA: I mean, you're spending it seems, like an order, or even orders of magnitude more than DeepSeek allegedly spent, although I know there's controversy around that

08:46Are you confident that the actual better model is going to be recognized? Or are you actually like, isn't this in some ways life-threatening to the notion that, yeah, by going to massive scale, tens of billions of dollars of investment, we can maintain an incredible lead

08:59SA: All day long, I call people and beg them to give us their GPUs

09:03We are so incredibly constrained

09:07Our growth is going like this

09:11DeepSeek launched, and it didn’t seem to impact it

09:14There's other stuff that's happening

09:16CA: Tell us about the growth, actually

09:19You gave me a shocking number backstage there

09:22SA: I have never seen growth in any company, one that I've been involved with or not, like this, like the growth of ChatGPT

09:27It's really fun

09:31I feel Like, great, deeply honored, but it is crazy to live through, and our teams are exhausted and stressed, and we’re trying to keep things up

09:36CA: How many users do you have now? SA: I think the last time we said was 500 million weekly actives, and it is growing very rapidly

09:42CA: I mean, you told me that like doubled in just a few weeks, like in terms of compute or in terms of..

09:49SA: I said that privately, but I guess..

09:53CA: Oh

09:57(Laughter) I misremembered, Sam, I'm sorry

10:01We can edit that out of the thing if you really want to, and no one here would tweet it

10:04SA: It's growing very fast

10:07(Laughter) CA: So you're Confident, you're seeing it grow, take off like a rocket ship, you're releasing incredible new models all the time

10:13What are you seeing in your best internal models right now that you haven't yet shared with the world, but you would love to here on this stage? SA: So first of all, you asked about, are we worried about this model or that model? There will be a lot of intelligent models in the world

10:27Very smart models will be commoditized to some degree

10:38I think we’ll have the best, and for some use you'll want that

10:48But like, honestly, the models are now so smart that for most of the things most people want to Do they're good enough? I hope that'll change over time because people will raise their expectations, but like, if you're kind of using ChatGPT as a standard user, the model capability is very smart, but we have to build a great product, not just a great model

10:59And so there will be a lot of people with great models, and we will try to build the best product, and people want their image-gen, you know, you saw some Sora examples for video earlier

11:05They want to integrate it with all their stuff

11:11We just launched a new feature, it's still called Memory, but it's way better than the Memory before, where this

11:17Model will get to know you over the course of your lifetime, and we have a lot more stuff to come to build like this great integrated product

11:23And, you know, I think people will stick with that

11:30So there will be many models, but I think we will, I hope, continue to focus on building the best defining product in this space

11:38CA: I mean after I saw your announcement yesterday that ChatGPT will know all of your query history, I entered, "Tell me about me, ChatGPT, from all you know." And my jaw dropped, Sam, it was shocking

11:49It knew who I was and all these sort of interests that hopefully mostly were pretty much

12:01appropriate and shareable, but it was astonishing, and I felt the sense of real excitement, a little bit queasy, but mainly excitement, actually, at how much more that would allow it to be useful to me

12:12SA: One of our researchers tweeted, you know, kind of like yesterday or this morning, that the upload happens bit by bit

12:19It’s not, you know, that you plug your brain in one day, but you will talk to ChatGPT over the course of your life, and some day, maybe if you want, it'll be listening to you throughout the day and sort of observing what you're doing, and it'll get to know you and it'll become this

12:32Extension of yourself, this companion, this thing that just tries to, like, help you be the best, do the best you can

12:39CA: In the movie "Her," the AI basically announces that she's read all of his emails and decided he's a great writer and, you know, persuades a publisher to publish him

12:53That might be coming sooner than we think

12:59SA: I don't think it will happen exactly like that, but yeah, I think something in the direction where AI -- you don’t have to just, like, go to ChatGPT or whatever and say, I have a question, give me an answer, but you're getting, like, proactively pushed things that help you, that Make you better or whatever

13:09That does seem like it's soon

13:19CA: So what have you seen that's coming up, internally, that you think is going to blow people's minds? Give us at least a hint of what the next big jaw dropper is

13:29SA: The thing that I'm personally most excited about is AI for science at this point

13:35I am a big believer that the most important driver of the world and people's lives getting better and better is new scientific discovery

13:41We can do more things with less, we sort of push back the frontier of what's possible

13:46We're starting to hear a lot from scientists with our latest models that They're actually just more productive than they were before

13:53That's actually mattering to what they can discover

14:00CA: What's the plausible near-term discovery, like, room temperature -- SA: Superconductors? CA: Superconducting, yeah

14:06Is that possible? SA: I don't think that's prevented by the laws of physics, so it should be possible, but we don't know for sure

14:12I think you'll start to see some meaningful progress against disease with AI-assisted tools

14:26You know, physics maybe takes a little bit longer, but I hope for it

14:30So that's like, one direction

14:34Another that I think is big is Starting pretty soon, like in the coming months

14:40Software development has already been pretty transformed

14:46Like, it’s quite amazing how different the process of creating software is now than it was two years ago, but I expect like another move that big in the coming months as agentic software engineering really starts to happen

14:52CA: I've heard engineers say that they've had almost like religious-like moments with some of the new models where suddenly, they can do in an afternoon what would have taken them two years

15:03SA: Yeah, it's like mind -- it really like, that’s been one of my big “feel the AGI” moments

15:10CA: But talk about what is the scariest thing that you've seen, because, like, outside, a lot of people picture you as, you know, you have access to this stuff, and we hear all these rumors coming out of AI, and it's like, "Oh my God, they've seen consciousness," or "They've seen AGI," or "They've seen some kind of apocalypse coming." Have you seen, has there been a scary moment when you've seen something internally and thought, "Uh oh, we need to pay attention to this?" SA: There have been like moments of awe, and I think with that is always like, how far is this going to go? What is this going to be? But there's no, like, we don't secretly have, we're not secretly sitting on a conscious model or something that's capable of self-improvement or anything like that

15:31You know, I..

15:51people have very different views of what the big AI risks are going to be, and I myself have, like, evolved on thinking about where we're going to see those

16:12I continue to believe there will come very powerful models that people can misuse in big ways

16:19People talk a lot about the potential for new kinds of bioterror, models that can present, like, a real cybersecurity challenge, models that are capable of self-improvement in a way that leads to some sort of loss of control, so I think there are big risks there

16:32And then there's a lot of other stuff, which honestly is kind of what I think many people mean, where people talk about disinformation or models saying things that they don't like or things like that

16:45CA: Sticking with the first of those, do you check for that internally before release? SA: Of course, yeah, so we have this preparedness framework that outlines how we do that

16:50CA: I mean, you've had some departures from your safety team

16:59How many people have departed, why have they left? SA: We Have, I don't know the exact number, but there are clearly different views about AI safety systems

17:04I would really point to our track record

17:09There are people who will say all sorts of things

17:14You know, something like 10 percent of the world uses our systems now a lot, and we are very proud of the safety track record

17:19CA: But track record isn't the issue in a way -- SA: No, it kind of is

17:27CA: Because we're talking about an exponentially growing power where we fear that we may wake up one day and the world is ending, so it's really not about track record, it's about plausibly saying that the pieces are in place

17:33to shut things down quickly if we see a danger

17:40SA: Yeah, no, of course, of course that's important

17:47You don't, like, wake up one day and say, "Hey, we didn't have any safety process in place, now we think the model is really smart, so now we have to care about safety." You have to care about it all along this exponential curve

18:00Of course the stakes increase, and there are big challenges, but the way we learn how to build safe systems is this iterative process of deploying them to the world, getting feedback, while the stakes are relatively low, learning about, like, this is something we have to address

18:14And I think as we move into these agentic systems, there's a whole big category of new things we have to learn to address

18:19CA: So let's talk about agentic systems and the relationship between that and AGI

18:24I think there's confusion out there, I'm confused

18:29So artificial general intelligence, it feels like ChatGPT is already a general intelligence

18:33I can ask it about anything, and it comes back with an intelligent answer

18:38Why isn't that AGI? SA: It doesn't..

18:41First of all, you can't ask it anything

18:44That's very nice of you to say, but there's a lot of things that it's still embarrassingly bad at

18:47But Even if we fixed those, which hopefully we will, it doesn't continuously learn and improve

18:53It can't go get better at something that it's currently weak at

18:59It can't go discover new science and update its understanding and do that

19:06And it also kind of can't, even if we lower the bar, it can't just sort of do any knowledge work you could do in front of a computer

19:11I actually, even without the sort of ability to get better at something it doesn't know yet, I might accept that as a definition of AGI

19:18But the current systems, you can't say like, hey, go do this task for my job, and it goes off and clicks around the

19:24Internet and calls someone and looks at your files and does it

19:34And without that, it feels definitely short of it

19:43CA: I mean, do you guys have internally a clear definition of what AGI is, and when do you think that we may be there? SA: It's like the joke, if you’ve got 10 OpenAI researchers in a room and ask to define AGI, you’d get 14 definitions

19:53CA: That's worrying, though, isn't it? Because that has been the mission initially, “We’re going to be the first to get to AGI

19:57We'll do so safely

20:02But we don't have a clear definition of what it is." SA: I was going to finish the answer

20:11CA: Sorry

20:21SA: What I think

20:31Matters, though, and what people want to know is not where is this one, you know, magic moment of, “We finished.” But given that what looks like is going to happen is that the models are just going to get smarter and more capable and smarter and more capable, on this long exponential, different people will call it AGI at different points, but we all agree it’s going to go way, way past that, you know, to whatever you want to call these systems that get much more capable than we are

20:41The thing that matters is how do we talk about a system that is safe through all of these steps and beyond, as the system gets more

20:46capable than we are, as the system can do things that we don't totally understand

20:52And I think more important than when is AGI coming and what's the definition of it, it's recognizing that we are in this unbelievable exponential curve

20:59And you can, you know, say this is what I think AGI is

21:07You can say you think this is what you think AGI is

21:15Someone else can say superintelligence is out here, but we're going to have to contend and get wonderful benefits from this incredible system

21:22And so I think we should shift the conversation away from what's the AGI moment to a recognition that, like, this thing is Not going to stop, it's going to go way beyond what any of us would call AGI, and we have to build a society to get the tremendous benefits of this and figure out how to make it safe

21:37CA: Well, one of the conversations this week has been that the real change moment is -- I mean, AGI is a fuzzy thing, but what is clear is agentic AI -- when AI is set free to pursue projects on its own and to put the pieces together -- you’ve actually, you've got a thing called Operator which starts to do this, and I tried it out

22:01You know, I wanted to book a restaurant, and it's kind of incredible

22:06It kind of can go ahead and do it, but This is what it said

22:11You know, it was an intriguing process, and, you know, “Give me your credit card” and everything else, and I declined on this case to go forward

22:17But I think this is the challenge that people are going to have

22:24It's kind of like, it's an incredible superpower

22:28It's a little bit scary

22:33And Yoshua Bengio, when he spoke here, said that agentic AI is the thing to pay attention to

22:37This is when everything could go wrong as we give power to AI to go out onto the internet to do stuff

22:44I mean, going out onto the internet was always, in the sci-fi stories, the moment where, you know, escape happened and potential -- things could go horribly wrong

22:53How do you both release agentic AI and have guardrails in place that it doesn't go too far? SA: First of all, obviously you can choose not to do this and say, I don't want this

23:01I'm going to call the restaurant and read them my credit card over the phone

23:07CA: I could choose, but someone else might say, “Oh, go out, ChatGPT onto the internet at large and rewrite the internet to make it better for humans,” or whatever

23:14SA: The point I was going to make is just with any new technology, it takes a while for people to get comfortable

23:21I remember when I Wouldn't put my credit card on the internet because my parents had convinced me someone was going to read the number, and you had to fill out the form and then call them

23:29And then we kind of all said, OK, we'll build anti-fraud systems, and we can get comfortable with this

23:38I think people are going to be slow to get comfortable with agentic AI in many ways, but I also really agree with what you said, which is that even if some people are comfortable with it and some aren't, we are going to have AI systems clicking around the internet, and this is, I think, the most interesting and consequential safety

23:47challenge we have yet faced

23:57Because AI that you give access to your systems, your information, your ability to click around on your computer, now, those, you know, when AI makes a mistake, it's much higher stakes

24:04It is the gate on -- so we talked earlier about safety and capability

24:11I kind of think they're increasingly becoming one-dimensional, like a good product is a safe product

24:18You will not use our agents if you do not trust that they’re not going to like empty your bank account or delete your data or who knows what else

24:25And so people want to use agents that they can really trust, that are really safe, and I think we are gated on our ability to make progress on our ability to do that, but it's a fundamental part of the product

24:39CA: In a world where agency is out there and say that, you know, maybe it’s open models are widely distributed and someone says, "OK, AGI, I want you to go out onto the internet and, you know, spread a meme however you can that X people are evil,” or whatever it is

24:53It doesn't have to be an individual choice

25:07A single person could let that agent out there, and the agent could decide, "Well, in order to execute on that function, I've got to copy myself everywhere," and, you know

25:20Are there red lines that you have clearly drawn internally, where you know what the danger moments are, and that we cannot put out something that could go beyond this? SA: Yeah, so this is the purpose of our preparedness framework, and we'll update that over time, but we’ve tried to outline where we think the most important danger moments are, or what the categories are, how we measure that, and how we would mitigate something before releasing it

25:33I can tell from the conversation you're not a big AI fan

25:46CA: Actually, on the contrary, I use it every day

25:50I'm awed by it

25:53I think this is an incredible time

25:57To be alive

26:00I wouldn't be alive any other time, and I cannot wait to see where it goes

26:04We've been holding..

26:07I think it's essential to hold..

26:10like we can’t divide people into those camps

26:13You have to hold a passionate belief in the possibility, but not be overseduced by it because things could go horribly wrong

26:16(Applause) SA: What I was going to say is I totally understand that

26:25I totally understand looking at this and saying this is an unbelievable change coming to the world

26:38And, you know, maybe I don't want this, or maybe I love parts of it

26:51Maybe I love talking to ChatGPT, but I worry about what's going to happen to art, and I worry about the pace of change, and I worry about these agents clicking around the internet, and maybe, on balance, I wish this weren't happening, or maybe I wish it were happening a little slower, or maybe I wish it were happening in a way where I could pick and choose what parts of progress were going to happen, and I think, the fear is totally rational, sort of, the anxiety is totally rational

27:05We all have a lot of it, too, but, A, there will be tremendous upside, obviously, you know, you use it every day, you like it

27:16B, I really believe that society figures out, over time, with some big mistakes along the way, how to get technology right

27:23And C, this is going to happen

27:28This is like a discovery of fundamental physics that the world now knows about, and it's going to be part of our world

27:32And I think this conversation is really important

27:36I think talking about these areas of danger [is] really important to talk about

27:40New economic models are really important, but we have to embrace this with, like, caution but not fear, or we will get run by with other people that use AI to do better

27:48CA: You've actually been one of the most eloquent proponents of safety

27:58You testified In the Senate, I think you said basically that we should form a new safety agency that licenses any effort, i.e., it will refuse to license certain efforts

28:11Do you still believe in that policy proposal? SA: I have learned more about how the government works

28:21I don't think this is quite the right policy proposal

28:32CA: What is the right policy proposal? SA: But, I do think the idea that as these systems get more advanced and have legitimate global impact, we need some way, you know, maybe the companies themselves put together the right framework or the right sort of model for this, but we need some way that Very advanced models have external safety testing, and we understand when we get close to some of these danger zones

28:40I very much still believe in that

28:49CA: Struck me as ironic that a safety agency might be what we want, and yet agency is the very thing that is unsafe

29:03There's something odd about the language there, but anyway

29:17SA: Can I say one more thing on that? I do think this concept of we need to define rigorous testing for models, understand what the threats that we, collectively, society, most want to focus on, and make sure that as models are getting more capable, we have a system where we all get To understand what's being released in the world, I think this is really important, and I think we’re not far away from models that are going to be of great public interest in that sense

29:30CA: So Sam, I asked your o1-pro reasoning model, which is incredibly -- SA: Thank you for the 200 dollars

29:35CA: (Laughs) 200 dollars a month, it's a bargain at the price

29:40I said, what is the single most penetrating question I could ask you? It thought about it for two minutes, two minutes

29:46You want to see the question? SA: I do

29:52CA: "Sam, given that you're helping create technology that could reshape the destiny of our entire species, who granted you (or anyone) the moral authority to do that?" (Laughter) "And how are you personally accountable if you're wrong?" SA: No, it was good

29:59CA: That was impressive

30:06SA: You've been asking me versions of this for the last half hour

30:12What do you think? (Laughter and applause) CA: What I would say is this

30:19Here's my version of that question

30:25SA: But no answer? CA: What was your question to me? SA: How would you answer that one? CA: In your shoes? SA: Or as an outsider

30:29CA: I don't know

30:33I am puzzled by you

30:37I’m kind of awed by you, because you’ve built one of the most astonishing things out there

30:40There are two narratives about you out there

30:43One is, you know, you are this incredible visionary who's done the impossible, and you shocked the world

30:46With far fewer people than Google, you came out with something that was much more powerful than anything being done

30:53I mean, it is amazing what you've built

31:00But the other narrative is that you have shifted ground, that you've shifted from being OpenAI, this open thing, to the allure of building something super powerful

31:03And, you know, you’ve lost some of your key people

31:06There's a narrative out there

31:09Some people believe that

31:12You're not to be trusted in this space

31:19I would love to know who you are

31:26What is your narrative about yourself? What are your core values, Sam, that can give us, the world, confidence that someone with so much power here is entitled to it? SA: Look, I think like anyone else, I'm a nuanced character that doesn't reduce well to one dimension here

31:33You know, probably some of the good things are true, and probably some of the criticism is true

31:38In terms of OpenAI, our goal is to make AGI and distribute it, make it safe, for the broad benefit of humanity

31:44I think by all accounts, we have done a lot in that

31:49direction

31:53Clearly, our tactics have shifted over time

31:56I think we didn't really know what we were going to be when we grew up

32:06We didn't think we would have to build a company around this

32:16We learned a lot about how it goes and the realities of what these systems were going to take from capital, but I think we've been, in terms of putting incredibly capable AI with a high degree of safety in the hands of a lot of people and giving them tools to sort of do whatever amazing things they're going to do, I think it'd be hard to give us a bad grade on that

32:24I do think it's fair that we should be open sourcing more

32:32I think it was reasonable for all of the reasons that you asked earlier, as we weren't sure about the impact these systems were going to have and how to make them safe, that we acted with precaution

32:40I think a lot of your questions earlier would suggest at least some sympathy to the fact that we've operated that way

32:46But now I think we have a better understanding, as a world, and it is time for us to put very capable open systems out into the world

32:55If you invite me back next year, you will probably yell at me for somebody who has misused these open-source systems, and say, "Why did you do that?" You know, "You "Should have not gone back to your open roots." But, you know, there's trade offs in everything we do, and we are one player, one voice, in this AI revolution, trying to do the best we can and kind of steward this technology into the world in a responsible way

33:08We've definitely made mistakes, we'll definitely make more in the future

33:22On the whole, I think we have, over the last almost decade, it’s been a long time now, you know, we have mostly done the thing we’ve set out to do

33:29We have a long way to go in front of us

33:36Our tactics will shift more in the future, but adherence to this sort of mission and what we're Trying to do, I think, very strong

33:42(Applause) CA: You posted this -- well, OK, so here's the Ring of Power from "Lord of the Rings." Your rival, I will say, not your best friend at the moment, Elon Musk, claimed that, you know, he thought that you'd been corrupted by the Ring of Power, an allegation that, by the way -- (Laughter) SA: Hi, Steve

33:55CA: An allegation that could be applied to Elon as well, you know, to be fair

34:08But I'm curious, people, you have -- SA: I might respond

34:24I'm thinking about it

34:39I might say something

34:55(Laughter) CA: It's in everyone's mind, as we see technology CEOs get more powerful, get richer, is can they handle it, or does it become irresistible? Does the power and the wealth make it impossible to sometimes do the right thing, and you just have to cling tightly to that ring? What do you think? I mean, do you feel that ring sometimes? SA: How do you think I'm doing, relative to other CEOs, that have gotten a lot of power and changed how they act or done a bunch of stuff in the world, like how do you think? (Applause) CA: You have a beautiful..

35:10you are not a rude, angry person who comes out and says aggressive things to other people

35:14SA: Sometimes I do that

35:18That's my single vice, you know? (Laughter) CA: No, I think in the way that you personally conduct yourself, it's impressive

35:22I mean, the question some people ask is, is that the real you or, you know, is there something else going on? SA: No, I'll take the feedback

35:30You put up the Sauron Ring of Power or whatever that thing is, so I'll take the feedback

35:38What is, like, something I have done where you think I've been corrupted by power? CA: I think the fear is that just the transition of OpenAI to a for-profit model is, you know, some people say, well, there you go

35:49You got corrupted by the desire for wealth

35:53You know, at one point there was going to be no equity in it

35:56It will make you fabulously wealthy

35:59By the way, I don't think that is your motivation, personally

36:03I think you want to build stuff that is insanely cool, and what I worry about is the competitive feeling that you see other people doing it, and it makes it impossible to develop at the right pace

36:14But you tell me, if you don't feel that, like, what..

36:20so few people in the world have the kind of capability and potential you have, we don't know what it feels like

36:25What does it feel like? SA: Shockingly, the same as before

36:31I think you can get used to anything step by step

36:37I think if I Were like transported from 10 years ago to right now, all at once, it would feel very disorienting, but anything does become sort of the new normal, so it doesn't feel any different

36:43And it's strange to be sitting here talking about this, but like, you know, the monotony of day-to-day life, which I mean in the best possible way, feels exactly the same

36:54CA: You're the same person

36:59SA: I mean, I'm sure I'm not in all sorts of ways, but I don't feel any different

37:04CA: This was a beautiful thing you posted, your son

37:09I mean, that last thing you said there, "I've never felt love like this," I think any parent in The room so knows that feeling, that wild biological feeling that humans have and AIs never will, of you’re holding your kid

37:22And I'm wondering whether that's changed how you think about things like if, you know, say, here's a red box, here's a black box with the red button on it, you can press that button and you give your son likely the most unbelievable life, but also you inject a 10 percent chance that he gets destroyed

37:44Do you press that button? SA: In the literal case, no

37:51If the question is, do I feel like I'm doing that with my work, the answer is, I also don't feel like that

37:55Having a kid changed a lot

37:58of things

38:02And by far the most amazing thing that has ever happened to me, like everything everybody says is true

38:05A thing my cofounder Ilya said once is, I don’t know

38:12This is a paraphrase, something like, "I don't know what the meaning of life is, but for sure it has something to do with babies." And it's like, unbelievably accurate

38:24It changed how much I’m willing to like spend time on certain things and like the kind of cost of not being with my kid is just like crazily high

38:30And I..

38:36But, you know, I really cared about like not destroying the world before

38:40I really care about it now

38:43I didn't need a kid

38:47For that part

38:51(Applause) I mean, I definitely think more about, like, what the future will be like for him in particular, but I feel a responsibility to do the best thing I can for the future of everybody

38:54CA: Tristan Harris gave a very powerful talk here this week in which he said that the key problem, in his view, was that you and your peers in these other models all feel, basically, that the development of advanced AI is inevitable, that the race is on, and that there is no choice but to try and win that race and to do so as responsibly as you can

39:20And maybe there’s a scenario where your superintelligent AI Can act as a brake on everyone else's or something like that

39:28But that the very fact that everyone believes it is inevitable, that is a pathway to serious risk and instability

39:41Do you think that you and your peers do feel that it's inevitable, and can you see any pathway out of that where we could collectively agree to just slow things down a bit, have society as a whole weigh in a bit and say, no, you know, we don't want this to happen quite as fast, it's too disruptive? SA: First of all, I think people slow things down all the time because the technology is not ready, because something's not safe enough, Because something doesn't work

40:15There are, I think, all of the efforts, hold on things, pause on things, delay on things, don't release certain capabilities

40:21So I think this happens, and again, this is where I think the track record does matter

40:27If we were rushing things out and there were all sorts of problems, either the product didn’t work as people wanted it to, or there were real safety issues or other things there, and I will come back to a change we made, I think you could do that

40:32There is communication between most of the efforts, with one exception

40:37I think all of the efforts care a lot about AI

40:42safety

40:47And I think that -- CA: Who's the exception? SA: I'm not going to say

40:52And I think that there's really deep care to get this right

40:57I think the caricature of this as just like this crazy race or sprint or whatever, misses the nuance of people are trying to put out models quickly and make great products for people, but people feel the impact of this so incredibly that..

41:13you know, I think if you could go sit in a meeting in OpenAI or other companies, you'd be like, oh, these people are really kind of caring about this

41:30Now we did make a change recently to how we think about one part of what's Traditionally, it's been understood as safety, which is, with our new image model, we've given users much more freedom on what we would traditionally think about as speech harms

41:46You know, if you try to get offended by the model, will the model let you be offended? And in the past, we've had much tighter guardrails on this, but I think part of model alignment is following what the user of a model wants it to do within the very broad bounds of what society decides

42:06So if you ask the model to depict a bunch of violence or something like that, or to sort of reinforce some stereotype, there's a question of whether or Not it should do that, and we're taking a much more permissive stance

42:22There's a place where that starts to interact with real-world harms that we have to figure out how to draw the line for, but, you know, I think there will be cases where a company says, OK, we've heard the feedback from society

42:33People really don't want models to censor them in ways that they don't think make sense

42:39That's a fair safety negotiation

42:45CA: But to the extent that this is a collective, a problem of collective belief, the solution to those kinds of problems is to bring people together and meet at one point and make a Different agreement

42:51If there was a group of people, say, here or out there in the world, who were willing to host a summit of the best ethicists, technologists, but not too many people, small, and you and your peers to try to crack what agreed safety lines could be across the world, would you be willing to attend? Would you urge your colleagues to come? SA: Of course, but I'm much more interested in what our hundreds of millions of users want as a whole

43:24I think a lot of the room has historically been decided in small elite summits

43:30One of the cool new things about AI is our AI can talk to everybody on Earth

43:37And we can learn the collective value preference of what everybody wants, rather than have a bunch of people who are, like, blessed by society to sit in a room and make these decisions, I think that's very cool

43:43(Applause) And I think you will see us do more in that direction

43:51And when we have gotten things wrong, because the elites in the room had a different opinion about what people wanted for the guardrails on image-gen than what people actually wanted, and we couldn't point to real-world harm, so we made that change

43:58I'm proud of that

44:02(Applause) CA: There is a long track record of unintended

44:07consequences coming out of the actions of hundreds of millions of people

44:12SA: Also 100 people in a room making a decision

44:17CA: And the hundreds of millions of people don't have control over, they don't necessarily see what the next step could lead to

44:22SA: I am hopeful that -- that is totally accurate and totally right -- I am hopeful that AI can help us be wiser, make better decisions, can talk to us, and if we say, hey, I want thing X, you know, rather than like, the crowds spin that up, AI can say, hey, totally understand that's what you want

44:38If that's what you want at the end of this conversation, you're In control, you pick

44:43But have you considered it from this person's perspective, or the impact it will have on this? I think AI can help us be wiser and make better collective governance decisions than we could before

44:49CA: We're out of time

44:57Sam, I'll give you the last word

45:04What kind of world do you believe, all things considered, your son will grow up into? SA: I remember -- it's so long ago now, I don't know when the first iPad came out

45:12Is it like 15 years, something like that? I remember watching a YouTube video at the time, of like a little toddler sitting in a doctor's office waiting room or something, And there was a magazine, like one of those old, you know, glossy-cover magazines, and the toddler had his hand on it and was going like this and kind of angry

45:21And to that toddler, it was like a broken iPad

45:31And he never, she never thought of a world that didn't have, you know, touch screens in them

45:41And to all the adults watching this, it was this amazing thing because it was like it's so new, it's so amazing, it's a miracle

45:47Of course, you know, magazines are the way the world works

45:54My kid, my kids hopefully, will never be smarter than AI

45:57They will never grow up in a world where products and services are

46:01Not incredibly smart, incredibly capable

46:05They will never grow up in a world where computers don't just kind of understand you, and do, you know, for some definition of whatever you can imagine, whatever you can imagine

46:15It'll be a world of incredible material abundance

46:24It'll be a world where the rate of change is incredibly fast and amazing new things are happening, and it’ll be a world where, like individual ..

46:33ability, impact, whatever, is just so far beyond what a person can do today

46:39I hope that my kids and all of your kids will look back at us with some like pity and nostalgia and be like, "They lived such horrible lives

46:43They were so limited

46:47The world sucked so much." I think that's great

46:51(Applause) CA: It's incredible what you've built

46:58It really is, it's unbelievable

47:06I think over the next few years, you're going to have some of the biggest opportunities, the biggest moral challenges, the biggest decisions to make of perhaps any human in history, pretty much

47:10You should know that everyone here will be cheering you on to do the right thing

47:15SA: We will do our best, thank you very much

47:19CA: Thank you for coming to TED

47:22(Applause) Thank you

47:26SA: Thank you very much.