00:00I think back to when I first started learning coding

00:02I was the kid that sat in the back of math class in middle school, and I had my little TI-83 Plus calculator, and I would just program it with BASIC

00:12Cuz at some point, I realized that I can actually program the answers for the math test into the calculator

00:20[Music] Hey, I'm Alex

00:24I lead Cloud Relations here at Anthropic

00:26Today, we're going to be talking about Cloud Code and the future of software engineering

00:30And I'm joined by my colleague, Boris

00:33I'm Boris

00:36I'm a member of the technical staff here at Anthropic and creator of Cloud Code

00:39A lot has happened in the past 12 months, and things are moving very, very fast, especially in the coding domain

00:42For folks that, you know, maybe aren't following the news every single day or even staying on top of the latest, and I have trouble myself sometimes

00:50Can you kind of catch us up here on what's happened and where are we standing currently? Yeah, a year ago, coding was totally different than what it is today

01:01A year ago, if you wanted to write code, you had an IDE, you had some sort of autocomplete in the IDE, and then there was some sort of chat app, and you might like copy and paste code back and forth a little bit

01:12And that was the state-of-the-art

01:14That was AI in coding

01:17And I think maybe sometime around a year ago, we started to see agents appear as a thing that people earnestly use in coding

01:25It's like a part of the workflow, it's not a gimmick or a prototype

01:28It's actually part of the the inner loop when you're doing dev

01:31And I think this is the thing that's changed the most in the last year is now when you code, you use an agent

01:40You don't directly manipulate text in an IDE anymore

01:42It's not just about tab completion, it's about the model writing code for you

01:48And I think what we've started to see is the shift from directly manipulating text to having the model do the text manipulation for you

01:55And I think projecting it out, this is sort of the trajectory that we're on

01:59Is this continuing into the future? I see

02:02So, we've gone from it all being within a web app where you're copy-pasting the code out and you're making like very targeted edits, almost to just being a lot more hands-off and telling an agent what you want it to do, and then trusting it to go make tons of edits and create whole apps sometimes even by itself

02:14Yeah, exactly

02:26And this was something that I think the reason we couldn't do it a year ago, and you know, like people have tried to make AI do coding for the longest time and to, you know, just like automate more and more of coding in various ways, and it hasn't really worked

02:38I think probably for a couple of reasons

02:39One is the models weren't really good enough, and the second one is that like the scaffolding, the thing on top of the model wasn't good enough

02:45And when we initially launched Claude Code, the very, very first versions late last year, I think this is still using Sonnet 3.5, this wasn't even uh 3.6, six or Whatever we call this, like the new Sonnet 3.5, yeah, upgraded Sonnet

03:02Yeah, it it wasn't even this, and it it like sort of worked, you know, like I, I used it for maybe 10% of my code or something like that

03:07But even then, I remember when when we launched it, we gave it to the core team, and it it was just me and like a few other people on the team at the time

03:15And I remember walking in one morning and kind of on the way to my to my desk, there were a few engineers sitting there, and one of them was Robert, and there were a couple of other engineers

03:24And I just walked in and I saw Claude Code on their screen the first time, and like I just gave this to them the day before, and they're already using it

03:33Yeah

03:33And it was just the craziest thing

03:34And the model wasn't very good

03:34The harness wasn't very good

03:36But even in this early version, it was already a little bit useful

03:41And I think that over the last year, what's happened is the model has gotten way better at agent decoding

03:45And that's happened with like 3.7 and now 4.0 and Opus 41

03:51And the harness has also gotten a lot better

03:54And you know, obviously the harness is Claude Code because the the way you interact with the model, you can't just like directly use the model

03:58You have to use a harness

04:01It's sort of like, you know, like if you're riding a horse, you need some sort of saddle

04:04And like the saddle makes a giant difference when when you're riding a horse

04:08Um, I'm not a horse rider

04:13I like that analogy though

04:18I mean, it it is kind of like Claude is is the horse and as the engineer, you're trying to get it to go in a certain direction, and you're trying to guide it

04:22And like, you need some sort of scaffolding around it to be able to steer it correctly

04:29And the harness, in this case, just so we're on the same page, is everything from like the tools we're giving it to how we handle like the context and everything for the model

04:32Exactly

04:35Exactly

04:38It's it's like all of Claude Code

04:41The model is the thing behind the API, and then Claude Code is the it's the system prompt, it's context management, it's tools, it's the ability, uh, for, you know, to plug in MCP servers, uh, settings, permissions, all all this kind of stuff, all this interfaces with the model, and the model sees all the context, all the output from the stuff, and it makes a giant difference in the way that it performs

05:03And I, I think over the last year, we've learned how exactly we build for the model, and the model has kind of co-evolved with not just Claude Code, but all these different products that are using Anthropic models to build agent coding tools

05:13Maybe let's speak more on that

05:23When you say co-evolve, is that because it's like a deliberate thing in which we're doing with the training, or how, how is the model also getting better at these sorts of things as we make the product features itself better? It's pretty organic, honestly

05:33Like, you know, at Anthropic, everyone uses Claude Code, and that, that includes the researchers

05:39And so, every day, The people building the models are using the model in order to do their job, and I think as part of that, you kind of see these natural limits that you hit with a model

05:48So, you know, as an example, maybe the model's really bad at doing certain kinds of edits, and sometimes when you use Claude Code, you see like, "Oh, failed to replace string, failed to replace string." Like, this is a model capability, and we can improve this if we learn from it

06:02Or another example, maybe something like higher level is if you just let the model cook for like 30 minutes

06:08With 3.5, it could kind of do it for a little bit, maybe for like a minute or something, it would stay on track

06:15And then with newer models, it kind of gets longer and longer

06:18This kind Of this, this amount of time, the model can operate autonomously

06:22And I think this is really based on experience because you use the model, you kind of see where, as a human, you have to course-correct and steer it, and then we've learned from that and we can kind of incorporate that into the model and teach it better to do this itself

06:36When you're evaluating a new model, do you kind of have a vibe check set of tests that you run? Or if it's like a new feature that we're rolling out to make something better in the harness, how do you personally evaluate if the performance is getting better? I just do my work that day

06:53Interesting

06:55Yeah

06:57Like, my perfect day is I'm just coding all day, and you know, whatever the model is

07:00Whatever is the new thing we're testing, I'll just code using that and see what the pipe is

07:02There isn't like a specific thing I do, right? You just see how does it actually work for me in my day-to-day

07:09Yeah, exactly, exactly

07:16And like, you know, in day-to-day work, you do all sorts of stuff, like you're writing new code, you're maybe like fixing bugs, you're maybe reading Slack messages or GitHub issues to respond to feedback, and I think more and more the model is able to do more and more of this

07:23So actually, in a way, if you had maybe one thing that you always used the model for, you would miss out on some of these newer capabilities, like pulling in context through MCP, like reading your Slack messages or, you know

07:34automatically debugging stuff because you can pull in Sentry logs automatically

07:39Yeah

07:43So the best eval, in some sense, is the one that most looks like real life, and in that case, just using it gives you the best result

07:48We, we tried really hard when building Claude Code to build product evals

07:52Yeah

07:53You know, just like to have some sort of benchmark, like when we change a system prompt or whatever, is the model getting better? And we have a little bit of this, but honestly, it's just like so hard to build evals

08:02And by far, the biggest signal is just the vibes

08:05Like, does it feel smarter? Because there, there's such a broad range of tasks they use it for

08:10Yeah, that's actually a question I hear from developers all the time, is they would appreciate more guidance on how we go about prompt testing and iterating

08:19Um, I know for different products, we have like various sorts of eval, but for Claude Code, it really is just kind of this tight feedback loop that almost gives us like more immediate signal than any hard-coded set of evals

08:33I wonder if people kind of want to hear a better answer from a, but yeah, man, it's it's all vibes

08:39Yeah, I think at this point, we're, you know, the models are doing so good on evals like Swebench, you know, we're just trying to find these harder evals, and now there's like TBench, which is like a little bit less kind of saturated, but I think it's just really hard to find synthetic evals that capture all the complexity in software engineering

08:46Right

08:53Right

09:00Do Do you think there's something we did uniquely to set up that feedback loop internally? Because I feel like Claude Code has like the best dog fooding cycle I've seen of like any type of product

09:07Initially, I built it the way that I do any other product, which is just listen to users and make it as easy as possible to listen to users

09:15And I think one part of it is when we built Claude Code, there was just like a single feedback channel in Slack, and anytime anyone had feedback, I would just direct them to that, just be like, yeah, post there

09:26And I feel like people hesitated sometimes a little bit because sometimes when you give feedback, you expect that no one listens and it kind of goes into this black hole, like into a void

09:36And I think one of The things that we did really right was from the beginning, whenever someone gave feedback, I would try to fix it as fast as I can

09:44And sometimes I would kind of go into the office and then just spend like 3 hours or 2 hours or whatever, just go through as many bugs as I can and fix them as fast as I can

09:50And then every time, comment back and tell people it's fixed, right? And this kind of encourages them to keep giving feedback

09:56And to this day, the Claude Code feedback channel internally is just this fire hose, just non-stop

10:02Oh, totally

10:07I remember on those early days and still still do, dropping in there, posting something and immediately you're emoji reacting, or you're asking for more clarification and more questions

10:13And you do feel like, oh, okay, my feedback's being heard, and then you're able to like actually be incentivized to go post more feedback in the future

10:23Yeah, because, you know, honestly, like I don't know what I'm doing

10:25Like, no one really knows what they're doing with AI

10:27Like, we're kind of discovering this thing as we build it, and the best indicator is what the users want, right? So you just, you got to listen, right? Switching gears slightly, uh, what is like the current state of Claude Code as a product? Um, what are the latest features? What are you excited about? Uh, some things that you're seeing folks do with it right now

10:48Claude Code from the start was built to be the simplest thing it can and to be as hackable as possible

10:51And I think the hackability is something that we've been developing a lot, and that's something I'm really excited about

10:57So originally, the way to hack Claude Code is adding to its quad code, that was the original extension point

11:06And quad MD, as you know, is like this file you can put it in the root directory

11:11You can put it in child directories

11:12There's kind of different places you can put it, and it is just additional context to give Claude Code, and it kind of goes with your repo

11:17You often check it into your codebase

11:19So it's kind of, you know, a little bit more information about the code

11:23Mhm

11:23But over time, we've added a lot more extension points

11:25So now there's a very sophisticated setting system and, uh, permission system

11:30There are uh hooks now, which Dixon built

11:34Um, Dixon's an engineer on our team, and he just kind of saw all these different user apps coming in for, "I want to extend it this way

11:39I want to hook into this, hook into this." And so he built a super extensive, uh, hook system

11:45Um, MCP, obviously, this is a really great extension point

11:47Um, and now there's slash commands and sub-agents

11:49And user-defined slash commands is something we've invested in a lot

11:53And the idea is, it's just a workflow

11:56It's like a markdown file

11:57You put it in your code, and it's something that you can reuse a lot

12:00So, for example, I have a, uh, slash command for, uh, making commits

12:06And I have some instructions in there

12:09Here's how you write a good Git commit

12:12I pre-allow the Git commit

12:14bash command so I don't have to, uh, accept it every time, and the model can just do it

12:17Um, so I think slash commands are really interesting, and agents are kind of a different view of slash commands

12:24Like, it's like a slash command, but it has a forked context window

12:29And so you can kind of think of agents and slash commands as two sides of the same thing

12:32Um, and this is also very exciting

12:34It's just another way to extend Claude Code

12:37And so when I look at the future, I think a lot of it is just about, like, how do we extend Claude Code more? How do we make it easier for other people to build on top? How do we make the SDK more useful for people? So, it's useful for code if you want to build a coding agent, but also you can use it for other stuff like anything that you need an agent for, you can just use the SDK for

12:56Yeah

12:56And I think these are the things that I'm the most excited about

12:58And obviously, all this benefits from all the other work we're doing to make the model more autonomous, to make it work for longer periods of time, to make it better adhere to instructions, to make it remember things better

13:09Um, and so everything along the way benefits

13:10So, I'm using Claude Code, or whatever form of it, in six to twelve months

13:17What does my work actually look like? Am I reviewing PRs all day, or what? What does the day-to-day breakdown look like? Yeah, I think there's going to be a mix of more hands-on coding

13:28I, I don't think that's going away

13:31Yeah

13:32And maybe it'll look different though

13:35So

13:38Maybe hands-on coding today is directly manipulating text, but in the future, it might be using Claude to manipulate the text for you

13:41Okay

13:42And then, I think there's going to be this other bucket of maybe less direct coding where Claude proactively does something, and maybe Claude even reviewed it, and it's your job to decide if this is a change that you want or not

13:53Mhm

13:55And I think maybe 12 or 24 months from now, we're going to start seeing Claude that's more about goals and more about these higher-level things that it needs to do, and less about the specific tasks that go into it

14:09The same way that as an engineer, I think about what it is that I want to do over the next month, and I kind of make small changes to work towards that

14:17Maybe Claude will go through the same thing, right? Sort of moving up and up the stack to some degree of these like abstraction levels, of getting Claude to make individual changes to files, to getting Claude to make changes to a whole PR, to getting Claude to think about a goal of building an app or whatever else it is

14:23Yeah

14:29Okay

14:35That's that's interesting

14:41If I'm an engineer and I'm hearing that, uh, it seems like there's going to be a lot changing in a very short amount of time, especially with my role and what I should be doing

14:47What's your advice for folks out there that are looking to prepare themselves and adapt to this world about what they should be learning or what skills they should be developing? I think back to when I first started learning coding

14:59I was the kid that sat in the back of math class in middle school, and I had my little TI-83 Plus calculator

15:05It was like a transparent gray one, you could kind of see the circuit, and I would just program it with BASIC because at some point I realized that I can actually program the answers for the math test into the calculator, and you can get better grades that way

15:24And, uh, there's just something about kind of this visceral feeling of being able to hack, and having this idea of maybe there's this one program I can make, and just I go into my calculator and I code it, and then I can just restart and use it really quick

15:36This kind of feedback cycle that was really amazing, and it made it possible for me to build stuff that I never could have before, and it was just so easy to get started

15:47And I think about the difference between that world and the world before agent coding, where stacks just got way too complicated

15:58You know, if I wanted to make a JavaScript, you know, like website, I had to learn about React and maybe Next.js, and then three different build systems and a deploy system, and it was just so complicated

16:11And I think one really cool thing about agents is that they're changing this

16:13So with coding agents, it makes it really easy to get started

16:18And if you have an idea, you can just build it

16:19And it's, it's a lot more about the idea now than it is about the details, because just like Claude Code, you can rewrite the code over and over

16:27And, you know, Claude Code itself, we rewrite all the time

16:31And I think this is just something that coding agents enable

16:34The code itself is no longer precious

16:37And there's still an art to writing it

16:38And, you know, I'll still code by hand sometimes

16:40And one of the engineers on the team, Lena, she was talking about how on the weekends, she still sometimes writes C++ by hand just 'cause it's fun

16:45And, you know, as a coder, this is a, this is it can be a really joyous thing to do

16:53But I think more and more, it's going to be about the thing you make and not about the process of making it as much

16:57Mmm

17:01And I think, I think my advice for people learning to code today is, you still have to learn the craft

17:05So you still have to learn to code, learn languages, learn compilers, runtimes, how to build web apps, how to build programs, uh, system design

17:13You still have to know all the stuff, but also just start to get more creative, right? Um, and you know, if you have an idea for a startup or an idea for a product, you can just build it now in a way that you just couldn't before

17:27And we don't really understand what this means, but there's just so much potential that's about to be unlocked because of it

17:30Yeah, I love that

17:33Um, I think that's great advice, too

17:36Ideas suddenly become something you can action on in, you know, a span of a few minutes almost, whereas before it could be just in your backlog forever

17:41Yeah

17:46Um, before we wrap, I, I want to ask you as a creator of Claude Code

17:51What are your best practices for using Claude Code and any tips or tricks? Yeah, I think the biggest thing that I recommend, okay, maybe two, maybe two tricks

18:03Yeah

18:03So, one thing I recommend is that if you're brand new to Claude Code and you haven't used it before, don't use it to write code

18:12And instead, I know it sounds crazy, explain, explain, but you got to stop yourself

18:15Like, don't use it to write code yet

18:16The thing to start with is use it to ask questions about the codebase

18:22So you can ask, you know, if I want to make, if I want to add a new logger, how do I do that? And then ask, ask Claude Code to explore the codebase and figure it out for you, or why is this function designed the way that it is? Claude Code can, uh, go in and it can look

18:35through Git history and it can answer the stuff for you

18:38Uh, so I think ask Claude Code questions about the codebase and just don't code yet

18:43And then once you feel comfortable with using Claude Code this way and you get comfortable with this idea of an agent that's doing this research for you, then start to use it to code

18:52I think the second thing is, when you are using Claude Code to write code, think about what kind of work do you want to do and like, how big is the task? So for something that's really easy, I, I kind of, kind of in my mind, I have these three categories: easy, medium, and hard, very roughly

19:07And so easy tasks are something that Claude can write in one shot, like one prompt

19:10It'll get it pretty much right

19:13And nowadays, I'll just go to GitHub and I'll tag add Claude on an issue and just have Claude write the PR for me

19:17And this is how I do easy tasks, cuz that frees up my terminal

19:20I don't have to kind of spend it on these medium tasks

19:24I'll start in the terminal and I'll start in plan mode

19:26So just shift tab into plan and I'll align on a plan with Claude first

19:31And then once I feel good about the plan, I'll go into auto accept and I'll have it implemented

19:35And then for really hard tasks, I'm still the one driving

19:38And Claude is more of a tool

19:41And I'm kind of pairing with it, but really I'm the one in the driver's seat, not Claude for this

19:44Um, and so I'll use Claude maybe to do some code-based research

19:47Uh, maybe prototype a few ideas

19:49Maybe I'll just like vibe code a few options and to understand the boundaries of the system and what works well

19:55But I'll still mostly implement it myself, and maybe Claude will write the unit tests

19:59Um, but it's still mostly me doing the coding

20:04So, I think that'll be the second advice: just think about what's the task that you're doing and what's the right way to use Claude Code to do it

20:07Those are great tips

20:10Uh, really appreciate the time, Boris

20:13This has been awesome

20:14Thank you

20:15Yeah, thanks, Alex.