00:00[Music] Oh, hi
00:08Um, I'm here to uh to do 20 minutes of breakout, and uh I'm very close
00:14I've never been this far
00:30Uh, so I'm going to talk about Gemini CLI, right? So, Gemini CLI is a coding agent for the terminal
00:36So, it's Gemini, the model, and it has access to tools on your local machine, like run a shell command, uh search for text in this directory, open a file, make edits to it
00:50It's an agent, so LLM access to tools
00:56That's all there is
01:03And I I don't like showing slides, so I'm just going to go and and build this game, right? So, I create a directory
01:07Start my agent, Gemini
01:10Let's start
01:14I'll tell it to, is this Readable? Yes
01:17Very big screen
01:21Write the spec for a breakout clone, web based
01:25So it's assembling the interwebs
01:42So if this is going to take too long, I have a separate tab, but I, I like to, like, cut the suspension
01:53You have a spec, okay
02:01Now go and build a nojs
02:09So this is some spec nojs app
02:17So has anyone of you used, like, coding agents for the terminal before, like cloud code or Gemini CLI? Few people, nice
02:24How do you like it? For me, this was like life-changing
02:30I, I've more on that later
02:36Okay, so go
02:41What I really like about Gemini CLI is that it's not because I used to use Ader, which is focused on file
02:46Editing, but this can execute shell command, so it can actually propose to me like, "Hey, I want an MPM in it now," and I approve the tool call, and it, and it, um, and it does it for you
02:59So that also means that I can do a whole lot of things that are not strictly coding related, like I can tell it to use G-Cloud to deploy services, or use cubectl to debug my, my running cluster
03:05MPM install sounds good
03:11Okay, I'm going to switch to this second tab
03:16I already did the whole thing and walk you through what, what happened
03:20So I wrote this back
03:24This time it was a bit more concise
03:28I, I like that
03:31And then I said implement the game as a Note, JS service, `npm install yada yada`
03:35It, it comes up with CSS, and now, well, simple code
03:38I read all of this
03:43I'm an engineer
03:47I, uh, I don't want to just run it on
03:52There is a flag on Gemini CLI, `-yolo`, and then it will just auto-apply everything
03:56And I thought that was kind of nice because I could let, like, Gemma CLI do all the work for me, and I would just watch from a distance
04:01And then this one time, I asked it to write a test for me, and then it finished the test, and I said, "Okay, I'm, I'm done with the test
04:08Can I now clean up for you?" And it proposed the command `rm -rf`
04:17I no longer, I no longer run in yolo mode
04:21There is Sandboxing available
04:26If you want to do YOLO mode, it's fine
04:30Okay, so it comes up with a game, and then I ask it, "Hey, great results." Like, I noticed that Gemini is always giving me a compliment in the first sentence, right? So I started to do that back
04:35I like this over, like, this glazing
04:39So add a Docker file
04:44Use G-Cloud to deploy the project to Cloud Run
04:48So it writes my Docker file, deploys it, and here's something interesting happens: deployment failed
04:52Revision is not ready
04:56Cannot serve traffic
04:59User provided container field to start
05:02Now listen, listen on the port defined by the port 8080
05:08environment variable
05:13That's correct because my NodeJS thing that it created for me was listening on port 3000 without skipping a beat
05:19Uh, Gemini CLI proposed a change, like it read the error message, proposed this, like, hey, let's, let's listen to that environment variable, deploy it again, and it worked
05:32And then I told it, like, hey, I want to share this app with my friends, right? What are my options for creating an, uh, a global IP address and pointing the domain to my app? Well, there's a few options here: create a serverless on Google Cloud, create a serverless network endpoint group, global load balancer
05:45Backend service, network endpoint group, URL map, HTTP target proxy
05:52I like, who's, who's done this by hand before? Did you enjoy reading the docs, like the, the 12 pages and figuring out what all the resources were? There was, yeah, I always, I also, I enjoy it a lot, and um, but I'm happy that that, uh, so I, I tell it like, hey look, I'm, I'm get, I'm glad that you have a PhD in networking, I don't, but I trust you
06:04So, let's go
06:16And then it goes and it creates all the resources, and end of the story is we're here
06:22So, that was the demo of Gemini CLI
06:29It has a free tier, so if you have a personal Gmail account, you can go and try it out
06:36You have 1,000 free Gemini 2 to the 5 Pro prompts every day with higher limits if you're, uh, a subscriber to Google AI Pro
06:48And if you want to pay, there's subscriptions on Google Cloud, and also pay as you go, which is interesting because that comes with enterprise terms, right? So if you care about intellectual property and things, then this is something to investigate
07:08Confession: uh, I wasn't excited about LLMs
07:14Maybe I'm telling this to the wrong audience, but I've been building software professionally for over two decades, and a year ago, I, I, I, I failed to get excited
07:19There was all this hype
07:23I tried it
07:28It made me slower
07:33It was suggesting things that I didn't like, and I also worried about losing my craft, right? If you, if you're doing something for such a long time, and then there's agents that pretend to do it better
07:38So I, I kept searching for reasons to justify not to engage with this technology, but hey, I work at Google, so I'm contractually obliged to use AI, so I changed my opinion
07:53Um, I strong opinions loosely held, that's, uh, that's something I, I live by
07:59So in February, I, I coded a game for my kids on my phone, uh, in the train using Replit, and that was a nice experience
08:05It didn't get me
08:09Super far, but it was, I was like, hey, this is, this is kind of cool
08:14So, I, okay, I will use LLMs for recreational coding now
08:18And then in March, I explored clothes on it using Ader
08:24It's another, um, coding agent, and I used it to generate a lot of test data that I needed, and it was, and it needed to look okay
08:30It was a demo of an e-commerce, uh, an online store, and I wanted to have a real product, and then in March, I switched to Gemini 1.5 Pro, and it's kind of like this, this is getting powerful
08:42And then this June, I started using Gemini CLI exclusively, and now we're here at this talk, um, which is what I learned about Gemini CLI and using
08:54It effectively helps for building a medium-sized project, and I do want to share this project because I really like it
09:01So I used it to build a text-based adventure game for learning agent development, right? I'm in developer advocacy, so I'm teaching, writing, uh, coding, and I promised myself I will do zero code writing, only reading
09:21And to help me with that, I picked a programming language that I've refused to learn over the past 20 years, which is Python
09:27So, for those of you who aren't familiar, uh, what a text-based adventure game is, it's interactive fiction inspired by the early days of computer
09:32games, and it's like this: You're in this ant chamber of the Sunstone Pyramid
09:40An open doorway leads north, and on a pedestal, you see a piece of flint and a steel striker
09:47Now, what do you want to do? Examine the pedestal
09:51Okay, it's a plain stone pedestal
09:56There's nothing special about it
09:58Okay, let's go north
10:01You're in a dark chamber
10:04It's pitch black, and you're likely to be eaten by a Grue
10:06You feel a breeze from the south
10:11We're running this game today at the Google Cloud booth, uh, starting tomorrow
10:16So, you can get a high score, and there's a leaderboard, and the idea is that you build an agent to play this
10:21game for you
10:27So, who am I? I'm an engineer at Google in the Cloud Run team, based out of Amsterdam, and I'm a small-time Gemini CLI contributor
10:35If you like, if you find a bug and submit a PR and actually get it through the merge queue, that's an amazing achievement I wanted to highlight in it here
10:41Okay, so here's a couple of things that I learned and and and want to share with you
10:47So first, let's dive into how Gemini CLI talks with the model
10:54It's basically orchestrating, like it's, it's a surprisingly small thin layer, uh this agent, right? So it's, it's orchestrating a conversation
11:00It starts with your
11:06First message: "Hey, build me this breakout clone." It goes to Gemini
11:16Response comes back
11:25It's added to this list of messages, and then I type a new message: "Um, start this project and install my dependencies." And then this whole conversation goes to the model again, and Gemini comes back with the response, and maybe this time it wants to call a tool, it's read file or execute this shell command
11:35Now, Gemini CLI gets approval from the user and responds with, uh, with the content of that tool gone, the output of the shell command, or the results of the search on the file system
11:42But there's more, so let's go
11:48Back to the beginning of this conversation, because it doesn't really start with that first message from the user
11:57It starts with the system instructions
12:06The systems instructions are how all those coding agents that you've seen Google and other companies release differ from each other, right? So there's Google Jules, AI Studio, Gemini Canvas, Gemini CLI, Firebase Studio
12:14They all behave differently
12:21You get different results from them because of this system prompt, and they have the exact same model underneath, Gemini 205 Pro
12:29So this is where it says, like, hey, if you're building a JS app, and the user didn't specify "choose react" until wind or something, right, then there is a short preamble: "Hi, this is Gemini CLI, we're setting up contacts, today's date is October 6, my operating system is Darwin because I'm on a Mac," and then a recursive file listing of the directory, right? Gemini CLI, you run it always in a project directory, okay? And it just gives it the recursive file listing, and that's it
12:59That's all it knows about your project when you start it
13:05So all the other things, so if you're like just getting started and you have zero files, it doesn't matter at all, but if you're in a big project, You might want to add some more context for it, so that's where gemini.mmd comes in, and this is user-specified context
13:16It's a file, a markdown file, in the project root, and my heuristic for, like, what do you put in here, is put everything there that you want Gemini CLI to remember
13:31So whenever I'm working on a feature, and it goes down a rabbit hole that I don't want to go, I have it go into again, I'll put it into Gemini MD, and it can also be like, hey, uh, before you get started, just read the make file, so you know all the operations, and you've run my, my test suite in the, in the right way
13:50I'm not judging, okay
13:57So, another The important thing when you're working with those models is, is it in or out of training distribution? Because if Gemini has seen many examples in training of the language, library, or framework that you're using, it might very well write very great and idiomatic code for it if it hasn't
14:17So, here are a few examples with great results, right? There's React with Bootstrap CSS
14:22I'm not going to read the whole list
14:26You can find it
14:31Uh, but this is, I think, a fair distribution of popular libraries and frameworks, and there might be even more, right? But the only way to figure out if your favorite stack works
14:36Well, together with those models is to try it out, and if you constantly think this is terrible, it might not have seen great examples in training, but there's a solution for that
14:48It just is a bit more work because those models, they can learn in context as well
14:53So if you fill your gemini.mmd with all your preferred ways to work with your uh framework, then you can teach it, and you can also, when you invoke Gemini, you can include additional directories, like I was writing code for uh Google's agent development kit, and um, it didn't exist when Gemini was trained, so it was like making all the all kind of weird
15:04assumptions, just riffing off on on on class names
15:15Oh, well maybe this method is available on this thing, makes sense to me
15:26So I include, I created an, um, I, I actually took, uh, the Python ADK source tree, and then I prompted Gemini CLI to, hey, using this framework, maybe you can write a whole lot of examples with it, and then we work together to, to create a bunch of examples, and that's what I now add as additional context when I'm writing code that uses ADK
15:52Another thing is finding the loop, which is really great
15:59So another example, build me a calculator
16:05Build a simple CLI based calculator that gives you a prompt and
16:11You can do like add a few, um, how do you say, operations, hit enter, you get a response, and you have variables
16:16I can actually show this
16:20Uh, here we go
16:24So this is the prompt I gave it, and then it started writing it
16:29You organize the code cleanly into modules, important
16:33So it writes it
16:38Okay
16:43Basic time to test, right? I tested it, uh, saw it work, and now I ask a very important question, give me options
16:48Oh, I come back later to this, because if you, if you just ask it like, hey, do this for me, then it will say, oh sure, I will go, and then it starts running, right? I will do just and build your entire testing setup, so I, I start with Hey, give me some options, and being an LLM, it wants those options to be like, not three exactly the same options with a minor difference, but actually three different things that make sense, right? So, built-in assert module, just for Mocha and Chai, and then it even gives me a recommendation, and then they say, okay, I also need a way to track test coverage, because that's important, right? Okay, okay, since test test coverage is a requirement, use Jest
17:33I'm like, no, I don't want Jest, are you sure to give me more options? Okay, okay, there's Fight Test, NodeJS built-in test runner, and C8 gives me a, helps me choose, and now
17:47It recommends VI test, so clearly I say no, let's stick to the Node.js builtins
17:58Okay, okay
18:08I will use the Node.js test runner, installs it for me, edits the package.json, JSON writes a few first tests, runs it, comes up with the coverage, and now it gets interesting because without skipping a beat, it sees that the coverage is not right there where it wants it to be
18:19So, it starts adding, figuring out how to increase the test coverage, reads the code, sees the missing statement, uncovered line there, tries the thing, added the test, rerunning to check for 100% coverage
18:25I like this ambition
18:32share this ambition
18:39Gemini CLI and I runs it again, sees that it fails, tries another thing
18:42Okay, expecting 100% coverage now
18:45Still didn't work
18:48Like this is without me inputting anything, right? It gets into the loop
18:51If you let it into the loop long enough, at some point it will say, "Hey, automated loop detection kicked in, do you really want to continue, because you're burning tokens, that's expensive." So it tries another thing
19:03It reverts its earlier change, tries another thing, runs the test again, 100% coverage, boom
19:11Well, this is finding the loop
19:20If you, if you find a way to build something, and you have this way to give that LLM automated feedback on what it's doing, it will figure it out
19:29And if you can't find that, you will have to give the give the feedback
19:39Okay, so that was getting into lube
19:48100% test coverage is achievable without existential dread
19:57That's my another learning that I have, because tests are really important, right? We all write them, right? Everyone writes them, but it's also a lot of tedious and boring and well, tedious work, and LLMs excel at doing this
20:06So you go and tell Gemini to cover all the branches and run coverage and only stop when you have 100%
20:12Branch coverage, and you give it feedback while it's doing that to make sure it hits the right level, like, is it unit test, component test, integration test, is it trying to cheat to hit a branch? It will give it feedback and it will write great tests, and you won't feel this existential threat and dread, which is great
20:37So even my demo text-based adventure game has 100% unit test coverage, and this allows me to go incredibly fast, like I do the wildest refactorings, and I trust that all the levels that I created are still completable because I asked Gemini CLI to write walkthroughs for all the levels and
20:54all the edge cases and all the room dynamics, so I know I'm not breaking my, my work
21:01I've, I've never written so many tests in my life
21:09And then if you want to radically change something and you have to rework all the tests because they kind of still did depend on the implementation, then you just let Gemini CLI do it for you
21:14So okay, another one, interview me to write an implementation plan, and this is where I think, yeah, I, I'll go and show my source code for my adventure game thing
21:21So there's a recent change that I made
21:27Let's pick a nice one
21:34Oh, that, yeah
21:40So this is a plan to, so I'm working on scoring, right? So
21:45There's different ways to score
21:49There's time based, and there is also an exploration component
21:54So if you examine the pedestal, you might get an interesting bonus
22:04I'm, I'm giving you tips to get to the top of the leaderboard right now, like you're a very special group
22:15Um, so it has an overview, like, hey, what, what's this plan about? New scoring component, um, no prerequisites for this work, here are the relevant files, so you should, before you go about this task, read these files and create this one and edit these files, right? So if I give this plan to Gemini CLI, it will know what to do, to how to explore the codebase
22:25Because when it starts, it only had that first message and a recursive file listing, didn't know anything else
22:32So this is an important section, and then phased implementation, three phases: first, update data models, game logic, uh, add more config, and finally, add more comprehensive tests, because I know that my, my human really appreciates it when I do that
22:56What's the phase? Very important success criteria for phase one, second phase, also success criteria for, for, for a second phase, and what's not in scope, right? So now you're wondering, did I write this by hand? Who's wondering this? No one? Okay, I didn't write
23:14This by hand, uh, because I really like it when Gemini helps me with this, and my, my favorite trick is to say, "Why don't you interview me and figure out what I really mean and write down this non-ambiguous plan for me?" I also have a template for my plan, which looks like this, like it's the structure of the document, and what, what needs to go in what section
23:39So you recognize the relevant file sections and whatever I want to be in the, in the phases, and that they need to have a success criteria
23:57So my favorite trick to write plans is tell it to act as a principal engineer using the guided discovery method, and your goal is To help me create a robust technical implementation plan, ask me one probing, open-ended question at a time, and before every question, you should explore the codebase, right? And maybe even do a web search to do research, because otherwise it just thinks, okay, building an extra bonus plan
24:19Uh, sure, I know everything about bonuses, and then it comes up with something that's not related to the codebase
24:25So, this is important
24:32I can actually show, uh, because I was working on, we're running this same workshop in Spain, uh, in about two weeks, and so I want to, I want to make this app multilingual
24:37So, I tell it, to act as a principal engineer
24:42Help me figure out how
24:45Interview me discovery style
24:49Ask one probing question at a time
24:52I'm typing out the prompts, but there's also a feature in Gemini CLI that is custom slash commands, which lets you save a prompt, and you do just slash interview, and then it will print the prompt and run it for you
25:09Okay, so it does a bit of research first and see some templates with hardcoded text
25:12Yeah, I know
25:16Then it wants to know if I also want to translate the levels, like you're innocent stone and the chamber
25:20Are you planning to translate both the user interface and in-game? content, right? So now it goes to interviewing mode, and I first said yes, and then I said no
25:31I only want to translate the UI, yada yada yada, no
25:39And then I, I, I do this trick again, like, hey, you need to give me options to choose out of, right? Just telling me you're going to do it this way is not helpful
25:50I want to have options, so standard library, lighter weight, JSON based approach, or a third party translation service
25:56Wow, okay, fine, we'll research even deeper
26:03Okay, now I go like, write this plan to the plans directory
26:16I also like about LLMs that are not super, uh, sensitive to typing errors, because if I'm on A stage my typing ability decreases by a lot
26:23Use add doc templates, so you can use ads to reference files implementation
26:29Yep
26:34Okay, there you go
26:40So it reads my template and start writing a plan for me, and when the plan is there, I'll use slash clear to clear the session and then give it the plan and say go implement this now
26:55And I'm doing this because I'm sensitive of this uh 79% context left
27:02Sometimes it goes a bit deeper, but the more you have in your context, the longer this whole conversation goes, the harder it is for for Gemini because well, it's it's a transformersbased model, right? So the Harder it gets for, it gets, gets for the model to, to figure out, uh, what's relevant in the context and what's not
27:13Okay, so there you have the implementation plan
27:23This is something I can hand, hand, hand out to, uh, to Gemini CLI and, and get started
27:34And because I have the GH client installed here, I can also, uh, just tell it to create a GitHub issue with this plan and maybe someone else can pick up the, the change
27:42Good
27:49Um, always ask for options, notice context rot, and what to do about it
27:54Yeah
27:58So, I already referenced that a bit
28:02There's this little marker like that tells you how much context is still left, if it
28:06gets down to 80%, I really start to notice that it needs, uh, clean up, right? So, I, I will wrap up the session
28:10I'll say, "Hey, summarize what we've done so far in a markdown file, uh, or update the plan we've been working on and tell me where you stopped and we'll restart the session and start again." To, uh, it also saves tokens in cost
28:32Another thing is that I noticed, uh, that there is no substitute for really engineering and thinking about how do you divide your code, where, where does what go, right? Because if there's a certain structure in place, it will just keep on adding, like just like I did when I was still Manually coding, right? And at some point, you discover that this main or this engine.py file has 800 lines and you don't know, it's all over the place, and you can't make sense of it anymore
29:01So Gemini has the same issue with it, with this sort of, it will have trouble with, with doing file replaces and edits in the, in the, in very large files
29:13So at some point, I ask it, like, hey, look at all the functions and maybe we can figure out three different ways, ask for options again, three different ways to reorganize this code into, into different modules
29:24I pick one approach, have it write an implementation plan, and go
29:28About it
29:33Now, do I use this like, write me a plan and then do it approach for everything? No
29:37If it's just, oh, uh, there's a CSS bug on the leaderboard thing, then I'll just prompt it like, hey, fix the CSS bug
29:44Yeah
29:51So then, but you'll develop a feeling for when you need to take which approach, uh, pretty soon
29:55Yeah
30:00So that's about my, uh, my contributions from experiencing Gemini CLI or for building a text-based adventure game
30:02So understand conversation structure
30:05It's super important
30:07Read, maybe read the system prompt of Gemini CLI
30:10It's on GitHub, because then you'll understand why it tries to
30:16Do certain things, like if you ask it to do a git commit, why does it always read the logs to figure out the existing commit message structure? Because it's in the system prompt, right? So if you have questions like these, the answer is in the system prompt
30:29Is it in or out of training distribution? Uh, you will have a great time if you use Python and fast API
30:36Other frameworks maybe not likely because it didn't see many of those examples in training
30:41Find the loop
30:45It's uh, satisfying
30:48Gemini CLI likes writing tests so much that there's really no excuse for uh, 100% coverage
30:51And this goes for all the boilerplate code, ask the metal model to interview you, always ask for options, uh, notice context rot, and restart sessions strategically, and continue refactoring mercilessly, and that's, that's it for today
31:15[Applause] [Music]
