00:00Let me show you how to use a local LLM inside of Cursor

00:02Now, obviously for this to work, you need to download the Cursor IDE, so you can do that at cursor.com and then download for whatever operating system that you're using

00:09The next thing that we want to download is Olama, and this allows us to run a bunch of different models on our local system

00:15So, to do that, we can go to omama.com, and then once we're here on the main screen, we can click on download

00:21Now, download and run the installer for your particular operating system

00:24So, since I'm on a Windows machine, I will be showing you how to install it using

00:28Windows

00:31Once you open up the installer, you'll get met with this screen, and all it's telling you to do is click on the install button, so that's what we'll do

00:35Now, once the installation process is complete, it will automatically close that window, but we can see that it's running by going into the bottom right for Windows and then clicking on this carrot symbol

00:46And up here in the top right, we have Ollama, so this is how we know that it installed correctly and is running

00:50We can now open up our terminal of choice

00:53For me, I'll be using PowerShell, and if we type Ollama in, we get a list of commands that we can

00:57Use for the Ollama CLI tool

00:59Now, we do need to look for a model that we want to use

01:01To do that, we can go back to the Ollama.com website and then go to models

01:05And from here, we can scroll through this whole list to see which one we want to use

01:08I'll use the 12B Gemma 3 model because it says it's the current most capable model that runs on a single GPU, and I'm only running it on a single GPU

01:18Now, you'll see over here on the right that we have a command that we could copy

01:21What this is going to do is, it's going to automatically pull down this model as well as serve it so that it starts running immediately

01:29For right now, I just want to copy the name of the model, so I'll highlight it here and hit Ctrl + C

01:34Now, while we could run that whole command, the run command, uh, what I want to do is I want to show you how to do it separately so that you can just pull down the model without running it and then run it later

01:43So we use `olama pull` and then the name of the model, and then hit enter, and it will download the model

01:49Once you've downloaded the model, we need to run the web app in order to actually use it

01:52To do that, we run `Olama serve`, but first, we actually need to set the Olama origins to any origin

01:59Basically, any request that comes in, we are just going to allow each request through, regardless of where it came from

02:05So, what we need to do is set this OAMA origins environment variable, and you'll have to use whatever operating system function that you need

02:12For Windows, I use set, and then we set it equal to just a star

02:16Once you set that environment variable, we can then run O Lama serve, and that will start up the web app

02:22If you run into this bind error, what we need to do is we need to close out of the original O Lama running instance

02:28So to do that, I just highlight down here, click the carrot symbol, and then right-click on the llama icon and Then click quit llama

02:34Now, we technically have the model running at this URL here with the port 11434, and you'll see that this is 127.0.0.1, which is localhost

02:43However, for whatever reason, Cursor doesn't allow you to use local host as an accessible endpoint

02:50It requires that endpoint to be online and basically accessible from anywhere

02:54To accommodate that, we can use this tool called Enrock to basically take our local host and make it an online accessible URL

03:00So, we can go to the Enrock download page and scroll down to our operating system and select the installer that we want to download

03:08Now, Enrog does require you to sign up for a free account because you'll have to take an O token and authenticate the CLI tool with your account

03:17Also, in case you're curious, all of these links are in the description, so don't worry if you miss something

03:22Now, through the installation process, if you downloaded it directly, you need to take this executable and put it on your environment path

03:27Now, if you don't want to add it to your environment path, you can reference it directly by just using the full path

03:33So, here I'll just take this nrock.exe exe and I'll drag it onto my PowerShell

03:38Now, the Installation does say that we need to add our O token so that it knows who we are

03:43So, we'll copy this and then paste it in our terminal, but we have to replace that token

03:47So, you can either sign in, or if you haven't signed up already, you can click on this sign up button right inside of this config and run window

03:54Here you can fill out some basic information, or we can just log in with our Google OAuth or GitHub OAuth

04:00Once you've signed in, you might come to this get started page which welcomes you, and on the left hand side, we can click on your O token

04:06Now automatically, whether you just signed up or signed in, you will have an O token given to you, and we can click this eyeball, or we can click the copy button to copy that token

04:15So I'll go ahead and copy it, and then I'll paste it in here to configure

04:20Then once we've successfully configured, and don't worry, I'm going to reset this API key so nobody can use it, we can run Enro, and we do that by typing the HTTP command along with the port that we're using

04:31Now you'll remember on our setup, we have this 11434 for Olama, and then we type d-host- header equals, and then localhost colon, and that same port

04:43Once you've written that out, we can hit enter, and now we have a new URL that we Can use inside of Cursor

04:49Under forwarding, we just need to copy this URL

04:51Now go ahead and open up Cursor, and once you have it up, up in the upper right, we have this cog symbol

04:57We'll click on it, and then on the left, we need to click on models

05:00Once you're in models, scroll down and click on add model

05:02This is where we need to add the name of the model that we downloaded, so I'm adding Gemma 3 col 12B

05:08Then just click the add model button

05:10Now, if we scroll a little bit further down, we'll have this option here where we can override OpenAI base URL

05:16This is what we want to click on

05:18Now that we have that Inrock URL

05:19Running, we can just paste it here, but we also need to add the forward slash v1, so that Cursor can access the web app

05:26For the API key, by default, all it is, is O llama, so we don't need to have any specific API key enabled here, and once we're done with this, we can click on this save button

05:36Now, once we have both the URL and API key, we want to click this verify button

05:39Now we're met with the screen, are you sure you want to enable open AI API key? And we'll click on the button to enable it

05:45Now, you might get this message where it's saying you have an invalid API key, and if we hit show error message, we can see that we get This 404, and the reasoning is that the GPT40 is not found, but we weren't trying to use GPT40

05:57It's because this curl request actually contains the model GPT40 instead of the one that we have downloaded for me, the Jimma 3 model

06:07So how we fix this is kind of dumb, but also it works

06:12Uh, we close out of this, and then we unselect every other model except for the one that we're going to use

06:18So, we can scroll up and just start unselecting every single model, and once we unselect them all except for the one that we want to use, this GIMA 31 12B, we can scroll down and then click verify again

06:30Now, we're met with the same message, and we just click enable OpenAI API key

06:34Now, it goes through the whole verification process, and once it's done, it automatically switches it on

06:38Now, it will also say if you are on the prot trial or pro version that you don't have to use your own OpenAI key, but we're using this configuration with the override in order to use it locally

06:49So now with all of the models unselected, we should be able to go up here to the upper right and click on Gemma 312B and select it as a model

06:58And then we can just ask it a question

06:59So I'm going to ask, what does compression file do? Now, my computer isn't the fastest

07:03So, it's generating a response, and we can see that it clearly works, which is awesome

07:08So, we're using the local LLM, and it's producing some response for us

07:10We can even add a context by saying, "Here's the file," and it's a compression.go file

07:16Now, my computer sounds like a fighter jet as it's trying to load this whole response, but you can see that we have this running locally

07:22However, we do have like an online URL, so it's not completely offline, it's just a local model installation, but it's cool

07:29It looks like we got a response back from the model that we're using, and that's pretty much it

07:34Now, you

07:35You can use any model inside of Ollama, just ran locally, and with that, that is how you set up Cursor to use a local LLM.