00:00In this tutorial, I'm going to show you how to get a local language model running on your computer

00:06So, the first thing you're going to need to do is you're going to need to download your chosen language model

00:09You don't want to download anything particularly big because it's not going to run particularly well on your GPU or whatever system you're using if it is a huge language model, technically a large language model

00:24So, you're going to want to choose something small

00:26I like smaller Llama

00:28Just choose the, um, small or second smallest version of that

00:31You can find that on Hugging Face

00:32Just download all the files and put them in a directory

00:35And the directory above that, include, um, just have your query.py, whatever code you're using to run

00:44This is so first thing we're going to do is we got to build a querying piece of code

00:54We're just going to use some simple Python for this, and this is just so we can check what the language model outputs and we can see if our training works well, just to utilize the model at all

01:03We're going to need a few modules to get this working, and I'll list them here

01:08You just pip install or Conda install, whatever you want to use: Transformers, Torch, Protobuf, SentencePiece, Datasets, and Accelerate

01:25And it has to be at least 2.6

01:28You also need to note that you will need to use Python 3.7 to Python 3.10 because otherwise you will not be able to use Torch properly

01:36So what you need to do once you've got all that installed is you need to need to load it

01:46We need To load, we need to get the model and the tokenizer, and we also need to get torch

02:00So, the next thing we need to do is we need to actually load the model

02:03So, I'm going to print "loading"

02:09Model equals, it's just this part of the module here, and we get from pre-trained, and we need to use the directory that the model is in

02:37So, we're going to get the tokenizer and the model, and then we need to set up our device

02:46And we're just going to use a CPU for this, but if you do want to do anything seriously, you will want to use your GPU, but that's a lot harder to set up because you need to install, um, CUDA and a few other things to make sure that works correctly

03:01And we're just going to move the model to the device, and then we're just going to have, we actually going to include this in a while loop so we can run this many times

03:09We're going to have our input data, which we're just going to get from the user, so ask the AI

03:18Then we're going to get our tokens from our tokenizer, so this turns this input data that we as a human understand into something that the language model understands

03:35And then we're going to need to do another operation on them here, I think, load them in and all that

03:55And then we need to get our output from the model

03:58So, and then we put our tokens in

04:03We're going to tell it the max new tokens it's allowed, and we're just going to set that to 128

04:12Models like this generally can't have too many

04:19And then we're going to decode what it turns into something that we can understand, and we Don't actually need the special, the special tokens because those aren't something we can understand

04:38And then we're going to full loop through the text and print it all out

04:45So this should work

04:47And the model has loaded, and we're just going to ask it, just "hello." And I seem to have made a typo somewhere

04:55Uh, there we go

05:02It's a very simple mistake

05:10I'm just going to ask it, "hello." And I've made another simple mistake there

05:18This is why we use a proper IDE

05:27I just haven't set up one on this development, on this computer

05:36Again, why we set up a proper IDE

05:42And it's returned something

05:48It's not, well trained, so it's not going to be anything that's particularly useful

05:55It's just some basic contents of its training data

06:00For training, we're going to need a lot

06:03More imports

06:06So I've just written these out already

06:08Um, I didn't explicitly tell you to import pandas, but I believe that pandas is included in one of the other imports

06:16Um, when you download it, if it's not, just download pandas as well

06:22Um, so first thing we're going to have to do is we're going to have to regen our data

06:26I'm just using a really simple P CSV, which is actually a text document

06:33And then we're going to have to make our data set, so that should be very easy

07:03Then we're going to have to get our tokenizer, uh, same as last time, same place

07:10And then we're going to define a tokenized function

07:17So we're going to use this a little bit, um, so that I keep trying to define keywords

07:25I'm used to usually programming in other languages

07:28Those are not

07:40Keywords, sorry

07:52Toiz data equals

08:04Um, we're going to have to map this

08:17Then we have to create a data collector and assign the tokenizer

08:41Uh, then we're going to have to load the model like we did before, just, just on here

09:06And then we should pad

09:11We should assign the pad token

09:16This is just something that needs to be done to ensure it does work correctly

09:31And then we need to resize

09:46So it, we need to resize and then we need to assign the training AR them

10:01And so we're going to set our the output directory, and that's just going to be, um, results, results

10:08Yeah

10:15And then the evaluation strategy, uh, would be Epoch

10:22And then learning rate equals 3.3 * 10^-5

10:27Now, that is a good rate

10:30I found that anything substantially higher than that, um, is going to, is going to run into

10:55problems with things being learned way too quickly, and if you get some text, it will just latch onto that and can kind of overuse it, essentially overtraining

11:04These batch sizes are sort of also something you would want to keep this size if you're dealing with a GPU with a limited VRAM

11:19So we're just going to set three epochs

11:22[Music] And weight decay, I'm also going to set the trainer, trainer up as well, and that's we're just going to set the model ID equal to this, the um, the data set equal to, hang on, what did I call that? Um, to tokenized data, and we actually use the same evaluation data set, though if you wanted to set this up, you might want to use different ones

12:35And the collator is going to be, and the tokenizer is just going to be again, the tokenizer

12:49I almost Certainly made a spelling mistake somewhere along here

12:58Train, we get results by [Music] evaluate

13:02By evaluating, we print the results

13:05It'll, I think it'll print

13:08It actually prints a lot more than just this, so it'll keep company while it's running

13:11Now, this one runs particularly long, and we will save the pre-trained and we'll save it so we can edit it again

13:22If you want to train this, you will need to load it from that fine-tuned training place

13:39Now, for data, you're going to want to, um, kind of just write some stuff

13:43I'm using, first thing I thought of was just SC

13:46I don't know why

13:50I'm just going to input a load of stuff about that, some really random useless stuff

13:53Um, and what you want to use is, you want to start with text, then you want to have lines, and each one starts with um and ends with quotation marks

14:03So, we'll just create one here

14:07I know Whiterun is in Skyrim, and Skyrim has a hold called Whiterun

14:20Technically, you generally want longer training pieces of data than this, but this is fine for my use case of just demonstrating

14:31So, after fixing a few small levers, uh, you can just see what I had to fix

14:37It's just spelling mistakes

14:39So, I think Major just like didn't include four in the in the name of the import

14:45Um, that's trained, and you can see it does output a lot of, um, just a lot of stuff to make sure, you know, it's working correctly

14:53And so now we can go and test it

14:55For this bit of code, we're now going to have to repurpose it to read from the fine-tuned model, so we're just going to change

15:03where it loads the model from, and we're going to run that, and we're going to see what it gets

15:11Um, so we're asking the AI, um, and it's still not going to be very good

15:17It's still going to not understand what it's talking about, but as we can see, it sort of includes some of the stuff it's learned

15:31Um, you can sort of see that what it's learned already, um, but it's still very, very, um, it's not very smart

15:51You can see it definitely is spitting stuff back, so you'd have to train an awful lot more for it to be able to properly kind of be useful and be able to talk to it.