We're at the GoolyBib HQ on Monday morning and Gustav Gieger, the CEO, is briefing us on the newest task he has for us. He wants us to use OpenAI Fine-Tuning to write in his exact style. We need to collect at least 200 samples, and the cost of the technique is one tenth of 8 cents per 1000 tokens. We need to get the data in the right format and train the model on GPT 3.5 turbo 0 6 1 3. Once the training is completed, we'll have to calculate embedding distance and manual blind testing to make sure we got the right results.

We need to make sure this project is done right. This technique is going to be used to write in my exact style, so getting the right results is essential. We need to collect the samples, get the data in the right format, and train the model. Once we have the results, we'll need to do some manual blind testing to make sure everything is as it should be. Let's get this project moving!

OpenAI recently launched a new feature for their 3.5 model called fine-tuning. This feature allows users to customize their AI models to make them better suited to specific tasks. It is an important and useful development in the realm of AI and machine learning. At its core, fine-tuning involves getting data into the correct format, uploading it to OpenAI, and then training the model on the data. The process can be complicated, but it is not as difficult as it may seem. The first step is to get the data into the correct format. This typically involves converting the data into a JSONL format, where each object is on its own line. Once the data is prepared, it is then uploaded to OpenAI and the model is trained on the data. The benefit of fine-tuning is that the model can be tailored to specific tasks. For example, if you want to train the model to respond to a certain prompt, you can do so by providing examples of that prompt in the data. This allows the model to learn how to respond to a prompt more accurately. Additionally, the model can be fine-tuned to recognize and respond to certain language styles, such as informal language or more technical language.  It is important to note that for fine-tuning to be effective, there needs to be a sufficient amount of data to work with. OpenAI recommends at least 50 observations, but it is generally accepted that 200 observations are needed to get the best results. If you don't have enough data, you can either use prompt engineering or try to generate more data manually. The cost of fine-tuning is also something to consider. OpenAI charges 8 cents per 1000 tokens to train the model, and 1 cent per input and 1.5 cents per output when talking to the model. This is significantly cheaper than GPT-3, which typically charges 3-6 cents per token. As such, fine-tuning is a more cost-effective way to get better results from your AI models. Overall, OpenAI's fine-tuning feature is an important development in the realm of AI and machine learning. It allows users to customize their model to make it better suited to their specific needs. It is important to note that it requires a significant amount of data and can be costly, but the potential benefits are worth it. If you're looking to

OpenAI Fine-Tuning

Alright, now we're going to learn about fine tuning. And as of recording this video, it literally just came out yesterday. So you're getting this fresh the API. For open AI 3.5 model. Is pretty straightforward actually. And they have some details. On the documentation and also the blog post, where they came out with fine tuning as well. I've just followed along with the example here, but I'm going to explain to you what's happening at each stage. The first thing you need to do is get your data into the right format. If you're following along here, you can make a copy of this. My data from my blog is here. And you can open that up. I'll show you what that looks like. This is the format. So it's called Jason L a, which is like Jason, except every object is on it's own line. So you can see there's no parent object. And it just is a list of messages. So the messages are the system message. I've just kept that default, but if you testing, fine tuning for yourself and you already have a prompt, that's working in the system message, you might want to. Use that prompt here. I've just gone with the default. And I think, it takes a lot of experimentation, but you might get better results if you update the system message equally I've just used this really simple prompt which is write the section and then the section headed for the blog posts and then the blog title. What I'd done is I split, I downloaded all 48 of my blood posts. I split them into sections based on the section headers, the H one tags. And then I've put the name of that section. Into into here. And then the. The blog. Title here. Like the AI is in responding. So obviously the AI didn't really respond this is what I'm telling you. It should have responded. So that's how you training it saying like when the user prompts like this. Then give me this type of content. And then my hope is going to train it to talk like me. So that's the actual section text. Yeah. So getting into this format is the first. A challenge. I actually like the. Probably the hardest thing. What I did actually is I cheated a little bit and I used code interpreter. I'll just show you how I did that. I uploaded the blog posts. And then I just asked it to prep it into this format and I gave it one example. I just said, prepping to write the section header for the bulk of this article title and make sure we cover all the blood place in the CSV format is like this. I give it one example. I just took a section header. Why most agencies charge so much for an old posts. And then I paste it in that text manually. Cool. It actually did all the work and I can include. I'll include the code that it used. So I asked it to actually, so one thing is initially it had a bunch of HTML in that, so I asked it to get rid of that. But other than that, it was, it worked really well. And I asked it to package up. All the code. So I'll include this as a. Dot Py file. So hopefully you should be able to just run this or you could give this to Coda temperature and say, can you do that for, but for. For my data. Cool. And then once you ask it to generate its output, you can click and save the file. Cool. So that's how you get the data in the right format. This is how you load my data. So if you just want to test, you can load this from this link. The it's just extracting the fall ID and then pulling the data and then saving it locally. And then this loads the data. So you can see you have 304 observations. Typically with fine-tuning what open AI says is you probably need at least 50 observations in order to start making, noticing a difference in the model. And I've seen in a few papers that it's typically around like 200. 

It's 200 observations that you need in order to get to the point where fine tuning actually shows better results than just general prompt engineering. So I would say, if you have 200 samples and it probably in a good position to start fine tuning if not in that position, then, try and generate 200 samples manually by getting people to label them or you labeling them. Or you could just use prompt engineering for now, and then once you get to 200 samples, then you can do. This type of work. The the code here, research needs to get your 

 you can pay sat in there. You can get that in. Eh, view API keys and you can create a new one. And then this is the coal to the open AI fine tune. I guess this specifically is like the files. Section. So before, it can fine tune on your data. It needs to have your data, right? So this is just a way of uploading that file to open AI. And the space, it just splits it and opens it. And a file path. And then and then you get a response that says here's the file ID. You know how big it is and it was uploaded. Now. One thing I found is that sometimes with larger files, it takes a while to upload. So if you're trying to run the rest of the code, it doesn't work. So I've added this section here, which is just checking on that file. And it just, if you run that and then it just finds the latest file that you uploaded. And then once it says, processed then you. Pretty happy. Then you can keep running. Then then you start a fine tuning job. So again, pretty straightforward. Just call the fine tuning jobs API. Again with European AI key. And it is recommended that you train on this. GPT 3.5 turbo. 0 6 1 3. The reason is that it doesn't say they don't support you pretty for fine-tuning yet. Although they said it's coming in the fall, you can also tune on to other models, but they're the older models like GPT two. So it's not really worth messing with. Or at least. Yeah, not for most of the tasks I've found. Out for four. Cool. So when you call that then you get a job ID. And you can see that, like it's not finished yet. And it says status created. And again, you can just check the status. So if you run that. A few times, like it will say like we're running. If you scroll across. And then eventually they'll say status succeeded. And once it has succeeded, then you can see how long it took. In this case. It took 24 minutes. And then also you can calculate the price. So you can go through this ahead of time, but. The the cost for fine-tuning is as is. A 10th of 8 cents per 1000 tokens. So it costs me $2 22 to train at least $300 for patients. And then you get the actual fine tune model itself. So you can see that here, this, my specific code, it shows up in my account. And then you can just query it just like you. Otherwise word. And so this is just a normal query to to open AI, et cetera, in passing this new model name in. And if it's, I think it's really important that you keep the system message and the con and the prompt style at the same. But I'm getting, I think pretty good results. So This actually really does sound like me. I think. If I scroll back to the beginning, 

Eases a lot of the same kind of informal language, but quite direct, not much fluff. Yeah, like what it's like under the hood, I use that quite a lot. 

Yeah, here we go. I think that we're using figures and stuff like that. Yeah, I think it's I'm actually pretty impressed with the results. And then the other thing I think you should think about is like this was asking it to a section that like it already trained on. But really you want to try and write a section that it hasn't trained on. So here's the top of that. I have written about mathematics. I actually wrote a whole book about mathematics, but not it was only like one blog post on memetics. On my personal blog and it doesn't mention criticisms of Mobotix. So this is like a tangential. Topic. And I haven't written specifically the section, like why isn't metrics assigned? So this is a better test. And what I find is it's still sounds like me. It starts to quote different people. I've never read. But it's it's like doing, it's still doing a pretty good job. So I think that, more testing needs to be done. Ideally, what you would do is you would hold back some. Test data. When you have the 300 or something samples, if you held back like 10, 15% of that and then you could feed in those prompts generate and see how well they did relative to the real task. Sorry. I would say that's good. Good. Yeah, that's a good idea, really with any machine learning. But but yeah, that's how you're going to really tell whether it's done a good job. You can also tell programmatically. So there's like evils functionality and Lang chain or an open AI. You can do manual kind of blind, thumbs up, thumbs down is, does this sound like me or not like sentence by sentence? 

You could also do embedding distance. You could calculate the vector for for the text, the reference text, they actual writing that I did, and then calculate the the embedding for the the text that the fine-tune model generated. And then see what the difference is, see how close it gets, and you can try different fine tuning techniques. So you try with a different system message and see if that improves things and so on. So it is an iterative.  experimental process. The other thing to show you is that these models, there's not much interface, unfortunately. With these things, but these models do show up. So if I refresh and then you can see, I have my fine tunes that show up here. If I want to Say The section. 

What. It is. 

Hello. 





Just around the run. Just submit. 

So when it comes back with. 

Yeah, I. It doesn't. Always work, I think if it goes off off piece that tends to the examples, it doesn't do as good a job. Cool. The the other thing to talk about. Pricing. Here we go for fine tuning. Costs like 8 cents per thousand or less, sorry, one 10th of 8 cents for a thousand tokens to train. And then costs psych. Basically at 1 cent for input and No one and a half cent for output. When you're talking to the model. So if you compare this is what a G T a. 3.5 normally costs. And so you can see that. We're going from. 0.0015 to zero.  so it's an order of magnitude difference in sense of cost, but it's still pretty cheap, right? If you think about. GPT for the 3 cents and 6 cents. A token. And so you can, you're basically getting, much cheaper. Yeah. Much cheaper model than GP. and what a lot of people have been telling me is that you can get better results with a fine tuned. 3.5 muddle first specific tasks, then you can with a general GPT four. So it depends how you position it. It's either a really expensive GP through profile or. Like a really cheap. GPT for replacement for some tasks. So yeah, hopefully this is a useful for you guys. Again, this is like really new, so this might change, but hopefully you understand the kind of thinking behind it and how it works. And what's really great about this. This is pretty accessible. You didn't have to be a machine learning engineer to do this. Like I'm certainly not say. Hopefully. Yeah, you guys can experiment and find some interesting stuff.

How much does it cost to train a GPT-3.5 model?

What is the purpose of holding back a portion of the data when fine-tuning?

What is the format of the data needed for fine tuning?

What is the recommended model to use for fine tuning?

The content you get from ChatGPT sounds dull and robotic. Editing AI content in your voice is tedious.

Vexpower Learning Certificate of Completion

Congratulations Dr Ravi Prasad 💪