ULTRA OFFICE – PROMPT ENGINEERING MEETING

I'm familiar with language models, however how can I create completely custom tools and agents? 
I want to understand what is happening in the reAct (chain of thought) pattern?
It would be great if we have the flexibility to create custom agents from scratch, just in case we need to do something that langchain doesn't let us do easily.

Chain of Thought (CoT) is a novel prompting method designed to encourage large language models (LLMs) to explain their reasoning process by providing exemplars that demonstrate reasoning. This approach often leads to more accurate results in tasks like arithmetic, commonsense, and symbolic reasoning. However, CoT's performance gains are only observed when used with models of around 100 billion parameters, as smaller models tend to produce illogical chains of thought that worsen accuracy.

ReAct, introduced by Yao et al., is a framework that allows LLMs to generate both reasoning traces and task-specific actions in an interleaved manner. By generating reasoning traces, the model can induce, track, and update action plans, handle exceptions, and interact with external sources like knowledge bases or environments. This enables LLMs to retrieve additional information, leading to more reliable and factual responses. 

ReAct has proven effective across various tasks such as question answering, fact verification, and interactive decision-making benchmarks, outperforming imitation and reinforcement learning methods while maintaining human interpretability and trustworthiness.

reAct and Chain of Thought

 Hey, welcome back. And in this video, we're gonna have a look at how we can potentially use and create custom tools. Uh, using Lang chain, Uh, we're gonna code up a version of react ourselves. So we really understand what Lang chain and a lot of these kinds of auto G B T and baby AGI are doing underneath the hood. 

It's really important that we go beyond just creating prompts with language models and we've been tasked with. Creating some custom tools. So let's have a look and dive into, what we are looking at. So we are first importing the regular expression package and this open AI. Thing from Lang chain. And basically that's a way for us to call the GPT three model inside of Lang chain. 

Our objective is the following. We need to hand code a simple framework for understanding what the react pattern is with language models and how we can use this to build custom tools. React focuses on a couple of different things, such as chain of thought reasoning. A good example of chain of thought, for example, is where you are doing. Uh, what is Leonardo DaVinci's wife's age squared. 

We first have to break that down into a series of sub problems or sub thought. First we have to search for what Leonardo's uh, Leonardo's wife is. Then we have to find her age and then we have to square that ages number. And so we can kind of think of this as a chain of fo a chain of reasoning. And so the react pattern follows a similar kind of process where we observe the environment. We interpret the environment we've a thought, We decide on a relevant action to take, and that could be doing a Google search. It could be using a calculator tool. 

We act on the environment and then we repeat steps one to four until we've either found a solution. i e I found the final answer, or we hit some sort of max iterations inside the loop. To start with, let's explore how we can extract kind of the payloads that we might ex. Uh, get back from a language model when we're working with text. So as the language model kind of gives us these relevant actions that it might want to take with an input into the relevant tools, such as search on Google search on Wikipedia or a search on Google. 

Uh, we wanna catch at the last action input. Uh, And the reason why is uh, when chain of thought is happening, we're going through a se sequential. Uh, series of thoughts, but we're only interested in the last action and action input that we then need to put into the language model. And we've got some compiled red rejects expressions to, to deal with this. 

Uh, and this is for both the action and the action input pattern. And so we can use dot find all to find all of the relevant expressions. And then we can say, get me the last one. If there is actions else. none, and do the exact same for action inputs. We can see that when we run this better of code. 

We get the last action being search on Google and the last action input being some other query. Let's have a, just a quick look at what these redx is, is, uh, and and sort of breaking it down step by step. So the co I basically is an inline flag that makes the red rejects case intensive. So it doesn't matter if the language model uses action action or any kind of capitalization, we'll be dealing with that. 

Then we've got the action part of this, which is looking for the keyword action. Um, and as well as that, we then have the backslash slash Uh, back slash S star. Which matches against zero or more white space characters. Uh, we then have the colon. Uh, and then as well as that, we then match against zero more white space characters. again, we then have a capture group set up after that. Um, and this is basically denoted by the parenthesis, So it matches against all of those, which are not a new line character. Um, so we're matching everything apart from new lines. 

Um, and so that's kind of what the carrot is doing here. Uh, it negates the character class. So we're capturing everything after that co. Um, apart from anything on a new line, we then run it on multiline. So basically this pattern should be uh, app, pen, you know, applied line by line. Um, and yeah, you can kind of think of, this is a bit more explanation on how that carrot is kind of working. 

Uh, if we have a look at the action input pattern, you'll see, it's very similar. So avoiding the capitalization with the, uh, you know, the question mark and the eye, and then we've got the action input and then we're, you know, any white space before the colon. And then we're having the colon, then any white space after, and then we have this capture group that's capturing everything, but the new line. 

Uh, and so this allows us to match both the action or the tool name that we want to do, and also the inputs into that tool. And so we can kind of wrap this up in a nice python function called extract Glass action and input. And you'll see, given the previous text, we're able to find what the action is, which is search on Google. And uh, this is the tool. And then what we want to put into the tool is the action input. So some other query, um, so given that we kind of need to find out whether the language model has the final answer. 

We will use the following template. Uh, I will, I found the answer, right? So we want the language model to say, Hey, I found the answer. And so we will also build some rejects for this, uh, which is very similar. Uh, so we found the answer and you can see we've got this final answer is now showing up, given some text. So we've worked out how we can. 

Uh, you know, basically extract the tool name and the tool. Uh, inputs and we've looked at how we can figure out when the language model is truly finished with the chain of thought reasoning. Uh, so what we went want to do is also make a function to extract the final answer and see if it exists. And we can see once we've done that, we've got the final answer. And so now we've got all the, all the components. 

Uh, needed to kind of create this react pattern. Um, so we we define a fake tool. uh, And I would encourage you to, to play around with fake tools until you really understand the pattern. Uh, we then add the function here. And we, and give a, a description of what the tool is. The base prompt is very, very interesting. It's it's important that we never hallucinate the tool result. And we wait with a stop sequence. 

Uh, so that the tool. Uh, you know, the language model doesn't try and hallucinate what the tool might produce Uh, Let's have a read of the base prompt to understand this. You will attempt to solve the problem of finding the answer to a question. Use chain of thought, reasoning to solve through the problem using the following pattern. 

Observe the original question, create an observation with the following pattern. Observation, observation text, create a thought based on the observation with the following pattern, use tools to act on the fort with the following pattern action and action input. We also specify and tell it, do not guess or assume the tool results instead, provide a structured output. That includes the action and action input. You Have access to the following tools. We then say original problem is Jason  Darrio with a partner. We then make a call to uh, a language model. So we get the the keyword arcs and we put our stop sequence of tool result. Uh, and then as well as that, we then can print everything out. So we we call the language model. Once we then extract the action of the tool and the action input. Uh, so what is going to be inputted into the tool? Uh, and then we get the function with the input. So we get, we call the function with the input, get the tool results. And then we've got some print statements here to show what, what tool the agent decided to take given the thoughts, um, and, uh, and given the tools that it's been given. 

Uh, and so then we've got the final one here based on the provided tool results. you either provide the next observation action input, or the final answer if available. If you're providing the final answer, you must return the following pattern. I found the answer fine. So let's have a look at the print statements. 

To start with given the question. of this one here, which is uh, Is Jason Dillo with a partner. The observation is Jason Dillo is a popular singer. It might be possible to search on Google, to find out if he has a partner. Search on Google, Jason to relay partner. The agent, therefore has opted to use uh, The following tool search on Google and the, the input into this tool is Jason Dillo partner. The tool result is Jason Dillo Doesn't have a WI or partner. And remember our tool result came. From whatever the function we called inside of the language model. Right. Uh, the second prompt shows. Uh, you know, 

Based on the provided tool results. Jason Dillo doesn't have a partner. Eva provides the next observation. Right? So that's the part of the prompt and the model outputs. I found the original answer. Right. And so we found the answer here. Uh, and, and it's worth noting that we've done this without a for loop, just to get a good grasp on how the react pattern is working. However, we could go through a series of thoughts, Uh, and you can imagine adding a calculator as an extra tool. Um, into your arsenal. Uh, and that would allow you to then take the age of this partner, for example, and, and square that, or multiply that by a certain number. Uh, we are able to give language models a range of different tools, such as web browsing. And you'll find this is the case in some of the latest plugins with chat G P T where you can do web browsing. 

It's important that you're able to understand this pattern, uh, and this is a really great way for you to get. To grips with what's happening inside of auto G P T and baby AGI, and really understand kind of how you can create a single agent, Uh, from scratch. 

Now that you understand how to create single agents using the react pattern. There's. an in enhanced version of this on site of Lang Chain's documentation. You'll find that, for example, you can go to the agent section and they've got everything on this getting started. So we have the ability to load tools, create an agent type initialize, an agent. 

Uh, and you'll see that the zero shot react description. is very similar to what we were implementing with our custom base prompt. Uh, and it's just good to know that you're able to do this rather than solely relying on land chain Land chain's gonna be great for when you need to do a single agent. Uh, in a very standardized way, but having an understanding of how the react framework works underneath the hood is gonna allow you to choose when to use Lang chain or when to build your own custom single agents.

How did we extract the payloads from a Language Model?

What is the purpose of the extract_last_action_and_input function?

Are you looking for an efficient way to understand the reAct pattern? 

Learn how to quickly create single llm agents using regex, Python and OpenAI.

Vexpower Learning Certificate of Completion

Congratulations kelsie naugler 💪