What's the difference between Langchain Agents and OpenAI Functions?

Update: This article has been updated in November 2023 to include changes in the OpenAI SDK version 1.1.1+ and new features announced during OpenAI DevDay 2023.

Both Langchain agents and OpenAI functions let us connect AI to databases, APIs, and other external systems. Are they interchangeable?

Yes. The implementation is different, but we can get the same results using both.

OpenAI Functions

OpenAI Functions allow greater control over functions called by the AI because we have to parse the AI’s response, figure out which function the AI wants to call and pass the arguments to the function. We can intervene and modify the called function or its parameters at every stage.

Suppose we want to let AI use a REST API without specifying the operations it can do. We provide a generic REST API client and let AI decide which HTTP method and what parameters to use.

Defining a Function for AI

First, we must prepare a description of the available functions. In the case of the single-function REST API client, the description may look like this:

self.functions = [
{
    "type": "function",
    "function": {
        "name": "call_rest_api",
        "description": "Sends a request to the REST API",
        "parameters": {
            "type": "object",
            "properties": {
                "method": {
                    "type": "string",
                    "description": "The HTTP method to be used",
                    "enum": ["GET", "POST", "PUT", "DELETE"],
                },
                "url": {
                    "type": "string",
                    "description": "The URL of the endpoint. Value placeholders must be replaced with actual values.",
                },
                "body": {
                    "type": "string",
                    "description": "A string representation of the JSON that should be sent as the request body.",
                },

            },
            "required": ["method", "url"],
        },
    }
}
]

In addition to the description, we need an implementation of the function. After all, we will be responsible for calling the function later when we receive a response indicating that AI wants to call a function.

def call_rest_api(self, arguments):
    # reques.in is a hosted, fake REST API that we can use for testing
    url = 'https://reqres.in' + arguments['url']
    body = arguments.get('body', {})
    response = None
    if arguments['method'] == 'GET':
        response = requests.get(url)
    elif arguments['method'] == 'POST':
        response = requests.post(url, json=body)
    elif arguments['method'] == 'PUT':
        response = requests.put(url, json=body)
    elif arguments['method'] == 'DELETE':
        response = requests.delete(url)
    else:
        raise ValueError(arguments)

    if response.status_code == 200:
        return json.dumps(response.json())
    else:
        return f"Status code: {response.status_code}"

The function is generic and handles any HTTP request sent to any endpoint. We have to list the available endpoints and supported HTTP methods. I prefer to gather such arguments in one variable that I will later use to generate the prompt:

self.available_apis = [
        {'method': 'GET', 'url': '/api/users?page=[page_id]', 'description': 'Lists employees. The response is paginated. You may need to request more than one to get them all. For example,/api/users?page=2.'},
        {'method': 'GET', 'url': '/api/users/[user_id]', 'description': 'Returns information about the employee identified by the given id. For example,/api/users/2'},
        {'method': 'POST', 'url': '/api/users', 'description': 'Creates a new employee profile. This function accepts JSON body containing two fields: name and job'},
        {'method': 'PUT', 'url': '/api/users/[user_id]', 'description': 'Updates employee information. This function accepts JSON body containing two fields: name and job. The user_id in the URL must be a valid identifier of an existing employee.'},
        {'method': 'DELETE', 'url': '/api/users/[user_id]', 'description': 'Removes the employee identified by the given id. Before you call this function, find the employee information and make sure the id is correct. Do NOT call this function if you didn\'t retrieve user info. Iterate over all pages until you find it or make sure it doesn\'t exist'}
    ]

Now, we have to define AI’s task and the available functions. For this, we prepare a few messages before the user’s prompt.

self.messages = [
    {"role": "user", "content": "You are an HR helper who makes API calls on behalf of an HR representative"},
    {"role": "user", "content": "You have access to the following APIs: " + json.dumps(self.available_apis)},
    {"role": "user", "content": "If a function requires an identifier, list all employees first to find the proper value. You may need to list more than one page"},
    {"role": "user", "content": "If you were asked to create, update, or delete a user, perform the action and reply with a confirmation telling what you have done."}
]

Finally, we can implement interactions with AI. We define a function call_ai. The function accepts the user’s prompt and passes the prompt to the OpenAI model with the context description defined earlier. The context description also contains a description of the available function.

We call a function if the AI responds with a message containing a function_call. The message includes the function name and its parameters.

def call_ai(self, new_message):
    if new_message:
        self.messages.append({"role": "user", "content": new_message})

    response = self.client.chat.completions.create(
        model="gpt-3.5-turbo-1106",
        messages=self.messages,
        tools=self.functions,
        tool_choice="auto",
    )

    msg = response.choices[0].message
    self.messages.append(msg)
    if msg.content:
        logging.debug(msg.content)
    if msg.tool_calls:
        for tool_call in tool_calls:
            # we may get a request to call more than one function(!)
            function_name = tool_call.function.name
            function_args = json.loads(tool_call.function.arguments)
            if function_name == 'call_rest_api':
                # ['function_call']['arguments'] contains the arguments of the function
                logging.debug(function_args)
                # Here, we call the requested function and get a response as a string
                function_response = self.call_rest_api(function_args)
                logging.debug(function_response)
                # We add the response to messages
                self.messages.append({
                    "tool_call_id": tool_call.id,
                    "role": "tool",
                    "name": function_name,
                    "content": function_response
                })

        self.call_ai(new_message=None) # pass the function response back to AI
    else:
      # return the final response
      return msg

On the one hand, such an implementation makes the code quite verbose. As you see below, we have to select the Python function to call, pass the arguments, add the response to the chat history (as a message with role set to tool), and call the AI again with an updated chat history.

On the other hand, we retain complete control over called functions. We may call a different function, change parameters, or pass a fake response to AI instead of calling a function.

With the OpenAI implementation, we can also easily create long-running workflows where we store the chat history in a database, wait a few hours or days for a response to the function call requested by AI, and continue the workflow when we receive it.

Function Calls with Langchain Agents

Langchain Agents hide the complexity of calling functions. We still have to provide a function description and the implementation or use one of the available Langchain toolkits. The Langchain toolkits are a shortcut allowing us to skip writing a function and its description. Instead, we use the code prepared by toolkit authors.

With Langchain, we must choose one of the available agent types. The agent type defines the prompt to inform the model about the available functions.

React Agent

Agent types implement various interaction styles. For example, in the conversational-react-description Agent, the instructions part of the prompt looks like this:

To use a tool, please use the following format:

Thought: Do I need to use a tool? Yes
Action: the action to take, should be one of [{tool_names}]
Action Input: the input to the action
Observation: the result of the action

When you have a response to say to the Human, or if you do not need to use a tool, you MUST use the format:

Thought: Do I need to use a tool? No
{ai_prefix}: [your response here]

## https://github.com/langchain-ai/langchain/blob/9731ce5a406d5a7bb1878a54b265a6f7c728effc/libs/langchain/langchain/agents/conversational/prompt.py

When this prompt is used, Langchain lets AI generate output until it produces the first new line character after Action Input: . At this point, Langchain interrupts the AI call, finds the function using the name returned in the Action: line, and passes the Action Input: value as the parameter.

Later, when the function returns a response, the response is added to the prompt in the Observation: line, and Langchain calls the LLM again to continue handling the prompt. The LLM may produce another Action: + Action Input: pair to call another function or generate a line starting with Final Answer: to return the response to the user.

A popular zero-shot-react-description Agent has almost the same prompt. I used this kind of agent in many articles about Langchain, so if you are interested, look at one of them. For example, the one about Creating an AI Data Analyst bot for Slack that can lookup data in your database

OpenAI Functions in Langchain

We can choose an Agent type that uses the OpenAI functions but hides the complexity of selecting the function and passing the arguments.

Instead, we use the standard code structure of configuring a Langchain agent but choose the OPENAI_FUNCTIONS AgentType.

Unfortunately, Langchain tries to pack the entire function description into a single message, and long descriptions easily exceed the message length limit. Because of the implementation they chose, I had to shorten the prompt. If I needed all endpoints, I would split the call_rest_api function into multiple functions, each handling a single endpoint. However, for the sake of a tutorial, I decided to shorten the description and support only two endpoints.

import json
import requests
from langchain import OpenAI
from langchain.agents import initialize_agent, Tool
from langchain.agents import AgentType
from langchain.chat_models import ChatOpenAI


def call_rest_api(method_and_url):
    method, url = method_and_url.split(' ')
    url = 'https://reqres.in' + url
    response = None
    if method == 'GET':
        response = requests.get(url)
    elif method == 'DELETE':
        response = requests.delete(url)
    else:
        raise ValueError(method)

    if response.status_code == 200:
        return response.json()
    else:
        return f"Status code: {response.status_code}"

available_apis = [
        {'api ': 'GET /api/users?page=[page_id]', 'description': 'Lists employees. The response is paginated. You may need to request more than one to get them all. For example,/api/users?page=2.'},
        {'api': 'DELETE /api/users/[numeric user_id]', 'description': 'Removes the employee identified by the given id. Before you call this function, find the employee information and make sure the id is correct. Do NOT call this function if you didn\'t retrieve user info. Iterate over all pages until you find it or make sure it doesn\'t exist'}
    ]

api_description = json.dumps(available_apis)

function_description = f"""Sends a request to the REST API.

Accepts a single parameter containing two parts:
* method - The HTTP method to be used (GET, POST, PUT, or DELETE)
* url - The URL of the endpoint. Value placeholders must be replaced with actual values.

For example: GET /api/users?page=1 or DELETE /api/users/13

To find users by name, use the GET method first.

Available endpoints:
{api_description}"""

llm = ChatOpenAI(temperature=0, model="gpt-4", openai_api_key='sk-...')
tools = [
    Tool(
        name = "REST",
        func=call_rest_api,
        description=function_description
    )
]

agent = initialize_agent(tools, llm, agent=AgentType.OPENAI_FUNCTIONS, verbose=True)
agent.run("Fire Lawson")

In the console, we see the verbose log of the agent’s actions:

Invoking: `REST` with `GET /api/users?page=1`
{'page': 1, 'per_page': 6, 'total': 12, 'total_pages': 2, 'data': [...

Invoking: `REST` with `GET /api/users?page=2`
{'page': 2, 'per_page': 6, 'total': 12, 'total_pages': 2, 'data': [{'id': 7, 'email': 'michael.lawson@reqres.in', 'first_name': 'Michael', 'last_name': 'Lawson', 'avatar': 'https://reqres.in/img/faces/7-image.jpg'}, ...

Invoking: `REST` with `DELETE /api/users/7`
Status code: 204

User Lawson has been successfully removed from the system.

Which one to choose?

If I had a simple function with short descriptions, I would choose Langchain because Langchain hides many implementation details and lets us focus on preparing the prompts.

However, when the function description no longer fits into a message, we have no other option and must use OpenAI functions directly without Langchain.

Also, when you struggle to get the prompts right, I suggest switching to OpenAI Functions. It won’t magically solve the problems, but you must write the entire function call code. Therefore, you will be familiar with it, and debugging will be much easier.


Do you need help building AI-powered applications for your business?
You can hire me!

Older post

Save time and money by caching OpenAI (and other LLM) API calls with Langchain

How to use Langchain model response and document embeddings caching to save time and money when using Large Language Models

Newer post

Build a question-answering service with AI and vector databases in JavaScript

How to use Langchain.js to build a question-answering service that uses vector databases to store the documents and AI to generate the answers