Curated AI

The Converse API offers a unified method to access large language models (LLMs) on Amazon Bedrock, facilitating turn-based communication between users and the generative AI model. It also standardizes tool definitions for models capable of tool use (also known as “function calling”).

Tool use enables a large language model to instruct the calling application to execute a function with parameters provided by the model. The functions and their supported parameters are delivered to the model alongside a prompt. It’s crucial to understand that the large language model does not directly invoke a function; instead, it returns JSON, leaving the actual execution to the calling application.

Using tools with the Amazon Bedrock Converse API involves these steps:

• The calling application sends (A) tool definitions and (B) a triggering message to the large language model.
• If the request aligns with a tool definition, the model generates a tool use request, specifying the parameters for the tool.
• The calling application retrieves the parameters from the model’s tool use request and forwards them to the corresponding local function.
• The calling application can then use the tool’s result directly or send it back to the model for a follow-up response.
• The model either provides a final response or requests another tool.

One of the key advantages of the Converse API is its support for tool use, also known as function calling. This capability allows Amazon Bedrock models to request the use of predefined tools, enhancing the models’ ability to generate accurate and contextually relevant responses. By incorporating tool definitions into your interactions, you can optimize the performance of your AI models. The integration of tools with the Converse API marks a significant step forward in creating more efficient and effective AI-driven applications.

Let’s look at the Python code using boto3 with the Converse API:

import boto3
import json
import math
from botocore.exceptions import ClientError

client = boto3.client("bedrock-runtime", region_name="us-east-1")
model_id = "anthropic.claude-3-haiku-20240307-v1:0"

def cosine(x):
    return math.cos(x)

tool_config = {
    "tools": [
        {
            "toolSpec": {
                "name": "cosine",
                "description": "Calculate the cosine of x.",
                "inputSchema": {
                    "json": {
                        "type": "object",
                        "properties": {
                            "x": {
                                "type": "number",
                                "description": "The number to pass to the function."
                            }
                        },
                        "required": ["x"]
                    }
                }
            }
        }
    ]
}

messages = [
    {
        "role": "user",
        "content": [{"text": "What is the cosine of 10?"}]
    }
]

try:
    response = client.converse(modelId=model_id, messages=messages, toolConfig=tool_config)
    tool_request = response['output']['message']['content'][0]['toolUse']

    ##### Implement the tool logic here #####
    tool_result = cosine(tool_request['input']['x'])
    ##########################################

    tool_result_message = {
        "role": "user",
        "content": [
            {
                "toolResult": {
                    "toolUseId": tool_request['toolUseId'],
                    "content": [{"json": tool_result}]
                }
            }
        ]
    }

    final_response = client.converse(modelId=model_id, messages=[tool_result_message])
    print(final_response['output']['message']['content'][0]['text'])

except ClientError as e:
    print(f"ERROR: {e}")