Functionary Medium V3.2
Functionary Medium V3.2 is a unique language model that can interpret and execute functions/plugins, making it a powerful tool for tasks that require more than just text generation. It's designed to analyze functions, understand their outputs, and decide when to execute them, whether in parallel or serially. This model is capable of intelligent parallel tool use, providing relevant responses grounded in the outputs, and even deciding when not to use tools. With its ability to support code interpreters and its open-source nature, Functionary Medium V3.2 is a remarkable alternative to other models like GPT-4. But what makes it truly special is its ability to execute functions only when necessary, making it a more efficient and practical choice for real-world applications. So, how can you leverage this model's capabilities to streamline your workflow and get more done?
Table of Contents
Model Overview
The Functionary-Medium-V3.2 model is a unique language model that can interpret and execute functions/plugins. It’s like having a super smart assistant that can understand and use different tools to help with tasks.
Capabilities
Capable of generating both text and code, this model outperforms many open-source chat models across common industry benchmarks.
What can it do?
- Intelligent parallel tool use: It can analyze functions/tools outputs and provide relevant responses grounded in the outputs.
- Decide when to use tools: It can decide when to not use tools/call functions and provide normal chat responses.
- Support code interpreter: It can understand and execute code, making it a great tool for developers.
How does it work?
- Function definitions: You provide function definitions as JSON Schema Objects, similar to OpenAI GPT function calls.
- Trigger functions as needed: The model determines when to execute functions, whether in parallel or serially, and can understand their outputs.
- Provide relevant responses: It can analyze functions/tools outputs and provide relevant responses grounded in the outputs.
How to Use
To use the Functionary-Medium-V3.2 model, you’ll need to:
- Install the required libraries and tools.
- Define the functions and tools you want to use.
- Use the model to execute the functions and provide responses.
Here’s an example of how to use the model:
from transformers import AutoModelForCausalLM, AutoTokenizer
# Define the model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("meetkai/functionary-medium-v3.2")
model = AutoModelForCausalLM.from_pretrained("meetkai/functionary-medium-v3.2", device_map="auto", trust_remote_code=True)
# Define the tools and functions
tools = [
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Get the current weather",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA"
}
},
"required": ["location"]
}
}
}
]
# Use the model to execute the function
messages = [{"role": "user", "content": "What is the weather in Istanbul and Singapore respectively?"}]
final_prompt = tokenizer.apply_chat_template(messages, tools, add_generation_prompt=True, tokenize=False)
inputs = tokenizer(final_prompt, return_tensors="pt").to("cuda")
pred = model.generate_tool_use(**inputs, max_new_tokens=128, tokenizer=tokenizer)
print(tokenizer.decode(pred.cpu()[0]))
Performance
The Functionary-Medium-V3.2 model is a powerhouse when it comes to executing functions and plugins. But how does it perform in terms of speed, accuracy, and efficiency?
Speed
Imagine you’re asking the model to get the current weather in Istanbul and Singapore. How fast can it respond? With the Functionary-Medium-V3.2 model, you can expect quick responses, thanks to its ability to analyze functions and tools outputs and provide relevant answers grounded in those outputs.
Accuracy
But speed is not everything. What about accuracy? Can the model provide accurate results? The answer is yes! The Functionary-Medium-V3.2 model is designed to understand when to execute functions and when to provide normal chat responses. This means you can trust the model to provide accurate and relevant information.
Efficiency
Efficiency is also crucial when it comes to AI models. Can the Functionary-Medium-V3.2 model handle multiple tasks at once? Yes, it can! The model can analyze functions and tools outputs in parallel, making it an efficient choice for tasks that require multiple functions to be executed.
| Task | Functionary-Medium-V3.2 | ==Other Models== |
|---|---|---|
| Speed | Quick responses | Slower responses |
| Accuracy | Accurate results | Less accurate results |
| Efficiency | Handles multiple tasks at once | Struggles with multiple tasks |
Limitations
The Functionary-Medium-V3.2 model is an impressive tool, but it’s not perfect. Let’s talk about some of its limitations.
Complexity and Nuance
The model can struggle with complex or nuanced scenarios. For example, if you ask it to analyze a long piece of text or understand a subtle joke, it might not always get it right.
Function Execution
The model determines when to execute functions, but it’s not always clear why it makes certain decisions. This can lead to unexpected results or errors.
Tool Usage
The model is designed to use tools intelligently, but it’s not perfect. It might not always choose the best tool for the job or use it in the most effective way.
Limited Context
The model can only understand the context of the conversation up to a certain point. If the conversation gets too long or complicated, it might start to lose track.
Error Handling
The model can make mistakes, and it’s not always clear how to handle those errors. This can lead to frustration or confusion.
Comparison to Other Models
The Functionary-Medium-V3.2 model is often compared to ==Other Models==, but it’s not always clear how it stacks up. For example, GPT-4 might be more accurate in certain situations, but the Functionary-Medium-V3.2 model might be more flexible.
Code Interpreter
The model’s code interpreter is a powerful tool, but it’s not perfect. It might not always understand the code correctly, or it might not be able to execute it efficiently.
Getting Started
Getting started with the model can be a bit tricky. It requires some technical knowledge and setup, which can be a barrier for some users.
Example Use Cases
Let’s take a look at some example use cases to illustrate these limitations:
- Asking the model to analyze a long piece of text and summarize it
- Using the model to execute a complex function with multiple inputs
- Comparing the model’s performance to ==Other Models== in a specific task
These are just a few examples, but they illustrate the kinds of challenges you might face when working with the Functionary-Medium-V3.2 model.
Format
The Functionary-Medium-V3.2 model uses a transformer architecture and accepts input in the form of JSON objects containing role, content, and tool_calls fields.
Input Format
The model expects input in the following format:
{
"role": "user",
"content": "What is the weather in Istanbul and Singapore respectively?"
}
You can also define tools/functions as JSON objects, like this:
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Get the current weather",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA"
}
},
"required": ["location"]
}
}
}
Output Format
The model’s output is a JSON object containing the role, content, and tool_calls fields. For example:
{
"role": "assistant",
"content": "The weather in Istanbul is sunny, and the weather in Singapore is rainy."
}
Special Requirements
- The model requires a specific prompt template, which can be generated using the
apply_chat_template()method. - The model can execute functions in parallel or serially, depending on the input.
- The model can analyze the outputs of functions and provide relevant responses.
Example Code
Here’s an example of how to use the model:
from transformers import AutoModelForCausalLM, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("meetkai/functionary-medium-v3.2")
model = AutoModelForCausalLM.from_pretrained("meetkai/functionary-medium-v3.2", device_map="auto", trust_remote_code=True)
tools = [...] # define your tools/functions here
messages = [...] # define your input messages here
final_prompt = tokenizer.apply_chat_template(messages, tools, add_generation_prompt=True, tokenize=False)
inputs = tokenizer(final_prompt, return_tensors="pt").to("cuda")
pred = model.generate_tool_use(**inputs, max_new_tokens=128, tokenizer=tokenizer)
print(tokenizer.decode(pred.cpu()[0]))
Note that you’ll need to replace the tools and messages variables with your own definitions.
