meetkai
/

functionary-small-v2.5

@@ -21,12 +21,12 @@ The model determines when to execute functions, whether in parallel or serially,
 ## How to Get Started
-We provide custom code for both converting tool definitions into the system prompts and parsing raw model response into a JSON object containing `role`, `content` and `tool_calls` fields. This enables the model to be able to generate tool calls.
 ```python
 from transformers import AutoModelForCausalLM, AutoTokenizer
-tokenizer = AutoTokenizer.from_pretrained("meetkai/functionary-small-v2.5", trust_remote_code=True)
 model = AutoModelForCausalLM.from_pretrained("meetkai/functionary-small-v2.5", device_map="auto", trust_remote_code=True)
 tools = [
@@ -51,7 +51,6 @@ tools = [
 messages = [{"role": "user", "content": "What is the weather in Istanbul and Singapore respectively?"}]
 final_prompt = tokenizer.apply_chat_template(messages, tools, add_generation_prompt=True, tokenize=False)
-tokenizer.padding_side = "left"
 inputs = tokenizer(final_prompt, return_tensors="pt").to("cuda")
 pred = model.generate_tool_use(**inputs, max_new_tokens=128, tokenizer=tokenizer)
 print(tokenizer.decode(pred.cpu()[0]))
@@ -61,7 +60,7 @@ print(tokenizer.decode(pred.cpu()[0]))
 We convert function definitions to a similar text to TypeScript definitions. Then we inject these definitions as system prompts. After that, we inject the default system prompt. Then we start the conversation messages.
-This formatting is also available via our vLLM server which we process the functions into Typescript definitions encapsulated in a system message and use a pre-defined Transformers chat template. This means that lists of messages can be formatted for you with the apply_chat_template() method within our server:
 ```python
 from openai import OpenAI

 ## How to Get Started
+We provide custom code for parsing raw model responses into a JSON object containing role, content and tool_calls fields. This enables the users to read the function-calling output of the model easily.
 ```python
 from transformers import AutoModelForCausalLM, AutoTokenizer
+tokenizer = AutoTokenizer.from_pretrained("meetkai/functionary-small-v2.5")
 model = AutoModelForCausalLM.from_pretrained("meetkai/functionary-small-v2.5", device_map="auto", trust_remote_code=True)
 tools = [
 messages = [{"role": "user", "content": "What is the weather in Istanbul and Singapore respectively?"}]
 final_prompt = tokenizer.apply_chat_template(messages, tools, add_generation_prompt=True, tokenize=False)
 inputs = tokenizer(final_prompt, return_tensors="pt").to("cuda")
 pred = model.generate_tool_use(**inputs, max_new_tokens=128, tokenizer=tokenizer)
 print(tokenizer.decode(pred.cpu()[0]))
 We convert function definitions to a similar text to TypeScript definitions. Then we inject these definitions as system prompts. After that, we inject the default system prompt. Then we start the conversation messages.
+This formatting is also available via our vLLM server which we process the functions into Typescript definitions encapsulated in a system message using a pre-defined Transformers Jinja chat template. This means that the lists of messages can be formatted for you with the apply_chat_template() method within our server:
 ```python
 from openai import OpenAI