chat template doesn't include tools
Hi mistral team,
nice to see a new model from you guys, thanks a lot.
in the jinja chat template we don't have anything related to tool (not available nor to put the tools result in the history of messages sent to the model), is it intended?
ollama do include it on their side:
https://ollama.com/library/mistral-small/blobs/5de2b8ebfbdd
{{- range $index, $_ := .Messages }}
{{- if eq .Role "system" }}[SYSTEM_PROMPT] {{ .Content }}[/SYSTEM_PROMPT]
{{- else if eq .Role "user" }}
{{- if and (le (len (slice $.Messages $index)) 2) $.Tools }}[AVAILABLE_TOOLS] {{ $.Tools }}[/AVAILABLE_TOOLS]
{{- end }}[INST] {{ .Content }}[/INST]
{{- else if eq .Role "assistant" }}
{{- if .Content }} {{ .Content }}
{{- if not (eq (len (slice $.Messages $index)) 1) }}</s>
{{- end }}
{{- else if .ToolCalls }}[TOOL_CALLS] [
{{- range .ToolCalls }}{"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}}
{{- end }}]</s>
{{- end }}
{{- else if eq .Role "tool" }}[TOOL_RESULTS] {"content": {{ .Content }}}[/TOOL_RESULTS]
{{- end }}
{{- end }}
thanks a lot
We've tested function calling only with vLLM: https://huggingface.co./mistralai/Mistral-Small-24B-Instruct-2501#function-calling
The model should work very well for function calling tasks!
Can you give this a try?
Also, we'd be more than happy about any contribution to make function calling work with HF format.
It was working fine before and recent commit added strftime and it is bugging at Text-Generation-Inference now
@patrickvonplaten I was gonna test that today as well. That works without applying the template extension from OP or did you include it?
Also did you try this on openai compatible vllm endpoint or just the offline inference?
On TGI , it works without op template. Now it is broken after they included strftime