README.md

---
license: llama2
datasets:
- marclove/llama_functions
- timdettmers/openassistant-guanaco
language:
- en
library_name: transformers
pipeline_tag: conversational
---
# Model Card for Llama-2 7B Chat Functions

‼️ This model is still in a beta state. It will be retrained at a future data and updated, during which its prompting format may change. If you need to depend on it in its current state, please create your own fork and provide attribution to this original repository. ‼️

Llama Functions is a further fine-tuned version of [Llama-2-7b-chat-hf](https://huggingface.co./meta-llama/Llama-2-7b-chat-hf), using (1) a 50/50 mix of synthetic OpenAPI function calls and (2) chat completions from the [Guanaco subset of the OASST1 dataset](https://huggingface.co./datasets/timdettmers/openassistant-guanaco). 13B & 70B versions are coming soon.

The function calling dataset is mixed with Guanaco in order to maintain accuracy and helpfulness when calling a function is not the appropriate response. Guidelines for use, more detailed information regarding limitations, and eval stats of 7B, 13B, and 70B models.

There is no existing evaluation benchmark to measure the accuracy of function calls, which makes it hard during training to identify when we've maximized the balance of function calling accuracy and chat model performance. I'm working on a custom HF eval for this purpose, but until then I have chosen to mix the two datasets in equal parts to get a proxy of performance for both tasks in the eval & test stats during fine-tuning. The current checkpoint is at 1000 steps, when eval & test loss reached their lowest point.

- **Developed by:** Marc Love
- **License:** [Creative Commons' Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license](https://creativecommons.org/licenses/by-sa/4.0/)
- **Finetuned from:** [Llama-2-7b-chat-hf](https://huggingface.co./meta-llama/Llama-2-7b-chat-hf)

### Model Sources [optional]

<!-- Provide the basic links for the model. -->

- **Repository:** Coming soon
- **Demo:** [llama2-7b-chat-functions](https://huggingface.co./spaces/marclove/llama2-7b-chat-functions)

## Uses

Please note that the synthetic data portion of the dataset was generated using OpenAI models, which may or may not impact your ability to use the dataset, depending on your use case.

## Bias, Risks, and Limitations

No additional bias beyond that of the underlying model [Llama-2-7b-chat-hf](https://huggingface.co./meta-llama/Llama-2-7b-chat-hf) and those introduced by the [Guanaco subset of the OASST1 dataset](https://huggingface.co./datasets/timdettmers/openassistant-guanaco). 

This model can hallucinate function calls that do not exist in the system prompt. While I hope to improve this by iterating on the `llama_functions` dataset, the 7B model will likely continue to struggle with this. I'm hoping to see more accuracy and less hallucination in larger models and plan to experiment with inference strategies, such as [grammar-based sampling](https://github.com/ggerganov/llama.cpp/pull/1773) and classifier-based routing, to improve performance in smaller models.

At the very minimum, I encourage you to validate outputs before attempting to use responses to call any functions. For example, several people have found Pydantic to be a convenient way to both describe functions and validate calls prior to execution.


## Training Details

### Training Data

See the [`llama_functions` dataset](https://huggingface.co./datasets/marclove/llama_functions) for more information.

### Training Procedure 

Coming soon

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->


#### Training Hyperparameters

Coming soon

<!--- **Training regime:** [More Information Needed] <-- fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->

#### Sizes

11B & 70B chat and non-chat versions coming soon

<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->

## Evaluation

Coming soon
<!-- This section describes the evaluation protocols and provides the results. -->


## Citation

```
@misc{LlamaFunctions,
  title = {LlamaFunctions: An Open Dataset of Structured API Calls From Natural Language Prompts},
  author = {Marc Love},
  year = {2023},
  publisher = {HuggingFace},
  journal = {HuggingFace repository},
  howpublished = {\url{https://https://huggingface.co./marclove/llama_functions},
}
```