First commit

README.md

@@ -1,3 +1,64 @@
----
-license: apache-2.0
----
+# Invoker-13B-GPTQ
+
+This repo is a 4-bit quantized GPTQ implementation of [jeffrey-fong/invoker-13b](https://huggingface.co/jeffrey-fong/invoker-13b).
+
+## Model Description
+Invoker is a suite of large language models based on Llama-2 and is finetuned to plan between calling functions and providing responses directly. It behaves similar to OpenAI's function-calling models which intelligently chooses the best function to call among a list of functions provided to it and summarizes the function responses.
+
+This model stands out for its ability to plan between function calls and returning model responses directly. The fine-tuning process was performed with a 4096 sequence length on an 2x A100 80GB machine.
+
+For more details, refer to [https://github.com/jeffrey-fong/Invoker](https://github.com/jeffrey-fong/Invoker)
+
+## Model Usage
+
+#### Prompt Format
+The prompt to the model consists of a list of available of functions to call and the chat messages
+
+You must provide the list of functions to the model. All functions passed in should be in the same JSON format as OpenAI function-calling. If no functions are to be passed to the model, provide `None` in the `Available Functions` Field. 
+
+````text
+Available Functions:
+```json
+<function1 name and description>
+```
+```json
+<function2 name and description>
+```
+A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions. The assistant calls functions with appropriate input when necessary.
+USER: <query>
+ASSISTANT:
+````
+
+or
+
+```text
+Available Functions:
+None
+A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions. The assistant calls functions with appropriate input when necessary.
+USER: <query>
+ASSISTANT:
+```
+
+## Model Training
+The model was trained using QLora which significantly reduces the computational resources required. Training was also accelerated using DeepSpeed Zero Stage 2 which provides fast data parallelism.
+
+## Training Data
+
+We use a variety of sources when building our training dataset. All the datasets are carefully chosen to improve both the conversational and function-calling capability of the model.
+
+#### ToolBench (0830 updated)
+
+ToolBench is an open-source, large-scale and high quality instruction tuning SFT dataset to facilitate the training of LLMs with general tool-use capability. It consists of multi-turn conversations where the assistant, who is presented with several potential functions to call, will call one or multiple functions before returning its response to the user. We had undergone rigorous cleaning of the data where we
+- Removed all datapoints that do not end with the assistant returning a summarized response
+- Cleaned datapoints with unnecessary calls to the same function
+- Changed all function names and descriptions to not include the domain name, so the functions feels more generic
+
+#### ShareGPT-34K
+
+ShareGPT-34K is a filtered dataset containing high quality multi-turn conversations between a user and an assistant. Some of the assistant responses are generated from OpenAI's GPT-3.5-Turbo while some are from GPT-4.
+
+#### OASST1
+
+OASST1 is a human-generated and human-annotated assistant-style conversation corpus. We filtered out the conversations in English.
+
+All the datasets used are under Apache-2.0 License. Therefore, this dataset will also be under the same license.

config.json

@@ -0,0 +1,26 @@
+{
+  "_name_or_path": "jeffrey-fong/invoker-13b",
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "hidden_act": "silu",
+  "hidden_size": 5120,
+  "initializer_range": 0.02,
+  "intermediate_size": 13824,
+  "max_position_embeddings": 4096,
+  "model_type": "llama",
+  "num_attention_heads": 40,
+  "num_hidden_layers": 40,
+  "num_key_value_heads": 40,
+  "pad_token_id": 0,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "tie_word_embeddings": false,
+  "torch_dtype": "float16",
+  "transformers_version": "4.31.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}

generation_config.json

@@ -0,0 +1,7 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "pad_token_id": 0,
+  "transformers_version": "4.31.0"
+}

gptq_model-4bit-128g.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2e9fa9e4f522ed3c107a8f7db83cfecc57623fb532211e1240553d575d5f1fea
+size 7259449576

quantize_config.json

@@ -0,0 +1,11 @@
+{
+  "bits": 4,
+  "group_size": 128,
+  "damp_percent": 0.01,
+  "desc_act": false,
+  "static_groups": false,
+  "sym": true,
+  "true_sequential": true,
+  "model_name_or_path": null,
+  "model_file_base_name": null
+}

special_tokens_map.json

@@ -0,0 +1,23 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9e556afd44213b6bd1be2b850ebbbd98f5481437a8021afaf58ee7fb1818d347
+size 499723

tokenizer_config.json

@@ -0,0 +1,34 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "bos_token": {
+    "__type": "AddedToken",
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "clean_up_tokenization_spaces": false,
+  "eos_token": {
+    "__type": "AddedToken",
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "legacy": true,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": null,
+  "sp_model_kwargs": {},
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": {
+    "__type": "AddedToken",
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}