File size: 1,075 Bytes
64bbc61 a33cb2e 64bbc61 91dbf0a dd64252 91dbf0a 64bbc61 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 |
## Training details
- Dataset used: Explanation style datasets from psmathur/WizardLM_Orca and Dahoas/cot_gsm8k
- Techniques: fp16 bit precision training + QLoRA + DeepSpeed
- Machine: V100 (16GB) * 2
## Inference
```python
from peft import PeftModel
from huggingface_hub import hf_hub_download
from transformers import LlamaTokenizer, LlamaForCausalLM
import json
model_name = "shahules786/open-llama-7B-orcastyle"
config = hf_hub_download(repo_id=model_name, filename="adapter_config.json", local_dir=".")
config = json.load(open("adapter_config.json"))
base_model = config["base_model_name_or_path"]
tokenizer = LlamaTokenizer.from_pretrained(model_name)
model = LlamaForCausalLM.from_pretrained(base_model)
model.resize_token_embeddings(len(self.tokenizer))
model = PeftModel.from_pretrained(model, model_name).eval()
tokenizer.padding_side = "left"
inputs = tokenizer("This is a sample run", return_tensors="pt")
model.generate(**inputs)
```
Checkout training and inference code [here](https://github.com/explodinggradients/Funtuner/tree/main/funtuner)
|