LoRA Info:
Please note that this is a highly experimental LoRA model. It may do some good stuff, it might do some undesirable stuff. Training is basically done now. Feel free to try it!~
Important Note: While this is trained on a cleaned ShareGPT dataset like Vicuna used, this was trained in the Alpaca format, so prompting should be something like:
### Instruction:
<prompt> (without the <>)
### Response:
Current upload: Fully trained adapter model (3 epochs).
Secondary upload: checkpoint of epoch 2.97 (of 3)
Thanks to MetaIX for initial seemingly successful testing of the first uploaded checkpoint (epoch 0.8) as well as epoch 1.
Benchmarks
wikitext2: 4.372413635253906
ptb-new: 24.69171714782715
c4-new: 6.469308853149414
Results generated with GPTQ evals (not quantized) thanks to Neko-Institute-of-Science