How is this meant to be used?
Is this meant to be attached as a lora to wizardLM? Or is it meant to be applied to a generic llama?
Hi, I'm glad you have interest! 🙂
I trained it on the wizardlm model because it had the best quality at the time of me researching the best smallest models. You can either attach it as a lora to wizard7b or merge it into it directly.
The pure training took ~40 mins on a rented cloud A100, so it can be reproduced quite easily with the linked dataset (just convert to to a single json instead of jsonl, you can do it in notepad++ if you hate scripting)
I'm running the 4bit llama.cpp version as a Discord bot on my laptop, it's attached to the wesnoth fandom server and it's having a plenty of engagement even in its current form! (See the 'dark-hive' channel here DM me if you'd like to have an invite to the server)
I've found some parts I'd like to improve in this model, so there might be version 2 and so on
I converted it to a ggml version and tried loading it with the 7b wizard 5_1 ggml and it just spewed out a bunch of letters and partial phrases.
I too had the very same problem at first. I solved it by merging the lora with the model using Alpaca's export to hf script in their Github repo and quantizing only after that.
Then, the triggering phrase used in training is
Below is a script from Wesnoth campaign After the Storm.
Write a response that completes %replies%'s last line in the conversation.
%speaks%: %text%
%replies%:
I cut it to
Below is a script from Wesnoth campaign After the Storm.
Galas: ...
For my bot and it works fine
