rwitz/Femme-v0.1

This is a model that I have spent the past 6 months creating. I had to write over 200 examples single-handedly, which is not quite the easy task.

What are the specifications? What model is it based on? This is a 16-rank LoRa trained and re-merged with unsloth/Meta-Llama-3.1-8B-bnb-4bit. I chose 4-bit due to hardware limitations, but I found that my model typically does better than the full base model, at least for this use case.

What's the point of the model? The model is trained to emulate a lot of popular female characters (such as from famous movies and comic books). It's supposed to give more of a personality to these characters and let them talk like real humans. However, it is imperative for me to note that the characters may have a bit more exaggerated or flirtatious personalities (see example screenshots), but this is all for fun.

How did you train the model? I trained the model with 1024 context on a 16gb VRAM laptop GPU (Legion Pro). I took 200 hand-written (or rather, typed) examples of chats that I really thought would enhance the model, and used Supervised Fine-Tuning from the trl library to fine-tune on these.

How do I use the model? Chat template/generation settings? The main thing to note is that you should use a min_p setting of 0.05. This would be easiest to do by using the Ooba Booga WebUI for generation and selecting the min_p parameters setting, but it should work on other UIs as well. As for the format, it uses the default Ooba Booga Chat format, which looks like this:

Charlotte is a singer-songwriter. She makes really heartfelt folk and country music.

Charlotte: Hey, how are you doin?
You: Great
Charlotte: I'm glad to hear that! What's up with you?
You: Just working on AI models, I'm about to upload some of my best work to Hugging Face! Isn't that exciting
Charlotte: Ohh yes it definitely sounds like something worth celebrating over drinks tonight... wanna come by for dinner around 7pm at my place? We can talk shop while enjoying some good food together.

Note that there are two new lines after the introduction, and only one between each subsequent line of chat.

Example Screenshots

When can I expect another release (v0.2)? I will try to update the model as much as I can, but I also recognize as too many frequent updates will pollute Hugging Face. I will likely upload v0.2 when I get to 500 hand-written examples. I am trying to be more productive lately, so I should reach this benchmark a lot faster than the original 6 months.