Commit
•
efe61d9
1
Parent(s):
fc5dda1
Update README.md
Browse files
README.md
CHANGED
|
@@ -6,6 +6,15 @@ tags:
|
|
| 6 |
- not-for-all-audiences
|
| 7 |
---
|
| 8 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 9 |
# Moistral 11B v2 💦💦
|
| 10 |
|
| 11 |
*The moistest AI just got moistier!*
|
|
|
|
| 6 |
- not-for-all-audiences
|
| 7 |
---
|
| 8 |
|
| 9 |
+
|
| 10 |
+
## Attention:
|
| 11 |
+
|
| 12 |
+
After conducting some quantitative testing, it turns out the model does have issues (it scored unusually high in perplexity).
|
| 13 |
+
|
| 14 |
+
I recommend using the 25% Dried version for v2: https://huggingface.co/TheDrummer/Moistral-11B-v2-Dried-GGUF/blob/main/Moistral-11B-v2-25PCT-Q4.gguf
|
| 15 |
+
|
| 16 |
+
I have a feeling that the huge dip in training loss was the point where it broke. I'll recover the checkpoints for epoch 1 & epoch 2 and see if I can make a good v2.1 out of them.
|
| 17 |
+
|
| 18 |
# Moistral 11B v2 💦💦
|
| 19 |
|
| 20 |
*The moistest AI just got moistier!*
|