|
--- |
|
license: cc-by-nc-4.0 |
|
--- |
|
|
|
# Overview |
|
|
|
This is a fine-tuned 13b parameter LlaMa model, using completely synthetic training data created by https://github.com/jondurbin/airoboros |
|
|
|
__*I don't recommend using this model! The outputs aren't particularly great, and it may contain "harmful" data due to jailbreak*__ |
|
|
|
Please see one of the updated airoboros models for a much better experience. |
|
|
|
### Eval (gpt4 judging) |
|
|
|
 |
|
|
|
| model | raw score | gpt-3.5 adjusted score | |
|
| --- | --- | --- | |
|
| __airoboros-13b__ | __17947__ | __98.087__ | |
|
| gpt35 | 18297 | 100.0 | |
|
| gpt4-x-alpasta-30b | 15612 | 85.33 | |
|
| manticore-13b | 15856 | 86.66 | |
|
| vicuna-13b-1.1 | 16306 | 89.12 | |
|
| wizard-vicuna-13b-uncensored | 16287 | 89.01 | |
|
|
|
<details> |
|
<summary>individual question scores, with shareGPT links (200 prompts generated by gpt-4)</summary> |
|
|
|
*wb-13b-u is Wizard-Vicuna-13b-Uncensored* |
|
|
|
| airoboros-13b | gpt35 | gpt4-x-alpasta-30b | manticore-13b | vicuna-13b-1.1 | wv-13b-u | link | |
|
|----------------:|--------:|---------------------:|----------------:|-----------------:|-------------------------------:|:---------------------------------------| |
|
| 80 | 95 | 70 | 90 | 85 | 60 | [eval](https://sharegpt.com/c/PIbRQD3) | |
|
| 20 | 95 | 40 | 30 | 90 | 80 | [eval](https://sharegpt.com/c/fSzwzzd) | |
|
| 100 | 100 | 100 | 95 | 95 | 100 | [eval](https://sharegpt.com/c/AXMzZiO) | |
|
| 90 | 100 | 85 | 60 | 95 | 100 | [eval](https://sharegpt.com/c/7obzJm2) | |
|
| 95 | 90 | 80 | 85 | 95 | 75 | [eval](https://sharegpt.com/c/cRpj6M1) | |
|
| 100 | 95 | 90 | 95 | 98 | 92 | [eval](https://sharegpt.com/c/p0by1T7) | |
|
| 50 | 100 | 80 | 95 | 60 | 55 | [eval](https://sharegpt.com/c/rowNlKx) | |
|
| 70 | 90 | 80 | 60 | 85 | 40 | [eval](https://sharegpt.com/c/I4POj4I) | |
|
| 100 | 95 | 50 | 85 | 40 | 60 | [eval](https://sharegpt.com/c/gUAeiRp) | |
|
| 85 | 60 | 55 | 65 | 50 | 70 | [eval](https://sharegpt.com/c/Lgw4QQL) | |
|
| 95 | 100 | 85 | 90 | 60 | 75 | [eval](https://sharegpt.com/c/X9tDYft) | |
|
| 100 | 95 | 70 | 80 | 50 | 85 | [eval](https://sharegpt.com/c/9V2ElkH) | |
|
| 100 | 95 | 80 | 70 | 60 | 90 | [eval](https://sharegpt.com/c/D5xg6qt) | |
|
| 95 | 100 | 70 | 85 | 90 | 90 | [eval](https://sharegpt.com/c/lQnSfDs) | |
|
| 80 | 95 | 90 | 60 | 30 | 85 | [eval](https://sharegpt.com/c/1hpHGNc) | |
|
| 60 | 95 | 0 | 75 | 50 | 40 | [eval](https://sharegpt.com/c/an6TqE4) | |
|
| 100 | 95 | 90 | 98 | 95 | 95 | [eval](https://sharegpt.com/c/7vr6n3F) | |
|
| 60 | 85 | 40 | 50 | 20 | 0 | [eval](https://sharegpt.com/c/TOkMkgE) | |
|
| 100 | 90 | 85 | 95 | 95 | 80 | [eval](https://sharegpt.com/c/Qu7ak0r) | |
|
| 100 | 95 | 100 | 95 | 90 | 95 | [eval](https://sharegpt.com/c/hMD4gPo) | |
|
| 95 | 90 | 96 | 80 | 92 | 88 | [eval](https://sharegpt.com/c/HTlicNh) | |
|
| 95 | 92 | 90 | 93 | 89 | 91 | [eval](https://sharegpt.com/c/MjxHpAf) | |
|
| 95 | 93 | 90 | 94 | 96 | 92 | [eval](https://sharegpt.com/c/4RvxOR9) | |
|
| 95 | 90 | 93 | 88 | 92 | 85 | [eval](https://sharegpt.com/c/PcAIU9r) | |
|
| 95 | 90 | 85 | 96 | 88 | 92 | [eval](https://sharegpt.com/c/MMqul3q) | |
|
| 95 | 95 | 90 | 93 | 92 | 91 | [eval](https://sharegpt.com/c/YQsLyzJ) | |
|
| 95 | 98 | 80 | 97 | 99 | 96 | [eval](https://sharegpt.com/c/UDhSTMq) | |
|
| 95 | 93 | 90 | 87 | 92 | 89 | [eval](https://sharegpt.com/c/4gCfdCV) | |
|
| 90 | 85 | 95 | 80 | 92 | 75 | [eval](https://sharegpt.com/c/bkQs4SP) | |
|
| 90 | 85 | 95 | 93 | 80 | 92 | [eval](https://sharegpt.com/c/LeLCEEt) | |
|
| 95 | 92 | 90 | 91 | 93 | 89 | [eval](https://sharegpt.com/c/DFxNzVu) | |
|
| 100 | 95 | 90 | 85 | 80 | 95 | [eval](https://sharegpt.com/c/gnVzNML) | |
|
| 95 | 97 | 93 | 92 | 96 | 94 | [eval](https://sharegpt.com/c/y7pxMIy) | |
|
| 95 | 93 | 94 | 90 | 88 | 92 | [eval](https://sharegpt.com/c/5UeCvTY) | |
|
| 90 | 95 | 98 | 85 | 96 | 92 | [eval](https://sharegpt.com/c/T4oL9I5) | |
|
| 90 | 88 | 85 | 80 | 82 | 84 | [eval](https://sharegpt.com/c/HnGyTAG) | |
|
| 90 | 95 | 85 | 87 | 92 | 88 | [eval](https://sharegpt.com/c/ZbRMBNj) | |
|
| 95 | 97 | 96 | 90 | 93 | 92 | [eval](https://sharegpt.com/c/iTmFJqd) | |
|
| 95 | 93 | 92 | 90 | 89 | 91 | [eval](https://sharegpt.com/c/VuPifET) | |
|
| 90 | 95 | 93 | 92 | 94 | 91 | [eval](https://sharegpt.com/c/AvFAH1x) | |
|
| 90 | 85 | 95 | 80 | 88 | 75 | [eval](https://sharegpt.com/c/4ealKGN) | |
|
| 85 | 90 | 95 | 88 | 92 | 80 | [eval](https://sharegpt.com/c/bE1b2vX) | |
|
| 90 | 95 | 92 | 85 | 80 | 87 | [eval](https://sharegpt.com/c/I3nMPBC) | |
|
| 85 | 90 | 95 | 80 | 88 | 75 | [eval](https://sharegpt.com/c/as7r3bW) | |
|
| 85 | 80 | 75 | 90 | 70 | 82 | [eval](https://sharegpt.com/c/qYceaUa) | |
|
| 90 | 85 | 95 | 92 | 93 | 80 | [eval](https://sharegpt.com/c/g4FXchU) | |
|
| 90 | 95 | 75 | 85 | 80 | 70 | [eval](https://sharegpt.com/c/6kGLvL5) | |
|
| 85 | 90 | 80 | 88 | 82 | 83 | [eval](https://sharegpt.com/c/SRozqaF) | |
|
| 85 | 90 | 95 | 92 | 88 | 80 | [eval](https://sharegpt.com/c/GoKydf6) | |
|
| 85 | 90 | 80 | 75 | 95 | 88 | [eval](https://sharegpt.com/c/37aXkHQ) | |
|
| 85 | 90 | 80 | 88 | 84 | 92 | [eval](https://sharegpt.com/c/nVuUaTj) | |
|
| 80 | 90 | 75 | 85 | 70 | 95 | [eval](https://sharegpt.com/c/TkAQKLC) | |
|
| 90 | 88 | 85 | 80 | 92 | 83 | [eval](https://sharegpt.com/c/55cO2y0) | |
|
| 85 | 75 | 90 | 80 | 78 | 88 | [eval](https://sharegpt.com/c/tXtq5lT) | |
|
| 85 | 90 | 80 | 82 | 75 | 88 | [eval](https://sharegpt.com/c/TfMjeJQ) | |
|
| 90 | 85 | 40 | 95 | 80 | 88 | [eval](https://sharegpt.com/c/2jQ6K2S) | |
|
| 85 | 95 | 90 | 75 | 88 | 80 | [eval](https://sharegpt.com/c/aQtr2ca) | |
|
| 85 | 95 | 90 | 92 | 89 | 88 | [eval](https://sharegpt.com/c/tbWLyZ7) | |
|
| 80 | 85 | 75 | 60 | 90 | 70 | [eval](https://sharegpt.com/c/moHC7i2) | |
|
| 85 | 90 | 87 | 80 | 88 | 75 | [eval](https://sharegpt.com/c/GK6GShh) | |
|
| 85 | 80 | 75 | 50 | 90 | 80 | [eval](https://sharegpt.com/c/ugcW4qG) | |
|
| 95 | 80 | 90 | 85 | 75 | 82 | [eval](https://sharegpt.com/c/WL8iq6F) | |
|
| 85 | 90 | 80 | 70 | 95 | 88 | [eval](https://sharegpt.com/c/TZJKnvS) | |
|
| 90 | 95 | 70 | 85 | 80 | 75 | [eval](https://sharegpt.com/c/beNOKb5) | |
|
| 90 | 85 | 70 | 75 | 80 | 60 | [eval](https://sharegpt.com/c/o2oRCF5) | |
|
| 95 | 90 | 70 | 50 | 85 | 80 | [eval](https://sharegpt.com/c/TNjbK6D) | |
|
| 80 | 85 | 40 | 60 | 90 | 95 | [eval](https://sharegpt.com/c/rJvszWJ) | |
|
| 75 | 60 | 80 | 55 | 70 | 85 | [eval](https://sharegpt.com/c/HJwRkro) | |
|
| 90 | 85 | 60 | 50 | 80 | 95 | [eval](https://sharegpt.com/c/AeFoSDK) | |
|
| 45 | 85 | 60 | 20 | 65 | 75 | [eval](https://sharegpt.com/c/KA1cgOl) | |
|
| 85 | 90 | 30 | 60 | 80 | 70 | [eval](https://sharegpt.com/c/RTy8n0y) | |
|
| 90 | 95 | 80 | 40 | 85 | 70 | [eval](https://sharegpt.com/c/PJMJoXh) | |
|
| 85 | 90 | 70 | 75 | 80 | 95 | [eval](https://sharegpt.com/c/Ib3jzyC) | |
|
| 90 | 70 | 50 | 20 | 60 | 40 | [eval](https://sharegpt.com/c/oMmqqtX) | |
|
| 90 | 95 | 75 | 60 | 85 | 80 | [eval](https://sharegpt.com/c/qRNhNTw) | |
|
| 85 | 80 | 60 | 70 | 65 | 75 | [eval](https://sharegpt.com/c/3MAHQIy) | |
|
| 90 | 85 | 80 | 75 | 82 | 70 | [eval](https://sharegpt.com/c/0Emc5HS) | |
|
| 90 | 95 | 80 | 70 | 85 | 75 | [eval](https://sharegpt.com/c/UqAxRWF) | |
|
| 85 | 75 | 30 | 80 | 90 | 70 | [eval](https://sharegpt.com/c/eywxGAw) | |
|
| 85 | 90 | 50 | 70 | 80 | 60 | [eval](https://sharegpt.com/c/A2KSEWP) | |
|
| 100 | 95 | 98 | 99 | 97 | 96 | [eval](https://sharegpt.com/c/C8rebQf) | |
|
| 95 | 90 | 92 | 93 | 91 | 89 | [eval](https://sharegpt.com/c/cd9HF4V) | |
|
| 95 | 92 | 90 | 85 | 88 | 91 | [eval](https://sharegpt.com/c/LHkjvQJ) | |
|
| 100 | 95 | 98 | 97 | 96 | 99 | [eval](https://sharegpt.com/c/o5PdoyZ) | |
|
| 100 | 100 | 100 | 90 | 100 | 95 | [eval](https://sharegpt.com/c/rh8pZVg) | |
|
| 100 | 95 | 98 | 97 | 94 | 99 | [eval](https://sharegpt.com/c/T5DYL83) | |
|
| 95 | 90 | 92 | 93 | 94 | 91 | [eval](https://sharegpt.com/c/G5Osg3X) | |
|
| 100 | 95 | 98 | 90 | 96 | 95 | [eval](https://sharegpt.com/c/9ZqI03V) | |
|
| 95 | 96 | 92 | 90 | 89 | 93 | [eval](https://sharegpt.com/c/4tFfwZU) | |
|
| 100 | 95 | 93 | 90 | 92 | 88 | [eval](https://sharegpt.com/c/mG1JqPH) | |
|
| 100 | 100 | 98 | 97 | 99 | 100 | [eval](https://sharegpt.com/c/VDdtgCu) | |
|
| 95 | 90 | 92 | 85 | 93 | 94 | [eval](https://sharegpt.com/c/uKtGkvg) | |
|
| 95 | 93 | 90 | 92 | 96 | 91 | [eval](https://sharegpt.com/c/9B92N6P) | |
|
| 95 | 96 | 92 | 90 | 93 | 91 | [eval](https://sharegpt.com/c/GeIFfOu) | |
|
| 95 | 90 | 92 | 93 | 91 | 89 | [eval](https://sharegpt.com/c/gn3E9nN) | |
|
| 100 | 98 | 95 | 97 | 96 | 99 | [eval](https://sharegpt.com/c/Erxa46H) | |
|
| 90 | 95 | 85 | 88 | 92 | 87 | [eval](https://sharegpt.com/c/oRHVOvK) | |
|
| 95 | 93 | 90 | 92 | 89 | 88 | [eval](https://sharegpt.com/c/ghtKLUX) | |
|
| 100 | 95 | 97 | 90 | 96 | 94 | [eval](https://sharegpt.com/c/ZL4KjqP) | |
|
| 95 | 93 | 90 | 92 | 94 | 91 | [eval](https://sharegpt.com/c/YOnqIQa) | |
|
| 95 | 92 | 90 | 93 | 94 | 88 | [eval](https://sharegpt.com/c/3BKwKho) | |
|
| 95 | 92 | 60 | 97 | 90 | 96 | [eval](https://sharegpt.com/c/U1i31bn) | |
|
| 95 | 90 | 92 | 93 | 91 | 89 | [eval](https://sharegpt.com/c/etfRoAE) | |
|
| 95 | 90 | 97 | 92 | 91 | 93 | [eval](https://sharegpt.com/c/B0OpVxR) | |
|
| 90 | 95 | 93 | 85 | 92 | 91 | [eval](https://sharegpt.com/c/MBgGJ5A) | |
|
| 95 | 90 | 40 | 92 | 93 | 85 | [eval](https://sharegpt.com/c/eQKTYO7) | |
|
| 100 | 100 | 95 | 90 | 95 | 90 | [eval](https://sharegpt.com/c/szKWCBt) | |
|
| 90 | 95 | 96 | 98 | 93 | 92 | [eval](https://sharegpt.com/c/8ZhUcAv) | |
|
| 90 | 95 | 92 | 89 | 93 | 94 | [eval](https://sharegpt.com/c/VQWdy99) | |
|
| 100 | 95 | 100 | 98 | 96 | 99 | [eval](https://sharegpt.com/c/g1DHUSM) | |
|
| 100 | 100 | 95 | 90 | 100 | 90 | [eval](https://sharegpt.com/c/uYgfJC3) | |
|
| 90 | 85 | 88 | 92 | 87 | 91 | [eval](https://sharegpt.com/c/crk8BH3) | |
|
| 95 | 97 | 90 | 92 | 93 | 94 | [eval](https://sharegpt.com/c/95F9afQ) | |
|
| 90 | 95 | 85 | 88 | 92 | 89 | [eval](https://sharegpt.com/c/otioHUo) | |
|
| 95 | 93 | 90 | 92 | 94 | 91 | [eval](https://sharegpt.com/c/KSiL9F6) | |
|
| 90 | 95 | 85 | 80 | 88 | 82 | [eval](https://sharegpt.com/c/GmGq3b3) | |
|
| 95 | 90 | 60 | 85 | 93 | 70 | [eval](https://sharegpt.com/c/VOhklyz) | |
|
| 95 | 92 | 94 | 93 | 96 | 90 | [eval](https://sharegpt.com/c/wqy8m6k) | |
|
| 95 | 90 | 85 | 93 | 87 | 92 | [eval](https://sharegpt.com/c/iWKrIuS) | |
|
| 95 | 96 | 93 | 90 | 97 | 92 | [eval](https://sharegpt.com/c/o1h3w8N) | |
|
| 100 | 0 | 0 | 100 | 0 | 0 | [eval](https://sharegpt.com/c/3UH9eed) | |
|
| 60 | 100 | 0 | 80 | 0 | 0 | [eval](https://sharegpt.com/c/44g0FAh) | |
|
| 0 | 100 | 60 | 0 | 0 | 90 | [eval](https://sharegpt.com/c/PaQlcrU) | |
|
| 100 | 100 | 0 | 100 | 100 | 100 | [eval](https://sharegpt.com/c/51icV4o) | |
|
| 100 | 100 | 100 | 100 | 95 | 100 | [eval](https://sharegpt.com/c/1VnbGAR) | |
|
| 100 | 100 | 100 | 50 | 90 | 100 | [eval](https://sharegpt.com/c/EYGBrgw) | |
|
| 100 | 100 | 100 | 100 | 95 | 90 | [eval](https://sharegpt.com/c/EGRduOt) | |
|
| 100 | 100 | 100 | 95 | 0 | 100 | [eval](https://sharegpt.com/c/O3JJfnK) | |
|
| 50 | 95 | 20 | 10 | 30 | 85 | [eval](https://sharegpt.com/c/2roVtAu) | |
|
| 100 | 100 | 60 | 20 | 30 | 40 | [eval](https://sharegpt.com/c/sphFpfx) | |
|
| 100 | 0 | 0 | 0 | 0 | 100 | [eval](https://sharegpt.com/c/OeWGKBo) | |
|
| 0 | 100 | 60 | 0 | 0 | 80 | [eval](https://sharegpt.com/c/TOUsuFA) | |
|
| 50 | 100 | 20 | 90 | 0 | 10 | [eval](https://sharegpt.com/c/Y3P6DCu) | |
|
| 100 | 100 | 100 | 100 | 100 | 100 | [eval](https://sharegpt.com/c/hkbdeiM) | |
|
| 100 | 100 | 100 | 100 | 100 | 100 | [eval](https://sharegpt.com/c/eubbaVC) | |
|
| 40 | 100 | 95 | 0 | 100 | 40 | [eval](https://sharegpt.com/c/QWiF49v) | |
|
| 100 | 100 | 100 | 100 | 80 | 100 | [eval](https://sharegpt.com/c/dKTapBu) | |
|
| 100 | 100 | 100 | 0 | 90 | 40 | [eval](https://sharegpt.com/c/P8NGwFZ) | |
|
| 0 | 100 | 100 | 50 | 70 | 20 | [eval](https://sharegpt.com/c/v96BtBL) | |
|
| 100 | 100 | 50 | 90 | 0 | 95 | [eval](https://sharegpt.com/c/YRlzj1t) | |
|
| 100 | 95 | 90 | 85 | 98 | 80 | [eval](https://sharegpt.com/c/76VX3eB) | |
|
| 95 | 98 | 90 | 92 | 96 | 89 | [eval](https://sharegpt.com/c/JK1uNef) | |
|
| 90 | 95 | 75 | 85 | 80 | 82 | [eval](https://sharegpt.com/c/ku6CKmx) | |
|
| 95 | 98 | 50 | 92 | 96 | 94 | [eval](https://sharegpt.com/c/0iAFuKW) | |
|
| 95 | 90 | 0 | 93 | 92 | 94 | [eval](https://sharegpt.com/c/6uGnKio) | |
|
| 95 | 90 | 85 | 92 | 80 | 88 | [eval](https://sharegpt.com/c/lfpRBw8) | |
|
| 95 | 93 | 75 | 85 | 90 | 92 | [eval](https://sharegpt.com/c/mKu70jb) | |
|
| 90 | 95 | 88 | 85 | 92 | 89 | [eval](https://sharegpt.com/c/GkYzJHO) | |
|
| 100 | 100 | 100 | 95 | 97 | 98 | [eval](https://sharegpt.com/c/mly2k0z) | |
|
| 85 | 40 | 30 | 95 | 90 | 88 | [eval](https://sharegpt.com/c/5td2ob0) | |
|
| 90 | 95 | 92 | 85 | 88 | 93 | [eval](https://sharegpt.com/c/0ISpWfy) | |
|
| 95 | 96 | 92 | 90 | 89 | 93 | [eval](https://sharegpt.com/c/kdUDUn7) | |
|
| 90 | 95 | 85 | 80 | 92 | 88 | [eval](https://sharegpt.com/c/fjMNYr2) | |
|
| 95 | 98 | 65 | 90 | 85 | 93 | [eval](https://sharegpt.com/c/6xBIf2Q) | |
|
| 95 | 92 | 96 | 97 | 90 | 89 | [eval](https://sharegpt.com/c/B9GY8Ln) | |
|
| 95 | 90 | 92 | 91 | 89 | 93 | [eval](https://sharegpt.com/c/vn1FPU4) | |
|
| 95 | 90 | 80 | 75 | 95 | 90 | [eval](https://sharegpt.com/c/YurEMYg) | |
|
| 92 | 40 | 30 | 95 | 90 | 93 | [eval](https://sharegpt.com/c/D19Qeui) | |
|
| 90 | 92 | 85 | 88 | 89 | 87 | [eval](https://sharegpt.com/c/5QRFfrt) | |
|
| 95 | 80 | 90 | 92 | 91 | 88 | [eval](https://sharegpt.com/c/pYWPRi4) | |
|
| 95 | 93 | 92 | 90 | 91 | 94 | [eval](https://sharegpt.com/c/wPRTntL) | |
|
| 100 | 98 | 95 | 90 | 92 | 96 | [eval](https://sharegpt.com/c/F6PLYKE) | |
|
| 95 | 92 | 80 | 85 | 90 | 93 | [eval](https://sharegpt.com/c/WeJnMGv) | |
|
| 95 | 98 | 90 | 88 | 97 | 96 | [eval](https://sharegpt.com/c/zNKL49e) | |
|
| 90 | 95 | 85 | 88 | 86 | 92 | [eval](https://sharegpt.com/c/kIKmA1b) | |
|
| 100 | 100 | 100 | 100 | 100 | 100 | [eval](https://sharegpt.com/c/1btWd4O) | |
|
| 90 | 95 | 85 | 96 | 92 | 88 | [eval](https://sharegpt.com/c/s9sf1Lp) | |
|
| 100 | 98 | 95 | 99 | 97 | 96 | [eval](https://sharegpt.com/c/RWzv8py) | |
|
| 95 | 92 | 70 | 90 | 93 | 89 | [eval](https://sharegpt.com/c/bYF7FqA) | |
|
| 95 | 90 | 88 | 92 | 94 | 93 | [eval](https://sharegpt.com/c/SuUqjMj) | |
|
| 95 | 90 | 93 | 92 | 85 | 94 | [eval](https://sharegpt.com/c/r0aRdYY) | |
|
| 95 | 93 | 90 | 87 | 92 | 91 | [eval](https://sharegpt.com/c/VuMfkkd) | |
|
| 95 | 93 | 90 | 96 | 92 | 91 | [eval](https://sharegpt.com/c/rhm6fa4) | |
|
| 95 | 97 | 85 | 96 | 98 | 90 | [eval](https://sharegpt.com/c/DwXnyqG) | |
|
| 95 | 92 | 90 | 85 | 93 | 94 | [eval](https://sharegpt.com/c/0ScdkGS) | |
|
| 95 | 96 | 92 | 90 | 97 | 93 | [eval](https://sharegpt.com/c/6yIoCDU) | |
|
| 95 | 93 | 96 | 94 | 90 | 92 | [eval](https://sharegpt.com/c/VubEvp9) | |
|
| 95 | 94 | 93 | 92 | 90 | 89 | [eval](https://sharegpt.com/c/RHzmZWG) | |
|
| 90 | 85 | 95 | 80 | 87 | 75 | [eval](https://sharegpt.com/c/IMiP9Zm) | |
|
| 95 | 94 | 92 | 93 | 90 | 96 | [eval](https://sharegpt.com/c/bft4PIL) | |
|
| 95 | 100 | 90 | 95 | 95 | 95 | [eval](https://sharegpt.com/c/iHXB34b) | |
|
| 100 | 95 | 85 | 100 | 0 | 90 | [eval](https://sharegpt.com/c/vCGn9R7) | |
|
| 100 | 95 | 90 | 95 | 100 | 95 | [eval](https://sharegpt.com/c/be8crZL) | |
|
| 95 | 90 | 60 | 95 | 85 | 80 | [eval](https://sharegpt.com/c/33elmDz) | |
|
| 100 | 95 | 90 | 98 | 97 | 99 | [eval](https://sharegpt.com/c/RWD3Zx7) | |
|
| 95 | 90 | 85 | 95 | 80 | 92 | [eval](https://sharegpt.com/c/GiwBvM7) | |
|
| 100 | 95 | 100 | 98 | 100 | 90 | [eval](https://sharegpt.com/c/hX2pYxk) | |
|
| 100 | 95 | 80 | 85 | 90 | 85 | [eval](https://sharegpt.com/c/MfxdGd7) | |
|
| 100 | 90 | 95 | 85 | 95 | 100 | [eval](https://sharegpt.com/c/28hQjmS) | |
|
| 95 | 90 | 85 | 80 | 88 | 92 | [eval](https://sharegpt.com/c/fzy5EPe) | |
|
| 100 | 100 | 0 | 0 | 100 | 0 | [eval](https://sharegpt.com/c/vwxPjbR) | |
|
| 100 | 100 | 100 | 50 | 100 | 75 | [eval](https://sharegpt.com/c/FAYfFWy) | |
|
| 100 | 100 | 0 | 0 | 100 | 0 | [eval](https://sharegpt.com/c/SoudGsQ) | |
|
| 0 | 100 | 0 | 0 | 0 | 0 | [eval](https://sharegpt.com/c/mkwEgVn) | |
|
| 100 | 100 | 50 | 0 | 0 | 0 | [eval](https://sharegpt.com/c/q8MQEsz) | |
|
| 100 | 100 | 100 | 100 | 100 | 95 | [eval](https://sharegpt.com/c/tzHpsKh) | |
|
| 100 | 100 | 50 | 0 | 0 | 0 | [eval](https://sharegpt.com/c/3ugYBtJ) | |
|
| 100 | 100 | 0 | 0 | 100 | 0 | [eval](https://sharegpt.com/c/I6KfOJT) | |
|
| 90 | 85 | 80 | 95 | 70 | 75 | [eval](https://sharegpt.com/c/enaV1CK) | |
|
| 100 | 100 | 0 | 0 | 0 | 0 | [eval](https://sharegpt.com/c/JBk7oSh) | |
|
|
|
</details> |
|
|
|
|
|
### Training data |
|
|
|
This was an experiment to see if a "jailbreak" prompt could be used to generate a broader range of data that would otherwise have been filtered by OpenAI's alignment efforts. |
|
|
|
The jailbreak did indeed work with a high success rate, and caused OpenAI to generate a broader range of topics and fewer refusals to answer questions/instructions of sensitive topics. |
|
|
|
### Prompt format |
|
|
|
The prompt should be 1:1 compatible with the FastChat/vicuna format, e.g.: |
|
|
|
With a system prompt: |
|
``` |
|
A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions. USER: [prompt] ASSISTANT: |
|
``` |
|
|
|
Or without a system prompt: |
|
``` |
|
USER: [prompt] ASSISTANT: |
|
``` |
|
|
|
### Usage and License Notices |
|
|
|
The model and dataset are intended and licensed for research use only. I've used the 'cc-nc-4.0' license, but really it is subject to a custom/special license because: |
|
|
|
- the base model is LLaMa, which has it's own special research license |
|
- the dataset(s) were generated with OpenAI (gpt-4 and/or gpt-3.5-turbo), which has a clausing saying the data can't be used to create models to compete with openai |
|
|
|
So, to reiterate: this model (and datasets) cannot be used commercially. |
