File size: 2,189 Bytes
43aa08a 9874c41 43aa08a b4dd518 be3b1d7 43aa08a 9874c41 871d983 43aa08a 9874c41 43aa08a 9874c41 43aa08a 9874c41 43aa08a |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 |
---
license: other
library_name: peft
tags:
- trl
- sft
- generated_from_trainer
base_model: meta-llama/Meta-Llama-3-8B-Instruct
model-index:
- name: llama3-8b-schopenhauer
results: []
language:
- en
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# llama3-8b-schopenhauer
This model is a fine-tuned version of [meta-llama/Meta-Llama-3-8B-Instruct](https://huggingface.co./meta-llama/Meta-Llama-3-8B-Instruct) on a synthetic dataset of argumentative conversations.
The model has been built by [Raphaaal](https://github.com/Raphaaal), [vdeva](https://github.com/vdeva), [margotcosson](https://github.com/margotcosson) and [basileplus](https://github.com/basileplus)
## Model description
The model as been trained to be an argumentative expert, following deterministic rethoric guidelines depicted by Schopenhauer in The Art of Being Right.
The model aims at showing how persuasive a model can be if we simply introduce some simple deterministic argumentative guidelines.
## Training and evaluation data
The model has been trained using LoRa on a small synthetic dataset which quality can be improved both in size and quality. The model has shown great performance in responding with short percuting answers to argumentative conversations. No argumentative metric has been implemented, interesting arguments evaluation benchmark can be found in [Cabrio, E., & Villata, S. (Year). Towards a Benchmark of Natural Language Arguments. INRIA Sophia Antipolis, France.](https://arxiv.org/pdf/1405.0941v1)
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3.0
### Framework versions
- PEFT 0.10.0
- Transformers 4.40.1
- Pytorch 2.3.0+cu121
- Datasets 2.19.0
- Tokenizers 0.19.1 |