metadata
_name_or_path: CustomGPT2ConversationalModel
model_type: gpt2
architectures:
- GPT2LMHeadModel
tokenizer_config:
bos_token_id: 50256
eos_token_id: 50256
n_positions: 2048
padding_side: left
truncation_side: right
config:
activation_function: gelu_new
attn_pdrop: 0.1
embd_pdrop: 0.1
initializer_range: 0.02
layer_norm_epsilon: 0.00001
n_ctx: 2048
n_embd: 2048
n_head: 16
n_layer: 36
n_positions: 2048
resid_pdrop: 0.1
vocab_size: 50257
scale_attn_weights: true
gradient_checkpointing: true
use_cache: true
task_specific_params:
conversational:
max_length: 1024
min_length: 20
temperature: 0.7
top_k: 40
top_p: 0.95
num_beams: 5
no_repeat_ngram_size: 3
early_stopping: true
length_penalty: 2
do_sample: true
frequency_penalty: 0.5
presence_penalty: 0.5
use_cache: true
torch_dtype: float32
transformers_version: 4.37.2
language: en
license: apache-2.0
metrics:
- perplexity
- accuracy
- bleu
tags:
- conversational
- state-of-the-art
🎉 Introducing CustomGPT2Conversational 🤖
🚀 Dive into the world of limitless conversations with our fine-tuned model!
🎨 Unique Features:
- 💬 Seamlessly generates engaging and contextually relevant responses.
- 🧠 Fine-tuned for conversational tasks, ensuring natural and fluid interactions.
- 🌟 Employs cutting-edge technologies to deliver state-of-the-art performance.
🔧 Technical Specifications:
- 🛠 Architecture: GPT2LMHeadModel
- 📏 Parameters: 24 layers, 2048 hidden size, 16 attention heads
- 🧮 Dropout: Attention dropout of 0.1, embedding dropout of 0.1, and residual dropout of 0.1
- 🎚 Activation Function: GELU (Gaussian Error Linear Unit)
- 📊 Metrics: Achieves outstanding perplexity and accuracy scores
🛠 Configuration Details:
- 📝 Task-Specific Parameters: Early stopping, length penalty, beam search, and more to enhance conversational quality.
- 📈 Dynamic Adaptation: Adaptable to changing environmental conditions for optimal performance.
- 📚 Language Support: Primarily English with potential for multilingual applications.
🌟 Pipeline Tag: Conversational 🗣️
📚 Dataset: Utilizes data from the LDJnr/Capybara dataset 🦙
🔗 Library and Version: Transformers v4.37.2 📚
🔑 License: Apache-2.0 📜
Get ready to experience the next level of conversational AI with CustomGPT2Conversational! 🚀🤖