Model Details
Model Description
This model is a fine-tuned version of the Qwen/Qwen2.5-3B-Instruct base model, optimized using QLoRA (Quantized Low-Rank Adaptation) on the Wealth Alpaca Dataset. It is designed to answer financial questions by combining domain-specific knowledge with the powerful capabilities of Qwen 2.5.
- Developed by: Ojaswa Yadav
- Model type: Conversational AI
- Language(s) (NLP): English (NLP)
- License: Apache 2.0
- Finetuned from model [optional]: Qwen/Qwen2.5-3B-Instruct
Direct Use
The model can be directly used for:
Financial question answering Analyzing financial reports Conversational AI for finance-related customer support
Downstream Use [optional]
Can be integrated into other systems for:
Financial sentiment analysis Advanced financial data retrieval pipelines
Out-of-Scope Use
The model is not intended for:
General-purpose chat Non-financial domains (accuracy not guaranteed)
Bias, Risks, and Limitations
Bias: The training dataset may introduce biases from Wealth Alpaca data. Use caution for sensitive or high-stakes decisions. Risks: Not suitable for real-time financial trading or critical decision-making without expert validation. Limitations: Focused on English financial data and may not generalize to other languages or domains.
Recommendations
Use the model with a RAG for best results
How to Get Started with the Model
from transformers import AutoTokenizer, AutoModelForCausalLM
model_name = "Ojaswa/QLoRA-Finetuned-Qwen-2.5-on-Wealth-Alpaca-Dataset" tokenizer = AutoTokenizer.from_pretrained(model_name) model = AutoModelForCausalLM.from_pretrained(model_name)
Example usage
inputs = tokenizer("Explain Stock Market to me?", return_tensors="pt") outputs = model.generate(**inputs) print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Training Details
Training Data
Dataset: Wealth Alpaca Dataset, consisting of preformatted financial Q&A pairs. Preprocessing: Combined input instructions, financial context, and output answers into tokenized prompts.
Training Procedure
Training Procedure Fine-tuning Method: QLoRA with 4-bit quantization. Targeted Layers: q_proj and v_proj of the attention mechanism. Dropout: 0.1 Optimizer: AdamW with learning rate 2e-5. Hardware: Trained on consumer-grade GPUs (NVIDIA L4).
Training Hyperparameters
Training Hyperparameters Training Regime: Mixed precision (FP16) Epochs: 3 Batch Size: 32
Speeds, Sizes, Times [optional]
Training Time: Approximately 24 hours
- PEFT 0.13.2
- Downloads last month
- 87