abhi9ab/DeepSeek-R1-Distill-Qwen-1.5B-finance-v1

Model Card

The goal of this model is to enhance the base model's performance on financial tasks by fine-tuning it on a specialized financial dataset. Using LoRA, this model has been optimized for low-rank adaptation, allowing efficient fine-tuning with fewer resources.

---

Model Details

• Base Model: DeepSeek-R1-Distill-Qwen-1.5B
• Model Type: Language Model (Distilled)
• Fine-Tuning Technique: LoRA (Low-Rank Adaptation)
• Fine-Tuned Model: DeepSeek-R1-Distill-Qwen-1.5B-Finance-v1
• Dataset: Josephgflowers/Finance-Instruct-500k (reduced to 5k JSONL entries)
• Platform: Free-tier Google Colab
• Library: Hugging Face Transformers

This model is a fine-tuned version of the DeepSeek-R1-Distill-Qwen-1.5B model, utilizing LoRA for efficient parameter adaptation. It has been specifically tuned on a reduced version (5k) of the Josephgflowers/Finance-Instruct-500k dataset to enhance performance in finance-related tasks.

---

Intended Use

The model is intended for tasks related to financial question answering, generation, and instructions that require domain-specific knowledge in finance. It can also be used in other natural language understanding and generation tasks that benefit from fine-tuning on a finance-specific dataset.

---

Dataset

The model was fine-tuned on a subset of the Finance-Instruct-500k dataset from Hugging Face, specifically reduced to 5,000 JSONL entries for the fine-tuning process. This dataset contains financial questions and answers, providing a rich set of examples for training the model.

---

Training Data

• Dataset Name: Josephgflowers/Finance-Instruct-500k
• Data Size: 5k samples (subset from original dataset)
• Domain: Finance
• Task: Instruction-based fine-tuning for financial information retrieval and generation.

---

Notes

• This fine-tuning was performed on the free-tier of Google Colab, so training time and available resources are limited.
• Ensure that your runtime in Colab is set to a GPU environment to speed up the training process.
• The reduced 5k dataset is a smaller sample for experimentation. You can scale this up depending on your needs and available resources.

---

Performance

The model performs well in financial instruction tasks, delivering accurate responses based on the reduced dataset. Performance can be further evaluated through specific finance-related benchmarks.

---

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("abhi9ab/DeepSeek-R1-Distill-Qwen-1.5B-finance-v1")
tokenizer = AutoTokenizer.from_pretrained("abhi9ab/DeepSeek-R1-Distill-Qwen-1.5B-finance-v1")

inputs = tokenizer("Example finance-related query", return_tensors="pt")
outputs = model.generate(inputs['input_ids'])

---

Acknowledgement

• Josephgflowers for the dataset.
• Hugging Face Transformers library for model implementation and LoRA-based fine-tuning.

---