|
--- |
|
library_name: transformers |
|
tags: |
|
- trl |
|
- sft |
|
license: mit |
|
datasets: |
|
- gbharti/finance-alpaca |
|
base_model: |
|
- mistralai/Mistral-7B-v0.1 |
|
language: |
|
- en |
|
--- |
|
### Model Description |
|
|
|
This model is based on the Mistral 7B architecture, PEFT fine-tuned on financial data. It is designed to handle various finance-related NLP tasks such as financial text analysis, sentiment detection, market trend analysis, and more. This model leverages the powerful transformer architecture of Mistral with specialized fine-tuning for financial applications. |
|
|
|
- **Developed by:** Cole McIntosh |
|
- **Model type:** Transformer-based large language model (LLM) |
|
- **Language(s) (NLP):** English |
|
- **Finetuned from model:** Mistral 7B |
|
|
|
## Uses |
|
|
|
### Direct Use |
|
|
|
The Mistral 7B Finance Fine-tuned model is designed to assist users with finance-related natural language processing tasks such as: |
|
|
|
- Financial report analysis |
|
- Sentiment analysis of financial news |
|
- Forecasting market trends based on textual data |
|
- Analyzing earnings call transcripts |
|
- Extracting structured information from unstructured financial text |
|
|
|
### Downstream Use |
|
|
|
This model can be fine-tuned further for more specific tasks such as: |
|
|
|
- Portfolio analysis based on sentiment scores |
|
- Predictive analysis for stock market movements |
|
- Automated financial report generation |
|
|
|
### Out-of-Scope Use |
|
|
|
This model should not be used for tasks unrelated to finance or those requiring a high level of factual accuracy in non-financial domains. It is not suitable for: |
|
|
|
- Medical or legal document analysis |
|
- General conversational chatbots (as the model may provide misleading financial interpretations) |
|
- Decision-making without human oversight, especially in high-stakes financial operations |
|
|
|
### Recommendations |
|
|
|
- Carefully review model outputs, especially in critical financial decisions. |
|
- Use up-to-date fine-tuning datasets to ensure relevance. |
|
- Cross-validate the model's predictions or insights with alternative data sources or human expertise. |
|
|
|
## How to Get Started with the Model |
|
|
|
You can use the Hugging Face `transformers` library to load and use this model. Here’s a basic example: |
|
|
|
```python |
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
|
|
# Load model and tokenizer |
|
tokenizer = AutoTokenizer.from_pretrained("colesmcintosh/mistral_7b_finance_finetuned") |
|
model = AutoModelForCausalLM.from_pretrained("colesmcintosh/mistral_7b_finance_finetuned") |
|
|
|
# Example usage |
|
inputs = tokenizer("Analyze the financial outlook for Q3 2024.", return_tensors="pt") |
|
outputs = model.generate(**inputs) |
|
print(tokenizer.decode(outputs[0])) |
|
``` |
|
|
|
## Training Details |
|
|
|
### Training Procedure |
|
|
|
The fine-tuning process used PEFT to accelerate training on GPUs. |
|
|
|
#### Summary |
|
|
|
The model performs well in finance-specific tasks like sentiment analysis and entity recognition. It demonstrates strong generalization across different sectors but shows slight performance drops when analyzing non-English financial texts. |
|
|
|
### Model Architecture and Objective |
|
|
|
The model is based on the Mistral 7B architecture, a highly optimized transformer-based model. Its primary objective is text generation and understanding, with a focus on financial texts. |
|
|
|
### Compute Infrastructure |
|
|
|
#### Hardware |
|
|
|
The model was fine-tuned using: |
|
|
|
- 1 NVIDIA A100 GPU (40 GB) |
|
|
|
#### Software |
|
|
|
- Hugging Face `transformers` library |
|
- PEFT finetuning |
