FINGU-AI/Qwen2.5-32B-Lora-HQ-e-635
Overview
FINGU-AI/Qwen2.5-32B-Lora-HQ-e-635
is a powerful causal language model designed for a variety of natural language processing (NLP) tasks, including machine translation, text generation, and chat-based applications. This model is particularly useful for translating between Korean and Uzbek, as well as supporting other custom NLP tasks through flexible input.
Model Details
- Model ID:
FINGU-AI/Qwen2.5-32B-Lora-HQ-e-635
- Architecture: Causal Language Model (LM)
- Parameters: 32 billion
- Precision: Torch BF16 for efficient GPU memory usage
- Attention: SDPA (Scaled Dot-Product Attention)
- Primary Use Case: Translation (e.g., Korean to Uzbek), text generation, and dialogue systems.
Example Usage
Installation
Make sure to install the required packages:
pip install torch transformers
Loading the Model
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
# Model and Tokenizer
model_id = 'FINGU-AI/Qwen2.5-32B-Lora-HQ-e-635'
model = AutoModelForCausalLM.from_pretrained(model_id, attn_implementation="sdpa", torch_dtype=torch.bfloat16)
tokenizer = AutoTokenizer.from_pretrained(model_id)
model.to('cuda')
# Input Messages for Translation
messages = [
{"role": "system", "content": "translate korean to Uzbek"},
{"role": "user", "content": """์๋ก์ด ์ํ ๊ณ์ข๋ฅผ ๊ฐ์คํ๋ ์ ์ฐจ๋ ๋ค์๊ณผ ๊ฐ์ต๋๋ค:
1. ๊ณ์ข ๊ฐ์ค ๋ชฉ์ ๊ณผ ์ ๋ถ ํ์ธ์ ์ํ ์๋ฅ ์ ์ถ
2. ์๋ฅ ๊ฒํ ๊ณผ์ ์ ๊ฑฐ์น๋ ๊ฒ
3. ๊ณ ๊ฐ๋์ ์ ์ ํ์ธ ์ ์ฐจ๋ฅผ ์งํํ๋ ๊ฒ
4. ๋ชจ๋ ์ ์ฐจ๊ฐ ์๋ฃ๋๋ฉด ๊ณ์ข ๊ฐ์ค์ด ๊ฐ๋ฅํฉ๋๋ค.
๊ณ์ข ๊ฐ์ค์ ์ํ์๋ ๊ฒฝ์ฐ, ์ ๋ถ์ฆ๊ณผ ํจ๊ป ๋ฐฉ๋ฌธํด ์ฃผ์๋ฉด ๋ฉ๋๋ค.
"""},
]
# Tokenize and Generate Response
input_ids = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
return_tensors="pt"
).to('cuda')
outputs = model.generate(
input_ids,
max_new_tokens=500,
do_sample=True,
)
# Decode and Print the Translation
response = outputs[0][input_ids.shape[-1]:]
print(tokenizer.decode(response, skip_special_tokens=True))
- Downloads last month
- 4