Create README.md

README.md

@@ -0,0 +1,51 @@
+---
+license: apache-2.0
+---
+# FINGU-AI/Q-Small-3B
+
+## Overview
+`FINGU-AI/Q-Small-3B` is a powerful causal language model designed for a variety of natural language processing (NLP) tasks, including machine translation, text generation, and chat-based applications. This model is particularly useful for translating between languages, as well as supporting other custom NLP tasks through flexible input.
+
+## Example Usage
+
+### Installation
+Make sure to install the required packages:
+
+```bash
+pip install torch transformers
+```
+### Loading the Model
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+import torch
+
+# Model and Tokenizer
+model_id = 'FINGU-AI/Q-Small-3B'
+model = AutoModelForCausalLM.from_pretrained(model_id, attn_implementation="sdpa", torch_dtype=torch.bfloat16)
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model.to('cuda')
+
+# Input Messages for Translation
+messages = [
+    {"role": "system", "content": "you are helpfull assistant."},
+    {"role": "user", "content": """what is large language model?"""},
+]
+
+# Tokenize and Generate Response
+input_ids = tokenizer.apply_chat_template(
+    messages,
+    add_generation_prompt=True,
+    return_tensors="pt"
+).to('cuda')
+
+outputs = model.generate(
+    input_ids,
+    max_new_tokens=500,
+    do_sample=True,
+)
+
+# Decode and Print the Translation
+response = outputs[0][input_ids.shape[-1]:]
+print(tokenizer.decode(response, skip_special_tokens=True))
+```