README.md

---
license: apache-2.0
inference: false
language: ja
---

# japanese-large-lm-1.7b-instruction-sft-4bit-128g-actorder_False

This repository provides a 1.7B parameters Japanese language **quantized** model, fine-tuned and trained by [LINE Corporation](https://linecorp.com/ja/).

## For Japanese

詳細な説明や実験に関しては「[【インターンレポート】量子化による大規模言語モデル軽量化の効果測定](https://engineering.linecorp.com/ja/blog/quantization-lightweighting-llms)」をご覧ください。

## How to use

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline

tokenizer = AutoTokenizer.from_pretrained("line-corporation/japanese-large-lm-1.7b-instruction-sft", use_fast=False)
model = AutoModelForCausalLM.from_pretrained("line-corporation/japanese-large-lm-1.7b-instruction-sft-4bit-128g-actorder_False")
 
generator = pipeline("text-generation", model=model, tokenizer=tokenizer, device=0)
 
input_text = """四国の県名を全て列挙してください。"""
text = generator(
    f"ユーザー: {input_text}\nシステム: ",
    max_length = 256,
    do_sample = True,
    temperature = 0.7,
    top_p = 0.9,
    top_k = 0,
    repetition_penalty = 1.1,
    num_beams = 1,
    pad_token_id = tokenizer.pad_token_id,
    num_return_sequences = 1,
)
print(text) # [{'generated_text': 'ユーザー: 四国の県名を全て列挙してください。\nシステム:  高知県、徳島県、香川県、愛媛県'}]
```

## Tokenization

We use a sentencepiece tokenizer with a unigram language model and byte-fallback.
We **do not** apply pre-tokenization with Japanese tokenizer.
Thus, a user may directly feed raw sentences into the tokenizer.

## License
[Apache License, Version 2.0](https://www.apache.org/licenses/LICENSE-2.0)