---
license: other
license_name: tongyi-qianwen-license
license_link: LICENSE
language:
- en
- ja
library_name: transformers
pipeline_tag: text-generation
---

# nakomata-14b-pfn-qfin

## Model Description
nekomata-14b-pfn-qfin is an fine-tuned model based on [rinna/nekomata-14b](https://huggingface.co./rinna/nekomata-14b/tree/main).
This is the base model, which is good at generating continuous sentences.
nekomata-14b-pfn-qfin is fine-tuned on 370M tokens from multiple special datasets generated by Preferred Networks, which is clear to use for commercial usage.
The fine-tuned were carried out at a 2048 context length.
This model is released under [Tongyi Qianwen LICENSE AGREEMENT](https://github.com/QwenLM/Qwen/blob/e8e15962d897714944773cca57fa2e460a3655e8/Tongyi%20Qianwen%20LICENSE%20AGREEMENT).

The research article will also be released later.

# Benchmarking
The benchmark score is obtained using [Japanese Language Model Financial Evaluation Harness](https://github.com/pfnet-research/japanese-lm-fin-harness)
For the benchmark, 0-shot and default prompts are used.
```
|      Task      |Metric|  nekomaba-14b   |       Ours      |
|----------------|------|------|---|------|------|---|------|
|chabsa          |f1    |0.7381|   |      |0.7428|   |      |
|cma_basics      |acc   |0.4737|±  |0.0821|0.5263|±  |0.0821|
|cpa_audit       |acc   |0.1608|±  |0.0184|0.1633|±  |0.0186|
|fp2             |acc   |0.3389|±  |0.0217|0.3642|±  |0.0221|
|security_sales_1|acc   |0.4561|±  |0.0666|0.5614|±  |0.0663|
|----------------|------|------|---|------|------|---|------|
|OVER ALL        |      |0.4335           |0.4716           |
```
## Usage
Install the required libraries as follows:
```sh
>>> python -m pip install numpy sentencepiece torch transformers accelerate transformers_stream_generator
```

Execute the following python code:
```python
tokenizer = AutoTokenizer.from_pretrained("pfnet/nakomata-14b-pfn-qfin", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("pfnet/nakomata-14b-pfn-qfin", device_map="auto", trust_remote_code=True)
text = "日本銀行は"
input_ids = tokenizer(text, return_tensors="pt").input_ids
with torch.no_grad():
  generated_tokens = model.generate(
      inputs=input_ids,
      max_new_tokens=32,
      do_sample=True,
      top_k=50,
      top_p=0.95,
      temperature=1.0,
  )[0]
generated_text = tokenizer.decode(generated_tokens)
print(generated_text)
# 日本銀行は、平成27年10月に、デフレからの脱却をより確実なものとするため、「長短金利操作付き量的・質的金融緩和」を導入しました。...
```

## Model Details
- Model size: 14B
- Fine-tuned tokens: 370M tokens (Japanese: 300M tokens, English: 13M tokens, Digits: 14M tokens)
- Context length: 2048
- Developed by: Preferred Networks, Inc
- Model type: Causal decoder-only
- Language(s): Japanese and English
- License: [Tongyi Qianwen LICENSE AGREEMENT](https://github.com/QwenLM/Qwen/blob/e8e15962d897714944773cca57fa2e460a3655e8/Tongyi%20Qianwen%20LICENSE%20AGREEMENT)

## Bias, Risks, and Limitations
nakomata-14b-pfn-qfin is a new technology that carries risks with use.
Testing conducted to date has been in English and Japanese, and has not covered, nor could it cover all scenarios.
For these reasons, as with all LLMs, nakomata-14b-pfn-qfin’s potential outputs cannot be predicted in advance, and the model may in some instances produce inaccurate, biased or other objectionable responses to user prompts.
This model is not designed for legal, tax, investment, financial, or other advice.
Therefore, before deploying any applications of nakomata-14b-pfn-qfin, developers should perform safety testing and tuning tailored to their specific applications of the model.

## How to cite
TBD

## Authors
Preferred Networks, Inc.
 - Masanori Hirano
 - Kentaro Imajo

# License
[Tongyi Qianwen LICENSE AGREEMENT](https://github.com/QwenLM/Qwen/blob/e8e15962d897714944773cca57fa2e460a3655e8/Tongyi%20Qianwen%20LICENSE%20AGREEMENT)