README.md

metadata

language:
  - ja
license: mit
library_name: transformers
tags:
  - fastText
  - embedding
pipeline_tag: feature-extraction
widget:
  - text: 海賊王におれはなる
    example_title: ワンピース

fasttext-jp-embedding

Pretrained FastText word vector for Japanese

Reference

• fastText
  https://github.com/facebookresearch/fastText

• word vector data
  https://dl.fbaipublicfiles.com/fasttext/vectors-crawl/cc.ja.300.vec.gz

Usage

Google Colaboratory Example

! apt install aptitude swig > /dev/null 
! aptitude install mecab libmecab-dev mecab-ipadic-utf8 git make curl xz-utils file -y > /dev/null 
! pip install transformers torch mecab-python3, torchtyping > /dev/null 
! ln -s /etc/mecabrc /usr/local/etc/mecabrc

from transformers import pipeline

pipeline = pipeline("feature-extraction", model="paulhindemith/fasttext-jp-embedding", revision="2022.11.6", trust_remote_code=True)
pipeline("海賊王におれはなる")