huggingface has released a Japanese model for BERT.
The Japanese model is included in transformers.
However, I stumbled over a few things before I could get it to actually work in a Mac environment, so I’ll leave a note.
Preliminaries: Installing mecab
The morphological analysis engine, mecab, is required to use BERT’s Japanese model.
The tokenizer will probably ask for mecab.
This time, we will use homebrew to install Mecab and ipadic.
[Read More]