P5-lingua-zh-toke

Jul 20, 2023

Chinese Tokenizer on steroids

This modules requires LinguaZHTaBE, a Chinese Tokenizer on steroids; it’s a thin wrapper around LinguaZHTaBE, as well as its interface with overload and utf8 semantics.



Checkout these related ports: