rairye-nlp

ja-sentence

Light-weight sentence tokenizer for Japanese.

zh-sentence

Light-weight sentence tokenizer for Chinese languages.

kr-sentence

Light-weight sentence tokenizer for Korean. Supports both full-width and half-width punctuation marks.

mnl-ws-norm

Light-weight tool for normalizing whitespace, splitting lines, and accurately tokenizing words (no regex). Multiple natural languages supported.

mnl-punct-norm

Tool for stripping and normalizing punctuation and other non-alphanumeric characters. Supports multiple natural languages. Useful for scrapping, machine learning, and data analysis.

convert-with-ents

Light-weight tool for converting characters in a string into common HTML entities (without regex).

st-no-love

Tool for escaping script tags using backslashes (no regex).