split-by-grapheme

How to split a string into characters in JavaScript?

You'd think it's just 'abc'.split('')? Not so fast.

"🏴󠁧󠁢󠁥󠁮󠁧󠁿 and 🏴󠁧󠁢󠁳󠁣󠁴󠁿".split("");

// [ "\uD83C", "\uDFF4", "\uDB40", "\uDC67", "\uDB40", "\uDC62", "\uDB40", "\uDC65", "\uDB40", "\uDC6E", "\uDB40", "\uDC67", "\uDB40", "\uDC7F", " ", "a", "n", "d", " ", "\uD83C", "\uDFF4", "\uDB40", "\uDC67", "\uDB40", "\uDC62", "\uDB40", "\uDC73", "\uDB40", "\uDC63", "\uDB40", "\uDC74", "\uDB40", "\uDC7F" ]

Why are there 33 weird chunks instead of the 2 emojis? Because many emojis, flags and rarely used characters are represented as a combination of several Unicode characters. The Scottish flag emoji is, for example, 28 bytes long, unlike the letter a which is only 1 byte long.

It's not just the emojis - there are also diacritics, ligatures and other special characters used in many languages that are represented as a combination of several Unicode characters.

Then how to split the string into characters properly, taking into account all the emojis, diacritics and special characters?

There are many ways proposed on the Internet, but most of them are either too complicated or don't work properly. Here are some ideas:

// Works for some characters, doesn't work for emojis
[...string];

// Looks for the combining characters, but doesn't work for emojis
string.match(/[\uD800-\uDBFF][\uDC00-\uDFFF]|[^\uD800-\uDFFF]/g);

// Works for some emojis, but doesn't work for flags
"Dragon 🐉".split(/(?!$)/u);

Some suggestions use regular expressions to find the specific ranges for emojis or other characters, but it's not a good idea because the ranges are not fixed and can change in the future.

The solution

Use the Intl.Segmenter API. It's supported in all recent browsers except Firefox, and in Node.js since version 16.

Also, to make the code easily readable and more semantic, let's call it this way:

import { byGrapheme } from "split-by-grapheme";

const str = "🏴󠁧󠁢󠁥󠁮󠁧󠁿 and 🏴󠁧󠁢󠁳󠁣󠁴󠁿";
str.split(byGrapheme); // [ "🏴󠁧󠁢󠁥󠁮󠁧󠁿", " ", "a", "n", "d", " ", "🏴󠁧󠁢󠁳󠁣󠁴󠁿" ]

How does it work?

The String.split() function takes an object as an argument and looks for a Symbol called Symbol.split inside that object. If it exists, it uses the function located at [Symbol.split] to split the string. And for the splitting algorithm itself, Intl.Segmenter API knows exactly how to split the string into characters (more precisely, into graphemes).

What's a grapheme?

A grapheme is a single unit of a writing system that usually looks like one character, even though technically it's a combination of a few characters. It can be a letter, a ligature, a letter with diacritics, a special symbol, an emoji, or a combination of several Unicode characters.

So, what about Firefox and old browsers?

Not much. Fall back to the spread operator [...str] and hope for the best.

Another package called grapheme-splitter is also available, with a polyfill for Intl.Segmenter.