UTF-16 UTF-16 (16-bit Unicode Transformation Format) Unicode
The internal string representation for JavaScript, Java, and Windows APIs — 2 bytes per unit for most characters, 4 bytes (surrogate pairs) for characters outside the Basic Multilingual Plane.
What it is
UTF-16 represents most Unicode characters using a single 2-byte unit, and characters outside the Basic Multilingual Plane (BMP) — including most emoji and many less-common scripts — using a pair of 2-byte units called a surrogate pair (4 bytes total). It's the internal string representation used by JavaScript, Java, and Windows APIs (the Windows WCHAR type), which is why understanding it matters even if you rarely encode files as UTF-16 directly — you're interacting with UTF-16 semantics constantly through those runtimes' string types.
UTF-16 is rarely used for file storage or network transmission on the modern web — UTF-8 dominates there — but it's unavoidable when reasoning about string length, indexing, and iteration behavior in JS and Java.
Byte structure
| Code point range | Units used | Representation |
|---|---|---|
| U+0000 – U+FFFF (Basic Multilingual Plane) | 1 unit (2 bytes) | Direct 16-bit value |
| U+10000 – U+10FFFF (Supplementary Planes) | 2 units (4 bytes, a "surrogate pair") | A high surrogate (0xD800–0xDBFF) followed by a low surrogate (0xDC00–0xDFFF), together encoding the full code point |
Surrogate pairs are the mechanism that lets a fundamentally 16-bit-unit format reach beyond 65,536 possible values — the two surrogate halves combine mathematically to address the full Unicode range up to U+10FFFF.
Why your text is garbled (symptom → cause)
| Symptom | Likely cause |
|---|---|
JavaScript .length reports the wrong character count for a string with emoji |
JS .length counts UTF-16 code units, not visible characters — an emoji requiring a surrogate pair counts as 2, not 1 |
| String slicing/indexing splits an emoji into two broken halves | Slicing at a raw index can land in the middle of a surrogate pair, producing two invalid "lone surrogate" units instead of a valid character |
| File shows as mostly null bytes or every-other-character garbage when opened as plain ASCII/UTF-8 | The file is actually UTF-16 (common on Windows, e.g. some .reg, .rtf, or PowerShell-generated files) being read with the wrong decoder |
| Byte order looks reversed compared to expected | UTF-16 has two byte-order variants — Big-Endian (UTF-16BE) and Little-Endian (UTF-16LE) — reading with the wrong endianness reverses each 2-byte unit |
Compatibility & gotchas
- JavaScript strings are UTF-16 under the hood.
.length,charAt(), and index-based access all operate on UTF-16 code units, not full characters — for text that might contain emoji or other supplementary-plane characters, useArray.from(str)or the spread operator ([...str]) to iterate by actual code point instead. - Java's
chartype is a UTF-16 code unit, not a full character — same surrogate-pair caveat applies;codePointAt()exists specifically to handle characters that don't fit in a singlechar. - Byte order matters and isn't self-evident — a raw UTF-16 file needs either an explicit BE/LE declaration or a byte order mark (BOM) at the start to be decoded correctly; without one, a decoder has to guess.
- Windows APIs use UTF-16LE internally (
WCHAR), which is part of why some Windows-generated text files (older.regfiles, some PowerShell output redirected to a file) show up as UTF-16LE rather than UTF-8, surprising developers used to Unix-family tooling defaults. - Not commonly used for storage or transmission on the web — UTF-8 is strongly preferred there due to ASCII compatibility and lack of byte-order ambiguity; UTF-16 mainly matters as an in-memory representation inside specific runtimes.
Fixing it
- JS iterating by actual character: use
for (const char of str)or[...str]instead of index-based loops, so surrogate pairs are treated as a single iteration step rather than two. - Reading a UTF-16 file correctly: explicitly specify the encoding and endianness — e.g., Python's
open(path, encoding='utf-16')(auto-detects BOM) orutf-16-le/utf-16-beif you know the byte order and there's no BOM. - Command-line conversion:
iconv -f UTF-16LE -t UTF-8 input.txt -o output.txt(adjustLE/BEbased on the source's actual byte order). - Detecting UTF-16 vs. UTF-8 mistakenly: a file that looks like it has null bytes between every character when viewed as plain text is a strong signal it's actually UTF-16 (each ASCII character is 2 bytes, with the second byte being
0x00).
Comparison
| Encoding | Key difference from UTF-16 |
|---|---|
| UTF-8 | Variable 1-4 byte width, ASCII-compatible, no byte-order ambiguity — the dominant choice for storage/transmission |
| UCS-2 | UTF-16's predecessor — fixed 2 bytes only, cannot represent characters outside the BMP at all (no surrogate pair mechanism) |
| UTF-32 | Fixed 4 bytes for every character, no surrogate pairs needed, but roughly double the storage cost of UTF-16 for BMP-heavy text |
FAQ
Why does JavaScript's string.length give the wrong count for emoji?
Because .length counts UTF-16 code units, not visible characters (technically "grapheme clusters"). Most emoji live outside the Basic Multilingual Plane and require a surrogate pair — 2 code units — so a string with one emoji reports .length as 2, not 1.
What's the difference between UTF-16 and UCS-2?
UCS-2 is UTF-16's fixed-width predecessor and can only represent the Basic Multilingual Plane (no surrogate pairs, no way to reach beyond U+FFFF). UTF-16 extended the same basic 2-byte-unit approach with surrogate pairs specifically to reach the full Unicode range once Unicode grew beyond 65,536 code points.
Do I need to worry about UTF-16 if I only use UTF-8 for storage?
Mostly no for file/network purposes — but if you're working in JavaScript or Java, you're interacting with UTF-16 string semantics regardless of what encoding your data is stored or transmitted in, since those languages' native string types are UTF-16 internally.
Fun fact
Unicode's original 1990s design assumed 16 bits (65,536 code points) would be enough for every character system in existence — UCS-2 was built on that assumption. Real-world script coverage requirements outgrew that ceiling faster than expected, which is why UTF-16's surrogate-pair mechanism had to be retrofitted on top of what was originally meant to be a simple fixed-width format.