UTF-16 is a variable-width Unicode encoding that represents each character using one or two 16-bit code units: characters in the Basic Multilingual Plane fit in a single unit, while characters beyond it are encoded as a pair of units called a surrogate pair. Because its code unit is two bytes, UTF-16 must also specify a byte order, which it signals with a byte order mark or an explicit endianness label. UTF-16 is important because several major platforms and language runtimes use it as their internal string representation, which shapes how string length and indexing behave.
Key takeaways
- UTF-16 uses 16-bit code units and is variable-width, not fixed-width.
- Characters up to
U+FFFF(the Basic Multilingual Plane) use one code unit. - Characters from
U+10000upward use a surrogate pair: a high surrogate (D800toDBFF) plus a low surrogate (DC00toDFFF). - Endianness is conveyed by a BOM (
FE FFbig-endian,FF FElittle-endian) or by labels likeUTF-16BE. - UTF-16 is not ASCII-compatible at the byte level and encodes ASCII characters in two bytes.
- Runtimes exposing UTF-16 code units count supplementary characters as length 2, a frequent bug source.
How does UTF-16 encode characters?
Unicode organizes code points into planes. The first plane, the Basic Multilingual Plane (BMP), covers code points U+0000 through U+FFFF and includes the vast majority of characters in common use. UTF-16 encodes any BMP character as a single 16-bit code unit whose value equals the code point.
Code points from U+10000 upward are called supplementary characters and cannot fit in one 16-bit unit. UTF-16 encodes these using two code units through a mechanism called surrogate pairs.
What is a surrogate pair?
A surrogate pair is two special code units that together represent one supplementary character. Unicode reserves the range U+D800 to U+DFFF exclusively for this purpose, and no real character is ever assigned there. The range splits into two halves:
- High (lead) surrogates:
U+D800toU+DBFF. - Low (trail) surrogates:
U+DC00toU+DFFF.
A valid supplementary character is always a high surrogate followed by a low surrogate. Because the two ranges do not overlap, a decoder can always tell whether a code unit begins a pair, completes one, or stands alone as a BMP character. The reserved range is why U+D800 to U+DFFF must never appear as isolated code points in well-formed Unicode text.
| Code point | UTF-16 code units | Note |
|---|---|---|
U+0041 (A) |
one unit: 0041 |
BMP character |
U+20AC (euro sign) |
one unit: 20AC |
BMP character |
U+1F600 (a supplementary emoji) |
two units: high + low surrogate | Surrogate pair |
How does UTF-16 handle byte order?
Since each code unit is two bytes, those bytes can be stored most-significant-first (big-endian) or least-significant-first (little-endian). UTF-16 resolves the ambiguity in two ways:
- A byte order mark at the start of the stream encodes
U+FEFF. If the reader seesFE FFit is big-endian; if it seesFF FEit is little-endian. - An explicit encoding label:
UTF-16BEorUTF-16LEfixes the order without needing a BOM.
When a stream is simply labeled UTF-16 with no BOM and no explicit endianness, the reader must fall back on an external convention, which is a common source of decoding errors.
Why do string lengths surprise developers in UTF-16 runtimes?
Several widely used language runtimes represent strings internally as sequences of UTF-16 code units and expose length and indexing in terms of those units. In such an environment, a BMP character counts as one unit, but a supplementary character, encoded as a surrogate pair, counts as two.
The practical effect is that a string containing a single emoji or other supplementary character can report a length of 2, and naive indexing can split a surrogate pair in half, producing an invalid lone surrogate. When you must handle text correctly, iterate by whole code points rather than by raw code units, and never slice a string at an arbitrary code-unit index without checking for surrogates.
How does UTF-16 compare to UTF-8?
Both encode the full Unicode range, but they make different trade-offs.
| Property | UTF-8 | UTF-16 |
|---|---|---|
| Code unit size | 8 bits (1 byte) | 16 bits (2 bytes) |
| ASCII compatibility | Byte-identical to ASCII | Not byte-compatible |
| Bytes for a basic Latin character | 1 | 2 |
| Endianness concern | None | Yes, needs BOM or label |
| Typical web and file use | Dominant default | Less common for interchange |
For English-heavy and mixed text, UTF-8 is usually more compact and avoids endianness entirely, which is why it dominates the web and file storage. UTF-16 shines mainly inside platforms whose APIs already speak it.
Where is UTF-16 actually used?
UTF-16 is most visible as an internal string representation. Some major operating system APIs and language runtimes store strings as UTF-16 code units, so when you call into those APIs you are effectively working with UTF-16 even if your files are UTF-8. It also appears in certain file formats and legacy data. The key takeaway is that you often meet UTF-16 at a boundary, converting to and from UTF-8, rather than choosing it as your storage format.
What mistakes should I avoid with UTF-16?
- Assuming one code unit equals one character. Supplementary characters are two units.
- Slicing at arbitrary indices. You can cut a surrogate pair in half and corrupt text.
- Ignoring endianness. Reading little-endian bytes as big-endian yields garbage; honor the BOM or label.
- Treating UTF-16 bytes as ASCII. Every character, including ASCII, occupies two bytes.
- Emitting lone surrogates. A high or low surrogate alone is invalid Unicode.
Frequently asked questions
Is UTF-16 a fixed-width encoding?
No. It is variable-width: characters in the Basic Multilingual Plane use one 16-bit code unit, while characters beyond it use two units called a surrogate pair.
What is a surrogate pair?
It is two 16-bit code units, a high surrogate in the range D800 to DBFF followed by a low surrogate in DC00 to DFFF, that together encode one supplementary character above U+FFFF.
How does UTF-16 indicate endianness?
Through a byte order mark at the start (FE FF for big-endian, FF FE for little-endian) or by an explicit label such as UTF-16BE or UTF-16LE.
Why do many languages report surprising string lengths?
Languages that expose UTF-16 code units count a supplementary character as two units, so a single emoji can have a length of 2 rather than 1.
Is UTF-16 compatible with ASCII bytes?
No. Unlike UTF-8, UTF-16 encodes even ASCII characters in two bytes, so ASCII text is not byte-identical and tools expecting single-byte ASCII will misread it.
What are surrogate code points used for?
The range U+D800 to U+DFFF is reserved exclusively for surrogates in UTF-16 and never represents characters on its own, so those code points must not appear alone.
When should I choose UTF-16 over UTF-8?
Mostly when interoperating with platforms or APIs that use UTF-16 internally. For storage and web interchange, UTF-8 is usually the better default.