Percent-encoding is the mechanism URLs use to carry a character as data when that character would otherwise be read as structure, and RFC 3986 decides which characters need it by sorting every character into three buckets: unreserved, reserved, and everything else. The reserved bucket splits further into general delimiters and sub-delimiters. Knowing which bucket a character is in, and which part of the URL it sits in, is the whole game: it tells you exactly when a character is safe as-is and when it must become a %XX escape. This article walks through those character sets and the per-component rules that make the same character safe in one place and dangerous in another.
How does percent-encoding work mechanically?
Percent-encoding replaces a character with a percent sign followed by the two hexadecimal digits of each of its bytes. A space is byte 0x20, so it becomes %20; a hash is 0x23, so it becomes %23. For characters beyond ASCII, the rule is to encode the UTF-8 bytes one escape at a time. The character é is two bytes in UTF-8 (0xC3 0xA9), so it encodes to %C3%A9. Decoding reverses this exactly, reading each %XX back into a byte and interpreting the byte stream as UTF-8. The mechanics are trivial. The judgement is knowing which characters to run through this transformation, and that is a question about character sets, not about the %XX mechanism itself.
Which characters are unreserved and always safe?
The unreserved characters may appear literally anywhere in a URL and must never be percent-encoded. RFC 3986 defines the set as the ASCII letters, the digits, and four punctuation marks:
A-Z a-z 0-9 - . _ ~These 66 characters are the bedrock. They have no structural meaning in any URL component, so they always stand for themselves, and a correct encoder leaves them untouched. Encoding them anyway is technically legal, since %41 decodes to A, but it is discouraged: it makes URLs longer and harder to read, and it can even cause two URLs that should be identical to be treated as different by systems that compare them before normalising. If a character is in this set, the answer is always “leave it alone”.
What are the reserved characters?
The reserved characters are the ones RFC 3986 gives a delimiting job, and they are split into two named groups: general delimiters and sub-delimiters. They are legal in a URL, but only when they are actually acting as delimiters; when you want one to appear as ordinary data, you percent-encode it so the parser does not mistake it for structure.
| Group | Characters | Typical role |
|---|---|---|
| gen-delims | : / ? # [ ] @ |
Separate major URL parts |
| sub-delims | ! $ & ' ( ) * + , ; = |
Structure within a component |
The general delimiters carve a URL into its big pieces. The colon separates the scheme and also the host from the port; the slash separates path segments; the question mark begins the query; the hash begins the fragment; the square brackets wrap an IPv6 host; the at-sign separates userinfo from the host. The sub-delimiters do finer-grained work inside a component, most familiarly the ampersand and equals sign that separate and assign query parameters. Everything not in the unreserved or reserved sets, such as a space, a double quote, or the angle brackets, is neither safe nor a delimiter and simply must be encoded to appear in a URL at all.
Why does the same character need encoding in one place but not another?
Because a reserved character is only “reserved” for the component where it does a job, so whether it must be encoded depends entirely on where in the URL it appears. A question mark separates the path from the query, but once you are inside the query, a literal question mark is unambiguous data and does not need encoding. A slash separates path segments, but inside a query value a slash is usually harmless. The clearest example is the difference between putting a value in the path versus the query. Consider these worked cases:
| Character | In a path segment | In a query value |
|---|---|---|
/ |
Must encode (%2F) to keep it in one segment |
Usually safe as-is |
? |
Safe (query has not started) | Safe (already in query) |
& |
Safe as-is | Must encode (%26) or it starts a new parameter |
# |
Must encode (%23) or it starts the fragment |
Must encode (%23) |
= |
Safe as-is | Encode in a value to avoid key/value confusion |
The lesson is that “does this need encoding?” is not a property of the character alone; it is a property of the character-plus-location. The hash is the one true exception: because it begins the fragment, which is the last part of a URL, a literal hash needs encoding almost everywhere it appears as data.
How does encoding differ across the URL’s parts?
Each component of a URL has its own set of characters that are legal unencoded, and the safe rule when inserting a value is to encode everything except the unreserved set. A URL breaks into scheme, authority (host and port), path, query, and fragment, and RFC 3986 defines slightly different allowances for each. Rather than memorise five different tables, the practical strategy is to encode a value you are inserting as if only the unreserved characters were safe, which is always correct if sometimes over-cautious. That is precisely what JavaScript’s encodeURIComponent does: it encodes everything except A-Z a-z 0-9 - _ . ~ and the marks ! * ' ( ). For most work, treating any single value you drop into a URL as a “component” and encoding it fully is the reliable habit. You can watch these transformations happen character by character in a browser-based URL encoder, which runs entirely on your machine, so you can paste a real value from an application and see exactly which characters change without sending anything to a server.
How is percent-encoding different from HTML escaping?
Percent-encoding and HTML escaping solve different problems in different contexts, and confusing them produces subtle bugs. Percent-encoding makes a character safe inside a URL using %XX byte escapes, and it is governed by RFC 3986. HTML escaping makes a character safe inside HTML markup using named or numeric entities such as & for an ampersand, and it is governed by the HTML specification. An ampersand in a URL that you then place inside an HTML attribute may need both: percent-encoded as %26 so it does not split a query parameter, and, if the surrounding context is HTML, the whole URL’s ampersands escaped as & so the markup parses. They stack rather than substitute. Keeping them straight is a matter of asking which layer you are protecting: the URL parser wants percent-encoding, the HTML parser wants entities.
Does encoding make a URL safe to trust?
No. Percent-encoding is about transport, not security, and it neither sanitises input nor prevents any kind of injection. Encoding guarantees that the bytes you put in come back out unchanged after the URL is parsed; it makes no promise about whether those bytes are safe to use in a database query, a shell command, or an HTML page. A correctly encoded value can still carry a malicious payload for whatever system ultimately consumes it. So encode values to move them through URLs intact, but validate and escape them again, separately, for their final destination. And because URLs frequently end up in logs, browser history, and referrer headers, keep genuine secrets out of them entirely rather than relying on encoding to obscure anything, since %XX escapes are trivially reversible and hide nothing.
Percent-encoding stops being mysterious once you hold two questions in mind: which character set is this character in, and which part of the URL does it sit in. The unreserved set is always safe, the reserved sets are safe only when doing their delimiting job, and anything else must be encoded. Add the rule that a value you insert should be encoded as a full component, and you can predict exactly which characters will turn into %XX and why.