Token Comparison Prefix Leak: How It Works

Yes, a token comparison can leak its matching prefix under specific conditions, typically when the comparison is done in a way that reveals partial information about the match. This article explains the mechanisms, real-world contexts, and how to mitigate the risk.

Background: How Token Comparisons Work

Tokens are used in authentication, API keys, session identifiers, and cryptographic signatures. When a system verifies a token, it typically compares the user-provided value against a stored value. The comparison can be done in several ways:

• Byte-by-byte comparison – The system compares each character one by one, stopping at the first mismatch.
• Hash comparison – The system hashes the input and compares the hash to the stored hash.
• Constant-time comparison – The system performs a comparison that takes the same amount of time regardless of where the mismatch occurs.

The risk of prefix leakage arises when the comparison is not constant-time and the system reveals (directly or indirectly) how many characters matched before a mismatch.

How a Prefix Leak Occurs

A prefix leak happens when an attacker can determine the correct prefix of a token without knowing the whole token. This is often exploited through timing attacks or error messages that reveal the position of the first mismatch.

For example, consider a login form that checks the password character by character and returns a generic error. If the response time varies slightly depending on how many characters were correct, an attacker can measure these times to deduce the correct prefix. By iterating over possible characters, they can gradually build the full token.

Another scenario is a web application that returns a specific error code when the first few characters of an API key are wrong, versus a different error when only the last characters are wrong. This directly reveals the matching prefix.

Real-World Contexts and Examples

Prefix leakage has been observed in various systems:

• Password checkers – Some legacy systems used early-exit string comparisons, allowing timing attacks.
• API key validation – If an API returns a distinct error for an invalid key format versus an invalid key content, it may leak the prefix.
• Database lookups – When a system queries a database with a partial token, the response time can indicate whether a matching prefix exists.
• Block cipher padding oracles – In certain encryption modes, error messages can reveal whether the padding is correct, effectively leaking prefix information.

One classic example is the Lucky Thirteen attack on TLS, which exploited timing differences in MAC verification to recover plaintext prefixes. Although not a direct token comparison, it demonstrates the principle.

How to Choose a Safe Comparison Method

To prevent prefix leakage, use a comparison method that does not reveal partial match information:

• Constant-time comparison – Ensure the comparison takes the same amount of time regardless of the input. This can be achieved by XORing the entire token and checking if the result is zero.
• Hash-based comparison – Compare the hash of the input to the stored hash. Even if the hashes differ, the comparison should be constant-time.
• Use a cryptographic library – Many languages provide functions like crypto.timingSafeEqual (Node.js) or hmac.compare_digest (Python) that are designed to be constant-time.

When implementing your own comparison, avoid early exits and ensure that the same operations are performed for every byte, regardless of whether they match.

Common Problems and Misconceptions

One common misconception is that hashing alone prevents timing attacks. While hashing can help, if the hash comparison itself is not constant-time, the attack still works. For example, comparing two hashes byte-by-byte with an early exit leaks information about the hash prefix, which can be used to recover the original token if the hash is reversible (e.g., unsalted MD5).

Another issue is that developers often assume that network latency makes timing attacks impractical. While remote timing attacks are harder, they are still possible with statistical analysis over many requests. In local or high-precision environments, the attack is much easier.

Also, error messages that differentiate between “invalid token format� and “invalid token value� can leak prefix information, even if the comparison itself is constant-time. Always use generic error messages.

Costs and Trade-offs

Implementing constant-time comparison may have a slight performance cost, but it is negligible for most applications. The main cost is developer time and the risk of introducing bugs. For example, writing a constant-time comparison from scratch is error-prone; using a well-tested library is safer.

There is also a trade-off between security and usability. Requiring longer tokens increases the difficulty of brute-force attacks, but also increases the burden on users. However, for tokens like API keys, the length is usually not a user concern.

Alternatives to Token Comparison

If you are concerned about prefix leakage, consider these alternatives:

• Use a keyed hash (HMAC) – Store an HMAC of the token instead of the raw token. Even if the comparison leaks, the attacker learns the HMAC, not the token, which is useless without the secret key.
• Use a lookup table with a random index – Store tokens in a database with a random ID, and compare only the ID. This avoids comparing the token itself.
• Use a zero-knowledge proof – For high-security scenarios, you can verify that a user knows the token without revealing any information about it.

Each alternative has its own trade-offs in complexity and performance, but they eliminate the risk of prefix leakage entirely.

Frequently Asked Questions

Can a timing attack leak the entire token?

Yes, in theory. If the comparison reveals the exact number of matching characters, an attacker can iteratively guess each character. For a token of length n, the attacker would need at most n × 256 attempts (for ASCII) to recover the full token. In practice, network noise and rate limiting may slow this down, but it is still a serious risk.

Is it safe to compare tokens using a simple == in JavaScript?

No. The == operator in JavaScript compares strings lexicographically and stops at the first mismatch, making it vulnerable to timing attacks. Use crypto.timingSafeEqual for constant-time comparison, but note that it requires both inputs to be of the same length, so you may need to hash them first.

Does using a hash prevent prefix leakage?

Not automatically. If you compare the hash values with a non-constant-time function, the comparison leaks information about the hash prefix. An attacker could potentially use that to recover the hash, and if the hash is weak (e.g., unsalted MD5), they might reverse it. Always use constant-time comparison for hashes as well.

How can I test if my comparison is constant-time?

You can measure the execution time for many random inputs and check if the distribution is independent of the input. Tools like dudect can automate this. Alternatively, use a library that is known to be constant-time and trust its implementation.

What is the difference between a timing attack and a side-channel attack?

A timing attack is a type of side-channel attack that uses the time taken to execute a computation to infer secret information. Side-channel attacks also include power consumption, electromagnetic emissions, and cache behavior. Timing attacks are the most common in web applications because they can be performed remotely.

Should I use a constant-time comparison for all sensitive data?

Yes, it is a good practice for any secret that is compared, such as passwords, API keys, and session tokens. However, it is not necessary for non-secret data where leakage has no security impact. Always weigh the performance cost against the security benefit.

Related

• Apple Watch Screen Replacement: Cost & How It Works
• Find a Lost Phone or Check an Unknown Number Safely
• Enable File and Folder Compression?