Canonical JSON: Sort Keys for Stable Hashes
You hash a JSON payload to build a cache key, then hash the same data from another service and get a different value. Nothing changed except key order or a space after a colon. Canonical JSON fixes this: one agreed byte string for each value, so sorting keys and hashing the result gives the same digest everywhere. Here is how to do it in JavaScript, Python, Go and jq, and where the simple version breaks.
Why the same JSON hashes differently
A hash function sees bytes, not data. These two documents parse to the same object:
Their SHA-256 hashes have nothing in common. Three things usually cause the mismatch:
- Key order. Most serializers write keys in insertion order, so two code paths that build the same object in a different order produce different text.
- Whitespace. Python's
json.dumpsputs a space after:and,by default.JSON.stringifydoes not. - Escapes and numbers. Python writes
"café"as"café"unless you passensure_ascii=False, and it writes1.0where JavaScript writes1.
Sort keys and drop whitespace
For data you control on both ends, a simple rule covers most cases: sort object keys recursively, leave arrays alone, and emit no whitespace.
JavaScript has no built-in option for sorted keys, so you write a small helper:
Don't sort arrays. ["a", "b"] and ["b", "a"] are different values in JSON, and if your hash treats them as equal you will eventually dedupe two requests that were not duplicates.
Python does it in one call, but you need all three arguments:
Leave out separators and you get {"a": 1, "b": 2} with spaces, which will never match the JavaScript output.
Go sorts map keys for you. json.Marshal on a map[string]any writes keys in sorted order, but a struct is written in field declaration order. Go also escapes <, > and & as <, > and & by default, which no other language here does. Turn that off with an encoder:
Encode appends a newline, so trim it before hashing. On the command line, jq does the whole thing with two flags:
-S sorts keys at every level and -c prints one compact line. jq adds a trailing newline too, which matters if you compare against a hash computed in code. And since jq 1.7 keeps number literals as they appeared in the input, 1e3 comes out as 1E+3, so jq output is stable but not canonical across producers.
What each language does by default
| Key order | Whitespace | Non-ASCII | 1.0 | Sorts keys with | |
|---|---|---|---|---|---|
JavaScript JSON.stringify | insertion, integer-like keys first | none | literal | 1 | custom helper |
Python json.dumps | insertion | ", " and ": " | é | 1.0 | sort_keys=True |
Go json.Marshal | sorted for maps, field order for structs | none | literal, but <>& escaped | 1 | automatic for maps |
| jq 1.7 | input order | pretty-printed | literal | 1.0 (kept as written) | -S |
{"total": 3, "404": 1} serializes as {"404":1,"total":3}, because object properties with integer-like names always come first. Sorting makes this irrelevant, which is one more reason to sort rather than trust insertion order.
Numbers are where the simple version fails
Sorted keys plus compact output gets you stable hashes within one language. Across languages, numbers still differ. Python keeps the float/int distinction, so json.dumps(1.0) is "1.0", while JavaScript only has one number type and prints 1. Large integers are worse: a JavaScript service that reads 9007199254740993 gets 9007199254740992, and its hash no longer matches the Python service that kept every digit. If your IDs are that large, send them as strings. The large numbers guide covers why.
There is also a smaller trap in sorting itself. JavaScript's .sort() compares UTF-16 code units, and Python's sorted() compares code points. For plain ASCII keys they agree. For keys that mix emoji with characters in the U+E000 to U+FFFF range, they don't.
RFC 8785: the standard for this
If two parties you don't control need to agree on bytes, for example when you sign JSON and someone else verifies it, use the JSON Canonicalization Scheme (JCS), RFC 8785. It pins down every choice above:
- object keys sorted by UTF-16 code units, recursively
- no whitespace between tokens
- numbers written the way ECMAScript
Number.prototype.toStringwrites them, so1.0becomes1and1e3becomes1000 - strings with the minimum escaping, and UTF-8 output
canonicalize on npm, rfc8785 on PyPI, or github.com/gowebpki/jcs in Go. My rule: hand-rolled sorting is fine for cache keys and dedup inside one codebase, and JCS is the only safe choice for signatures or anything another team verifies.
Checking that two payloads really are equal
Before you blame canonicalization for a mismatch, confirm the data is actually the same. Paste both documents into the JSON diff tool. It compares by key name, so reordered keys and whitespace never show up as differences, and anything it does flag is a real change in values or array order.
When you want to see what your canonical string looks like, the JSON minifier strips whitespace the same way JSON.stringify does. It keeps key order, so sort first if you need the canonical form. If the canonical string travels in a header or URL, encode it with the Base64 encoder after canonicalizing, not before, or you end up hashing the encoding instead of the data.
One last check worth adding to your tests: serialize the same object twice from different construction orders and assert the hashes match. It takes five lines and it catches the day someone swaps a map for a struct in Go, or drops separators from a Python call.