Skip to content
Back to Blog
By JSONConvert Team··6 min read

Canonical JSON: Sort Keys for Stable Hashes

You hash a JSON payload to build a cache key, then hash the same data from another service and get a different value. Nothing changed except key order or a space after a colon. Canonical JSON fixes this: one agreed byte string for each value, so sorting keys and hashing the result gives the same digest everywhere. Here is how to do it in JavaScript, Python, Go and jq, and where the simple version breaks.

Why the same JSON hashes differently

A hash function sees bytes, not data. These two documents parse to the same object:

Their SHA-256 hashes have nothing in common. Three things usually cause the mismatch:

  • Key order. Most serializers write keys in insertion order, so two code paths that build the same object in a different order produce different text.
  • Whitespace. Python's json.dumps puts a space after : and , by default. JSON.stringify does not.
  • Escapes and numbers. Python writes "café" as "café" unless you pass ensure_ascii=False, and it writes 1.0 where JavaScript writes 1.
Any one of these is enough to break a cache key, an idempotency check, or a signature.

Sort keys and drop whitespace

For data you control on both ends, a simple rule covers most cases: sort object keys recursively, leave arrays alone, and emit no whitespace.

JavaScript has no built-in option for sorted keys, so you write a small helper:

Don't sort arrays. ["a", "b"] and ["b", "a"] are different values in JSON, and if your hash treats them as equal you will eventually dedupe two requests that were not duplicates.

Python does it in one call, but you need all three arguments:

Leave out separators and you get {"a": 1, "b": 2} with spaces, which will never match the JavaScript output.

Go sorts map keys for you. json.Marshal on a map[string]any writes keys in sorted order, but a struct is written in field declaration order. Go also escapes <, > and & as <, > and & by default, which no other language here does. Turn that off with an encoder:

Encode appends a newline, so trim it before hashing. On the command line, jq does the whole thing with two flags:
-S sorts keys at every level and -c prints one compact line. jq adds a trailing newline too, which matters if you compare against a hash computed in code. And since jq 1.7 keeps number literals as they appeared in the input, 1e3 comes out as 1E+3, so jq output is stable but not canonical across producers.

What each language does by default

Key orderWhitespaceNon-ASCII1.0Sorts keys with
JavaScript JSON.stringifyinsertion, integer-like keys firstnoneliteral1custom helper
Python json.dumpsinsertion", " and ": "é1.0sort_keys=True
Go json.Marshalsorted for maps, field order for structsnoneliteral, but <>& escaped1automatic for maps
jq 1.7input orderpretty-printedliteral1.0 (kept as written)-S
That integer-like quirk catches people. In JavaScript, {"total": 3, "404": 1} serializes as {"404":1,"total":3}, because object properties with integer-like names always come first. Sorting makes this irrelevant, which is one more reason to sort rather than trust insertion order.

Numbers are where the simple version fails

Sorted keys plus compact output gets you stable hashes within one language. Across languages, numbers still differ. Python keeps the float/int distinction, so json.dumps(1.0) is "1.0", while JavaScript only has one number type and prints 1. Large integers are worse: a JavaScript service that reads 9007199254740993 gets 9007199254740992, and its hash no longer matches the Python service that kept every digit. If your IDs are that large, send them as strings. The large numbers guide covers why.

There is also a smaller trap in sorting itself. JavaScript's .sort() compares UTF-16 code units, and Python's sorted() compares code points. For plain ASCII keys they agree. For keys that mix emoji with characters in the U+E000 to U+FFFF range, they don't.

RFC 8785: the standard for this

If two parties you don't control need to agree on bytes, for example when you sign JSON and someone else verifies it, use the JSON Canonicalization Scheme (JCS), RFC 8785. It pins down every choice above:

  • object keys sorted by UTF-16 code units, recursively
  • no whitespace between tokens
  • numbers written the way ECMAScript Number.prototype.toString writes them, so 1.0 becomes 1 and 1e3 becomes 1000
  • strings with the minimum escaping, and UTF-8 output
You don't need to implement it. Use canonicalize on npm, rfc8785 on PyPI, or github.com/gowebpki/jcs in Go. My rule: hand-rolled sorting is fine for cache keys and dedup inside one codebase, and JCS is the only safe choice for signatures or anything another team verifies.

Checking that two payloads really are equal

Before you blame canonicalization for a mismatch, confirm the data is actually the same. Paste both documents into the JSON diff tool. It compares by key name, so reordered keys and whitespace never show up as differences, and anything it does flag is a real change in values or array order.

When you want to see what your canonical string looks like, the JSON minifier strips whitespace the same way JSON.stringify does. It keeps key order, so sort first if you need the canonical form. If the canonical string travels in a header or URL, encode it with the Base64 encoder after canonicalizing, not before, or you end up hashing the encoding instead of the data.

One last check worth adding to your tests: serialize the same object twice from different construction orders and assert the hashes match. It takes five lines and it catches the day someone swaps a map for a struct in Go, or drops separators from a Python call.

Related Tools