Hashing & signing

What is a hash function?

A hash function takes any input - a single character, a 10GB video file, a JSON blob - and produces a fixed-length output called a digest or hash. For SHA-256 that output is always 256 bits (32 bytes), usually written as a 64-character hex string:

SHA-256("hello")
→ 2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824

SHA-256("hello!")
→ 334d016f755cd6dc58c53a86e183882f8ec14f52fb05345887c8a5edd42c87b7

Change one character and the output is completely different - there is no visible relationship between the input and output. This is intentional.

A cryptographic hash function has three properties that matter for security:

• One-way (preimage resistance) - given a hash output, you cannot work backwards to find the input. There is no "unhash" operation.
• Collision resistant - it should be computationally infeasible to find two different inputs that produce the same hash output.
• Avalanche effect - a tiny change to the input (one bit) produces a completely different output. There is no "close to matching."

These properties are what make hashing useful for security. A hash is not a compressed version of the input - you cannot recover the original. It's a fingerprint.

MD5 - do not use for security

MD5 (Message Digest 5) was designed in 1991 and produces a 128-bit (16 byte) digest. For most of the 1990s it was the standard. It is now broken for any security purpose.

The specific failure is collision resistance. In 2004, researchers demonstrated the first practical MD5 collision - two different inputs producing the same hash. By 2008 this was exploited in a real attack: researchers created a rogue certificate authority certificate that was accepted as legitimate by browsers, because its MD5 hash matched a legitimately signed certificate. The collision was engineered. This was not a theoretical paper - it was a working attack against the certificate infrastructure the entire web relied on.

A 128-bit output also means there are only 2¹²⁸ possible hashes. That sounds large but a birthday attack (where you look for any two colliding inputs, not a specific one) only requires around 2⁶⁴ operations - which was already within reach of determined attackers with hardware available in the early 2000s.

MD5 is still fine for non-security purposes: checksums to detect accidental corruption, cache keys, deduplication fingerprints. If the question is "did this file download correctly?" MD5 is fast and adequate. If the question is "can an attacker forge this?" - no.

SHA-1 - also broken, but more recently

SHA-1 (Secure Hash Algorithm 1) was designed by the NSA and published in 1995. It produces a 160-bit digest. For a decade it replaced MD5 as the standard.

Collision attacks against SHA-1 were theorised for years. In 2017, Google published SHAttered- the first practical SHA-1 collision. They produced two different PDF files with identical SHA-1 hashes. The computation required 9.2 quintillion SHA-1 computations and took the equivalent of 6,500 years of single-CPU time - but it ran on Google's distributed infrastructure in a matter of months. It was a one-time cost to produce the collision.

Chrome stopped trusting SHA-1 TLS certificates in 2017. Git still uses SHA-1 internally for object IDs, though this is being migrated to SHA-256 - the Git use case is different (it's detecting accidental corruption, not defending against an adversary constructing collisions), but it has become increasingly uncomfortable.

SHA-1 is not safe for signatures, MACs, certificates, or any context where an adversary might benefit from finding a collision.

SHA-2 - the current standard

SHA-2 is a family of hash functions, all designed by the NSA and published in 2001. The number in the name is the output size in bits:

• SHA-256 - 256-bit (32 byte) output. The most widely used. Default choice for HMAC, TLS certificates, code signing, and most other security applications.
• SHA-512 - 512-bit (64 byte) output. Larger digest, slightly harder to brute-force. On 64-bit hardware it can actually be faster than SHA-256 due to how the algorithm works internally. Used where a larger digest is required or where you want headroom against future advances.
• SHA-384 - a truncated version of SHA-512. Used in some TLS cipher suites. You will encounter it but rarely need to choose it explicitly.
• SHA-224 - a truncated SHA-256. Rarely used.

No practical collision attack against any SHA-2 variant has been found. The algorithm has been intensively studied for over twenty years. SHA-256 is the safe, well-supported, widely-tested default - it is what you should use unless you have a specific reason to pick something else.

SHA-3 - the backup

SHA-3 was standardised by NIST in 2015 after a public competition (won by an algorithm called Keccak). It produces the same output sizes as SHA-2 (224, 256, 384, 512 bits) but uses a completely different internal construction called a sponge function.

It was not designed because SHA-2 was broken. It was designed as a hedge - if a structural weakness were ever found in the SHA-2 design, the world would have a ready alternative with a fundamentally different architecture. SHA-2 and SHA-3 share no internal design, so a weakness in one would not imply a weakness in the other.

In practice, SHA-3 is rarely required. You will encounter it in certain standards and protocols. If a library or spec asks for SHA-3, use it; otherwise SHA-256 is the right default. SHA-3 is not "better" than SHA-2 - it is different, with a different security profile and generally slower on software implementations.

Quick reference

Where you encounter hashing as a developer

Hashing turns up in several different contexts, and the right algorithm choice varies:

• TLS certificates- the CA's signature over a certificate uses SHA-256. SHA-1 certificates were deprecated and removed from browser trust.
• HMAC / webhook signing - SHA-256 is the standard. See the HMAC article.
• Password storage - do not use SHA-256 for passwords. SHA-256 is extremely fast, which makes brute-force attacks cheap. Password hashing needs a deliberately slow algorithm: bcrypt, scrypt, or Argon2. This is covered in the password hashing article.
• Git - every commit, tree, and blob has a SHA-1 hash as its ID. New repositories default to SHA-256 in recent Git versions.
• Content addressing / integrity checks - npm, Cargo, and other package managers hash packages (usually SHA-256 or SHA-512) so you can verify a download is byte-for-byte what was published.
• Digital signatures - when signing a document or JWT, you hash the content first (SHA-256), then sign the hash with a private key. You never sign the raw content directly - the hash reduces it to a fixed size that the signing algorithm can handle efficiently.

What "signing" means

A digital signature uses asymmetric cryptography (a private/public key pair) to prove that a specific party produced or approved a piece of data. The process:

1. Hash the content with SHA-256 (or similar) to get a fixed-size digest.
2. Encrypt the digest with your private key. That encrypted digest is the signature.
3. Distribute the content, the signature, and your public key.
4. Anyone who wants to verify: hash the content themselves, then decrypt the signature with your public key. If the two hashes match, the signature is valid - meaning it was produced by whoever holds the private key, and the content hasn't changed since.

This is what TLS certificates do: the CA hashes the certificate content and signs the hash with its private key. Your browser verifies by hashing the cert and checking the signature with the CA's public key (which the browser already has in its trust store).

HMAC is simpler than a full digital signature - it uses a shared secret rather than a key pair, so both sides need the secret. It cannot prove to a third party who produced a message (both parties could have). Digital signatures, where only the private key can sign, provide non-repudiation - the signer cannot later deny having signed it.