loc bengaluru, ist | local --:-- srijanshukla18@gmail.com →
[post]/you-a-junior-infra-techie/encryption-compression-encoding-hashing-checksums-and-error-correction

encryption, compression, encoding, hashing, checksums, and error correction

/ 5 min read· infra

Encryption, compression, encoding, hashing, checksums and error correction, explained as tools in a toolbox for juniors. What each one is for and the mix-ups to avoid.

I’ve found it a lil tough to translate my mental model about these concepts into words for juniors. I can explain it to them fine, but I’m making up examples on the fly when I do that. So writing this one down properly.

Juniors mix up encryption, encoding, hashing, compression, checksums, and error correction all the time because all of them take some input bytes and turn them into different-looking output bytes.

The easiest way to keep them straight is by asking: what problem are you actually trying to solve?

1. Encryption = keeping secrets (confidentiality)

You want to scramble data so nobody can read it without a secret key (e.g. AES-256, RSA, ChaCha20).

  • Reversible? Yes, if and only if you have the decryption key.
  • Two types:
    • Symmetric (AES): Same key encrypts and decrypts. Fast, used for encrypting big files, disks, or traffic after a handshake.
    • Asymmetric (RSA, Ed25519): Public key encrypts, private key decrypts. Used in TLS handshakes and SSH keys.
  • Use case: HTTPS traffic on the wire, encrypted EBS volumes, secrets in a vault.

2. Encoding = changing format for compatibility (NOT security!)

Encoding converts bytes from one representation to another so a system can transport or display it safely (e.g. Base64, URL encoding %20, UTF-8, hex).

  • Reversible? Always, by anyone, with zero secret key.
  • Use case: Putting binary data (like a small image or raw bytes) inside a JSON field or email attachment using Base64.
  • Classic junior mistake: Thinking base64 is encryption. Anyone can run echo "..." | base64 -d and read your Kubernetes Secret in 1 second. Base64 hides nothing!

3. Hashing = one-way fingerprint

Takes any size input (a 10-char password or a 10 GB ISO file) and turns it into a fixed-length string (e.g. SHA-256, bcrypt, Argon2).

  • Reversible? No. It is strictly one-way. You cannot “decrypt” a hash back into the original input.
  • Property: Change even 1 bit of the input file and the output hash changes completely.
  • Use case:
    • Storing user passwords in a DB (hash the password with Argon2/bcrypt; when the user logs in, hash what they typed and compare the two hashes).
    • Verifying a downloaded binary wasn’t tampered with (sha256sum).

4. Compression = shrinking byte size

Removes redundancy so the data takes up less disk space or network bandwidth (e.g. zstd, gzip, lz4, JPEG, MP4).

  • Two types:
    • Lossless (zstd, gzip, PNG): Uncompresses back to the exact original bytes bit-for-bit. Used for code, logs, DB backups, JSON API responses.
    • Lossy (JPEG, MP3, H.264): Throws away details humans barely notice so files get 10x smaller. You can’t get the original raw bytes back.

5. Checksums & Error Correction = catching/fixing accidental corruption

Network cables drop bits and hard drives flip bits. How do you know if a packet got corrupted in transit?

  • Checksums (CRC32, TCP checksum): A tiny fast math check tacked onto the end of a frame/file. It’s not cryptographically secure against a hacker (use SHA-256 for that), it’s just super fast at catching accidental bit flips on a noisy network or disk.
  • Error Correction (ECC RAM, RAID parity, Reed-Solomon in QR codes): Adds extra parity/redundancy bytes so that if a few bits get corrupted (or a whole disk dies in RAID, or 20% of a QR code gets smudged), you can reconstruct the missing data automatically without re-downloading it.

Quick cheat sheet

ToolGoalReversible?Needs a secret key?Examples
EncryptionHide data from attackersYesYesAES, RSA, TLS
EncodingFormat bytes safely for transportYesNoBase64, UTF-8, Hex
HashingOne-way unique fingerprintNoNo (unless HMAC)SHA-256, bcrypt
CompressionMake data smallerYes (lossless) / Approx (lossy)Nozstd, gzip, JPEG
Checksum / ECCCatch or repair accidental bit flipsDetects / repairs corruptionNoCRC32, RAID, ECC

Just remember the two big rules:

  1. Base64 is encoding, not encryption.
  2. SHA-256 / bcrypt is hashing, not encryption (you never “decrypt” a password hash).