Redis Explained: Data Structures, Persistence, Clustering and Real Use Cases

What Redis actually is

Redis is an in-memory data-structure server. You do not get rows and tables; you get strings, hashes, lists, sets, sorted sets, bitmaps, HyperLogLogs and streams, addressed by key, mutated with atomic commands, and served from RAM in microseconds.

Calling it "a cache" is the most common under-sell. Caching is the most popular use, but the same server backs rate limiters, leaderboards, session stores, distributed locks, job queues, pub/sub fan-out and geospatial lookups — because each of those is just a data structure plus atomic operations.

Why it is fast (and single-threaded)

• Everything is in memory. No disk seek on the read path.
• Command execution is single-threaded. One command runs at a time, so there are no locks, no race conditions between commands, and no context-switch tax.
• I/O multiplexing. An event loop (epoll/kqueue) handles tens of thousands of connections on that one thread.
• Simple protocol. RESP is trivial to parse; pipelining amortises RTT across many commands.

Modern Redis does use extra threads for I/O reads/writes and for background jobs (unlink, persistence forks), but the command execution core remains one thread. Scale out with more instances/shards, not bigger boxes.

The data structures that matter

-- KEYS[1] = user key, ARGV = now_ms, window_ms, limit
redis.call('ZREMRANGEBYSCORE', KEYS[1], 0, ARGV[1] - ARGV[2])
local used = redis.call('ZCARD', KEYS[1])
if used >= tonumber(ARGV[3]) then return 0 end
redis.call('ZADD', KEYS[1], ARGV[1], ARGV[1] .. ':' .. math.random())
redis.call('PEXPIRE', KEYS[1], ARGV[2])
return 1

Expiry and eviction

A TTL is not a scheduled deletion. Redis removes expired keys two ways: lazily when a key is touched, and via a sampling background job that repeatedly tests random keys with TTLs. So memory frees up slightly after expiry, not exactly on it.

When maxmemory is reached, the eviction policy decides what happens. Choose it deliberately:

Persistence: RDB vs AOF

Production default: enable both. RDB for fast restores and backups, AOF with appendfsync everysec for bounded data loss. And remember the fork: a snapshot on a 30 GB instance can transiently double memory usage if the write rate is high.

Replication, Sentinel and Cluster

• Replication is asynchronous. A primary acks your write before replicas have it, so a failover can lose recent writes. WAIT gives you partial control, not consensus.
• Sentinel monitors and promotes a replica on failure. It gives you availability for a single dataset, not more capacity.
• Cluster shards keys across 16384 hash slots. Multi-key commands only work when keys share a slot — use hash tags like user:{42}:cart.

Production patterns

Cache-aside with stampede protection

const hit = await redis.get(key);
if (hit) return JSON.parse(hit);

// only one caller rebuilds; others briefly serve stale or wait
const gotLock = await redis.set(key + ':lock', id, { NX: true, PX: 5000 });
if (!gotLock) return staleOr(await waitBriefly(key));

const fresh = await db.load(id);
await redis.set(key, JSON.stringify(fresh), { EX: 300 + jitter() });
return fresh;

Add jitter to every TTL. Identical TTLs set during a deploy expire together and hand your database a synchronised thundering herd.

Distributed locks — carefully

• Always SET key token NX PX ttl; never SETNX then EXPIRE (crash between the two = permanent lock).
• Release with a Lua compare-and-delete on your token, so you never unlock someone else's lease.
• Redis locks are best-effort. For correctness-critical mutual exclusion, use a fencing token checked by the resource itself.

Queues

BLPOP lists are fine for fire-and-forget. When you need at-least-once delivery, retries and consumer groups, use Streams with XREADGROUP, XACK and a periodic XAUTOCLAIM for stuck messages.

Pitfalls that cause outages

• KEYS, FLUSHALL, big SMEMBERS/HGETALL — O(N) commands on the shared thread. Use SCAN.
• Big keys. A 500 MB list is slow to read, slow to replicate, slow to delete. Use UNLINK for async deletion and shard the key.
• No maxmemory set. The OOM killer becomes your eviction policy.
• Treating pub/sub as durable. Pub/sub is fire-and-forget; a disconnected subscriber loses messages. Streams are the durable option.
• Unbounded connections. Pool them; connection churn costs more than the commands.
• Cache without a fallback path. If Redis being down takes your app down, it is not a cache, it is a database with no persistence.

Interview framing

"We put Redis in front of Postgres as a cache-aside layer with jittered TTLs and single-flight rebuilds so an expiry burst cannot stampede the database. Because command execution is single-threaded, we ban O(N) commands and big keys, and we run separate instances for cache (allkeys-lfu) and for queues (noeviction, AOF everysec). Rate limiting is a sorted-set sliding window in a Lua script so it is atomic. HA is Cluster with replicas; we assume async replication can lose the last few writes, so nothing that must survive lives only in Redis."