Look beneath the name
Technology feels magical when its mechanism is hidden. A database “finds” a row, a server “trusts” a token, and HTTPS “secures” a request. Those labels are useful shorthand—but engineering begins when you can trace the data, identify the state, and name the failure boundary.
1. Database index: change the lookup path
With no usable index, the database may inspect every candidate row and evaluate the predicate. An index stores selected column values in a separate search-friendly structure, usually alongside row pointers. The engine navigates that smaller structure, finds matching identifiers, and fetches only the required table pages.
What happens on a real query
- The optimizer estimates selectivity and chooses a table scan or index scan.
- The engine traverses index pages from root to leaf.
- A leaf entry yields the row locator—or all requested columns when the index covers the query.
- Each insert, update and delete must also maintain relevant indexes.
2. JWT: verify claims without a session lookup
A JSON Web Token packages claims into a signed envelope. The client sends the token; the server recomputes or verifies its signature with a trusted key, then validates claims such as issuer, audience and expiry. No central session lookup is required for every request—but revocation becomes harder because a valid token can live until it expires.
base64url(header) + "." + base64url(payload) + "." + signature
payload = { "sub": "user-42", "aud": "orders-api", "exp": 1790000000 }Never trust a token merely because it can be decoded. Pin the expected algorithm, validate issuer and audience, enforce expiry, rotate keys safely, and keep access tokens short-lived. Do not place secrets in the payload.
3. CDN: move reusable bytes closer
A content delivery network places edge caches near users. A cache key—often host, path, query and selected headers—identifies an object. On a hit, the edge returns its copy. On a miss, it requests the origin, applies cache rules, stores the response, and serves it.
The hard parts are freshness and identity
TTL controls how long an object stays fresh. Purges remove known stale objects; versioned filenames avoid invalidation altogether. A wrong cache key can mix user-specific responses, while no request coalescing can let thousands of simultaneous misses stampede the origin.
4. Connection pool: protect a finite database
Opening a database connection involves network setup, authentication and server memory. More importantly, the database can execute only finite work. A pool keeps a small set of established connections and lends them to requests. Once all are busy, callers wait rather than opening sockets without limit.
Size the pool for database capacity, not incoming request count. Always release connections, set acquisition and query timeouts, keep transactions short, and avoid immediate retries after timeout. Otherwise waiting becomes timeout, timeout becomes retry, and retry becomes a cascading failure.
5. Garbage collection: reclaim unreachable memory
Managed runtimes track objects reachable from roots such as active stack frames and static fields. Objects that cannot be reached are eligible for reclamation. Generational collectors exploit the observation that most objects die young; compacting collectors also move survivors to reduce fragmentation.
High allocation rate creates frequent collection work. A large live set makes tracing and compaction more expensive. Memory leaks still happen when code accidentally retains references: the collector correctly sees the objects as live. Useful signals include allocation rate, heap occupancy after collection, pause percent and promotion rate.
6. Rate limiter: meter admission before overload
A token bucket accumulates tokens at a fixed rate up to a capacity. Each request consumes one. Capacity permits a controlled burst; refill rate controls sustained traffic. With no token, the service rejects the request—usually with HTTP 429—or deliberately delays it.
A distributed limiter needs atomic shared state or carefully partitioned local limits. Decide what is being protected and who is being measured. Per-IP limits can punish users behind shared networks; per-user limits need authentication; per-tenant limits protect fairness. Return retry guidance and degrade safely if the limiter's store is unavailable.
7. Message queue: separate accepting work from doing work
A queue lets a producer durably hand off work without requiring the consumer to be available at that instant. The queue absorbs bursts, consumers pull at their own pace, and acknowledgements indicate completion. This improves temporal decoupling, but it does not make failure disappear.
- At-least-once delivery: acknowledge after success; duplicates are possible.
- Visibility timeout: unacknowledged work becomes available again.
- Dead-letter queue: poison messages stop blocking healthy work after bounded retries.
- Backpressure: queue age and depth reveal when consumers cannot keep up.
8. B+ tree: the shape behind many indexes
A B+ tree stores separator keys in internal pages and sorted key-to-row entries in leaf pages. Each page has high fan-out because it holds many keys. That keeps the tree shallow: a few page reads can narrow millions of records to one leaf.
Unlike a classroom binary tree, nodes are sized around storage pages and can have hundreds of children. Inserts may split full pages and propagate a separator upward. Deletes may merge or redistribute pages. This write amplification is the cost paid for predictable point and range reads.
| Operation | Path | Important cost |
|---|---|---|
| Exact lookup | root → internal pages → leaf | tree height + row fetch |
| Range scan | find first leaf → follow sibling leaves | number of matching leaf pages |
| Insert | find leaf → write → possibly split | page writes and rebalancing |
| Covering read | answer from leaf entry | avoids table-row fetch |
9. DNS: translate a name through delegated authority
DNS maps a domain name to records such as IP addresses. Resolution usually stops at a cache. On a miss, a recursive resolver walks the hierarchy: root servers point to the top-level-domain servers, which point to authoritative name servers, which return the requested record and its TTL.
Changes are not globally instant because independent caches retain earlier answers until TTL expiry. Multiple A/AAAA records can distribute clients, CNAME records create aliases, and DNSSEC can authenticate DNS data. DNS tells the client where to connect; TCP or QUIC and TLS happen afterward.
10. TLS: establish trust, then protect bytes
During a modern TLS handshake, client and server negotiate capabilities, exchange ephemeral key material, and derive shared session keys. The server presents a certificate whose signature chain connects its hostname to a trusted certificate authority. Finished messages prove both parties derived the same secret and observed the same handshake.
HTTPS is HTTP carried through this protected channel. TLS provides confidentiality, integrity and server authentication; it does not prove the application is honest or free of vulnerabilities. Certificate expiry, hostname mismatch, clock skew and incomplete chains are common operational failures.
11. Load balancer: choose an eligible backend
A load balancer accepts client connections and selects a backend using routing rules and current state. Round robin is only one policy. Real decisions can include health, active connections, response latency, weights, geography, session affinity and endpoint readiness.
Health checks need care: a shallow check may mark an instance healthy while its database dependency is dead; a deep check can remove every instance during one shared dependency outage. Graceful draining lets in-flight work finish during deployment. Load balancers themselves need redundancy and overload controls.
Connect the mechanisms
These mechanisms rarely operate alone. One page load can resolve DNS, negotiate TLS, reach a CDN, pass through a load balancer, verify a JWT, acquire a pooled connection, navigate a B+ tree, enqueue background work, and consume heap that a collector later reclaims.
| Mechanism | State it manages | Bounded resource | Failure signal |
|---|---|---|---|
| Index / B+ tree | sorted keys and row locators | pages, write I/O | scan chosen, page splits, cache misses |
| JWT | signed claims | token lifetime | invalid signature or claims |
| CDN | cached objects | edge storage, origin capacity | miss ratio, stale data |
| Connection pool | leased connections | pool and DB capacity | wait time, acquisition timeout |
| Garbage collector | reachable object graph | heap and CPU | pause time, allocation pressure |
| Rate limiter | tokens or counters | admission budget | 429 rate |
| Message queue | durable pending work | retention and consumer throughput | oldest-message age |
| DNS | cached delegated records | TTL and resolver capacity | lookup latency, stale answer |
| TLS | identity and session keys | handshake CPU and validity | certificate/handshake errors |
| Load balancer | backend eligibility | connections and backend capacity | health failures, queueing |
Continue with Tech You Know But Don't Really Understand, then practise applying these mechanisms in the free system quality fundamentals guide.