The delivery system between your laptop and a user
A production release is not a file transfer from a developer laptop to a server. It is a chain of evidence and custody. Git identifies the change. CI validates it. A build creates runnable bytes. A registry preserves those exact bytes. A deployment system places them on machines. Health checks decide whether they are eligible for traffic. Observability tells operators whether the release is actually succeeding.
1. It starts on your laptop
You write code in an IDE, start the application locally, exercise the changed path, and run automated tests. This loop is optimized for fast feedback. It can catch syntax errors, broken logic and integration mistakes before shared infrastructure spends time on them.
Local success is still incomplete evidence. Your machine may have undeclared packages, cached files, different environment variables, more memory, or a database state nobody else has. A reproducible project pins dependency versions, documents required services, and makes its checks runnable by both a developer and CI.
2. Git records a change, not merely a copy
A commit is a content-addressed snapshot with parent history and metadata. Pushing sends missing Git objects to a remote repository such as GitHub, GitLab or Bitbucket. The remote becomes the collaboration point: reviews attach to a commit, branch protections control merges, and the delivery system can build one unambiguous revision.
git add . git commit -m "Add payment API" git push origin feature/payment-api
- Commit SHA: identifies source history, such as
a84c2e7. - Branch: a movable name used for collaboration.
- Tag: a human-friendly marker often used for releases.
- Pull request: a review and policy boundary before merge.
3. A push triggers continuous integration
A webhook or repository event starts a pipeline on a clean runner. The runner checks out the exact commit, restores only controlled caches, installs locked dependencies, and executes validation. Independent checks may run in parallel, but publishing is gated on all required checks passing.
| Check | Question answered | Typical failure |
|---|---|---|
| Lint / static analysis | Does the code satisfy structural rules? | unsafe pattern or type error |
| Unit tests | Do small behaviors still hold? | regression in logic |
| Integration tests | Do components work together? | schema or contract mismatch |
| Dependency scan | Does the dependency graph contain known risk? | vulnerable package version |
| Build | Can this revision produce a runnable artifact? | compiler or bundler failure |
A green pipeline does not prove the application has no bugs. It means a predefined set of evidence passed. The value of CI depends on deterministic tests, realistic contracts, meaningful policies and fast feedback.
4. Source code becomes a deployable artifact
Production needs runnable output rather than an editor workspace. Java commonly produces a JAR; Go compiles a binary; frontend JavaScript becomes optimized bundles; Python may produce a wheel or package the application source with a locked environment. The build should be repeatable: the same source and declared inputs should produce equivalent output.
5. Docker packages a consistent runtime
A Docker image is an ordered set of immutable filesystem layers plus metadata such as the entrypoint. A Dockerfile describes how to produce it. At runtime, a container adds a thin writable layer and uses operating-system isolation. It is not a miniature virtual machine; containers on one host share its kernel.
FROM node:22-alpine AS build WORKDIR /app COPY package.json bun.lock ./ RUN npm ci COPY . . RUN npm run build FROM node:22-alpine WORKDIR /app COPY --from=build /app/dist ./dist CMD ["node", "dist/server.js"]
“Build once, run the same image” removes an important source of drift. Staging and production should pull the same digest; only runtime configuration changes. Keep images small, run as a non-root user, scan base layers, and never store secrets in image layers.
6. A registry preserves the exact image
CI authenticates to a container registry and pushes image layers plus a manifest. Registries deduplicate existing layers and identify the final manifest by a cryptographic digest. A tag such as v42 is a movable label; a digest such as sha256:7ab… identifies exact content.
Amazon ECR, Google Artifact Registry, Azure Container Registry and Docker Hub provide this role. Production nodes pull from the registry rather than rebuilding source. Retention rules, access controls, vulnerability reports and image signatures become part of the release supply chain.
7. The deployment system reconciles desired state
A deployment declares a desired image and replica count. In Kubernetes, a Deployment manages a ReplicaSet, which maintains Pods, which start containers from the selected image. Controllers continually compare desired state with observed state. If three replicas are required and only two exist, the controller creates another.
This is reconciliation—not one imperative “start three servers” command. During a rolling update, the controller gradually adds v42 pods and removes v41 pods while respecting availability and surge limits. Pinning the image digest ensures every new pod runs the artifact CI validated.
For a deeper look at controllers, pods and container isolation, continue with Tech You Know But Don't Really Understand.
8. Health checks decide restart and traffic eligibility
A running process is not automatically ready. It may still be loading configuration, warming caches or applying internal initialization. Kubernetes uses distinct probes because “started,” “ready for traffic,” and “still alive” are different claims.
- Startup probe: protects slow startup from premature liveness restarts.
- Readiness probe: adds or removes a pod from service endpoints.
- Liveness probe: asks whether restarting the container may recover it.
Do not make every shared dependency a liveness requirement. If the database has a brief outage and every pod fails liveness, the platform can restart the entire fleet while the database is already struggling. Probe thresholds, timeouts and semantics are production design decisions.
9. A user request crosses several routing layers
The user sees one hostname. DNS resolves it to an entry point. A cloud load balancer accepts the connection and selects a healthy gateway or ingress. Routing rules match the host and path. A Kubernetes Service provides a stable virtual endpoint over changing pod addresses, then forwards to a ready pod.
The gateway may terminate TLS, authenticate, rate-limit, validate and route. The load balancer distributes connections and removes unhealthy targets. The Service abstracts pod identity. These names are sometimes combined by a platform, but the responsibilities still exist. Explore the API gateway pattern, Nginx, or troubleshoot the path with The Request Never Reached the Server.
10. The application joins an ecosystem
Once the request reaches the application, it may read Redis, query a database, publish to Kafka, fetch an object, or call another company's API. Each network call adds latency, capacity limits, authentication, timeouts and a failure mode. “The pod is up” therefore does not mean the full user operation will succeed.
Use explicit timeouts, bounded retries with backoff and jitter, connection pools, circuit breakers, idempotency and fallbacks where the product allows them. Trace context should cross these calls so one slow dependency does not become an unexplained slow endpoint.
11. Deployment begins the feedback loop
Operators need to compare what the release intended with what users experience. Metrics show aggregate behavior, logs preserve discrete events, and distributed traces connect work across services. Alerts convert selected signals into action—but only when thresholds reflect user impact and provide a useful response.
| Signal | Best at answering | Example |
|---|---|---|
| Metrics | Is behavior changing across the fleet? | p95 latency, request rate, error ratio |
| Logs | What happened at a specific moment? | payment rejected with order ID |
| Traces | Where did one request spend time? | 820 ms waiting on inventory |
| Alerts | Does someone or something need to act? | error budget burn is too high |
Include deployment version, commit SHA and region in telemetry. Then a spike can be correlated with v42 instead of requiring guesswork.
12. Scaling changes capacity, not correctness
When demand rises from 100 to 10,000 requests per second, a horizontal autoscaler may increase replicas based on CPU, memory, request concurrency or a queue metric. The load balancer spreads new traffic across ready instances. Caches may reduce repeated work, and queues can absorb temporary bursts.
More pods help only when another bottleneck does not dominate. If every pod opens twenty database connections, scaling from three to twenty pods can increase connections from sixty to four hundred and overwhelm the database. Autoscaling needs minimums, maximums, stabilization windows, startup time, downstream capacity and a scale-down policy.
13. When something breaks, stop or reverse safely
Versioning and gradual delivery make change reversible. A rolling release, canary or blue-green deployment can expose v42 to a controlled amount of traffic. Health and business signals determine whether to continue, pause or restore v41.
Rollback does not automatically reverse every side effect. Database migrations must remain compatible with both versions during the transition. Messages already published, emails sent and external charges made require idempotency or compensating action. A strong release plan defines rollback before deployment.
| Failure | Detection | Typical response |
|---|---|---|
| Container crash loop | restart count + liveness | stop rollout; inspect logs/config |
| Readiness never passes | zero ready endpoints | keep old version serving |
| Error rate spike | version-tagged metrics | automated or manual rollback |
| Database unavailable | dependency errors + saturation | shed load; avoid retry storm |
| Bad schema migration | query failures | compatible roll-forward or planned reversal |
The complete journey
The path is local code → Git commit → CI evidence → build artifact → Docker image → container registry → deployment reconciliation → health eligibility → production routing → users. Metrics, logs and traces then report what happened. Scaling adds capacity; rollback restores a safer version.
Use the free system quality fundamentals guide to connect this flow to latency, availability, consistency, durability, scalability and throughput. For the network mechanics inside the traffic path, read Behind the Scenes: Tech Edition.