Docker & Kubernetes in Production: What Actually Works

Container orchestration with Kubernetes has moved from cutting-edge to table stakes over the past several years, but the gap between running a demo cluster and operating one reliably in production remains wide. Research published on arXiv by researchers studying Kubernetes security and deployment patterns consistently identifies the same root causes of production failures: misconfigured resource limits, inadequate observability, poor secrets management, and a gap between what works in development and what holds up under real traffic. This guide focuses on the patterns that engineering teams have found to actually matter — not theoretical best practices, but the specific decisions that separate clusters that quietly break from clusters that self-heal.

Why Container Orchestration Complexity Is Real

Docker made containers accessible. Kubernetes made orchestrating thousands of containers possible. But with that power comes significant configuration surface area. A 2020 arXiv study (Gao et al., "XI Commandments of Kubernetes Security") systematically catalogued Kubernetes security misconfigurations and found that the majority of production incidents stem from a small set of recurring configuration mistakes: over-permissive RBAC policies, containers running as root unnecessarily, missing resource limits, and lack of network policies. These aren't exotic vulnerabilities — they're the default state of a cluster that has been deployed but not hardened. The same issues that create security exposure also create operational instability, because a container that can consume unbounded memory will eventually take down its node.

Getting Resource Limits Right (It's Not Optional)

Resource requests and limits are the single most common source of production instability in Kubernetes clusters, and they are almost always underconfigured in teams new to orchestration. Every container should have explicit CPU and memory requests (what the scheduler uses to place the pod) and limits (the ceiling Kubernetes enforces). Without limits, a runaway container can exhaust node resources and trigger an OOMKill cascade. Without requests, the scheduler places pods based on guesswork, leading to hot nodes and idle nodes simultaneously.

The practical approach is to run your application under realistic load in a staging environment, observe actual usage with kubectl top pods or a metrics solution, and set requests at roughly the median observed usage and limits at 1.5–2x the peak. Kubernetes Vertical Pod Autoscaler (VPA) can recommend values based on observed usage history, which is a useful starting point. Horizontal Pod Autoscaler (HPA) then handles scaling replicas up or down based on CPU, memory, or custom metrics — but HPA only works correctly when resource requests are set, because it scales based on utilization percentage of the requested amount.

Image: Logging as a Service Architectural Model — Trinacrialucente (CC BY-SA 3.0), via Wikimedia Commons

Deployment Strategies That Minimize Downtime

Kubernetes supports several deployment strategies natively, and choosing the right one for each service matters more than most teams realize until they experience an incident during an upgrade.

Rolling updates (the default) gradually replace old pods with new ones. This works well for stateless services but requires that both the old and new versions can run simultaneously — a compatibility requirement that creates work during schema migrations or protocol changes. Setting maxUnavailable: 0 and maxSurge: 1 ensures no downtime by always bringing up a new pod before taking down an old one, at the cost of briefly running extra capacity.

Blue/green deployments maintain two identical environments and switch traffic via a load balancer change. This gives instant rollback (just point traffic back) but doubles resource cost during the switch window. It is the right choice for high-stakes releases where the cost of a bad deployment far exceeds the cost of temporary extra infrastructure.

Canary deployments, where a small percentage of traffic is sent to the new version before a full rollout, are the most operationally careful approach and work well when combined with robust metrics collection. A canary receiving 5% of traffic that shows elevated error rates can be rolled back before most users are affected. Tools like Argo Rollouts and Flagger automate the progressive traffic shift and rollback decision based on defined success metrics.

Networking, Services, and Ingress: The Hidden Complexity

Kubernetes networking has several abstraction layers that are easy to misunderstand. A Service exposes a set of pods under a stable DNS name and IP, handling load balancing across pod replicas. An Ingress (or Gateway API, the more recent standard) routes external HTTP/HTTPS traffic to Services based on hostname and path rules, enabling a single load balancer to front many applications. The most common mistake is conflating these two — and the consequence is either internal services accidentally exposed externally, or incorrect routing that is hard to debug.

Network policies are the Kubernetes equivalent of firewall rules between pods. By default, pods can talk to any other pod in the cluster — a permissive posture that is convenient for development and problematic in production. Implementing default-deny network policies and then explicitly allowing required traffic paths limits the blast radius of any compromised container. This maps directly to what the arXiv Kubernetes security research identified as one of the most impactful controls available without requiring specialized tooling.

Observability: Logs, Metrics, and Traces Are Not Optional

Kubernetes itself is opinionated about statelessness and scheduling but says nothing about observability — that is entirely the operator's responsibility. The minimum viable observability stack for a production cluster consists of three components: a metrics system (Prometheus is the de facto standard, with Grafana for dashboards), a centralized logging pipeline (Loki, Elasticsearch/OpenSearch, or a managed service), and ideally distributed tracing for services with complex inter-service communication.

The logging pipeline shown in the architectural diagram above illustrates a common pattern: agents on each node (often Fluentd or Fluent Bit DaemonSets) collect container logs, forward them to a central aggregation and retention layer, and make them queryable alongside alerting rules. Without this pipeline, debugging production issues means manually running kubectl logs against individual pods — and if the pod has already been rescheduled, those logs are gone.

Readiness and liveness probes belong in this observability discussion because they are Kubernetes's mechanism for acting on health signals. A readiness probe tells Kubernetes when a pod is ready to receive traffic — this is how rolling deployments prevent traffic from going to a pod that is still initializing. A liveness probe tells Kubernetes when to restart a container that has entered a broken state. Misconfigured probes — too aggressive, or targeting the wrong endpoint — are a leading cause of unnecessary pod restarts and cascading failures.

Image: 2nodeHAcluster — Georgewilliamherbert (CC BY-SA 2.5), via Wikimedia Commons

Secrets Management and Security Hardening

Kubernetes Secrets are base64-encoded by default, which is not encryption — it is encoding. Stored unencrypted in etcd and accessible to anyone with cluster access, they are a frequent source of credential exposure in misconfigured clusters. Production clusters should enable encryption at rest for etcd, or better, use an external secrets management solution like HashiCorp Vault, AWS Secrets Manager, or the Kubernetes External Secrets Operator pattern that pulls secrets from a trusted external store at pod startup.

RBAC (Role-Based Access Control) should follow least-privilege principles: every service account should have only the permissions it needs to function, and humans should access the cluster via short-lived credentials rather than static kubeconfig files. The arXiv study on Kubernetes security found that over-permissive cluster-admin bindings were among the most common critical findings in production clusters that had not been systematically hardened.

Frequently Asked Questions

When does Kubernetes become worth the operational complexity?

Kubernetes delivers the most clear value when you are running more than a handful of services that need independent scaling, when you need consistent deployment behavior across environments, or when your team is large enough that manual deployment coordination becomes a bottleneck. For a single small service with stable traffic, a simpler platform (a PaaS, or a single VM with Docker Compose) is often a better fit. The operational investment in Kubernetes pays off when the alternative — manual or script-based coordination — becomes more complex than the platform itself.

Should we self-manage Kubernetes or use a managed service?

For most teams, managed Kubernetes (EKS, GKE, AKS, or equivalent) is the right choice. Managing the control plane — etcd, API server, scheduler, controller manager — is nontrivial, requires expertise to do safely, and is rarely where engineering value is created. Managed services handle control plane upgrades, availability, and backups. The trade-off is some loss of fine-grained control and a dependency on the cloud provider. Self-managed Kubernetes makes sense for on-premises requirements, air-gapped environments, or organizations with specific regulatory constraints that prevent use of managed services.

How do we handle stateful workloads in Kubernetes?

Kubernetes was designed for stateless workloads and StatefulSets were added later to support ordered deployment and stable network identities for databases and similar services. Running production databases in Kubernetes is possible — PostgreSQL, MySQL, and Kafka all have mature Kubernetes operators — but requires careful attention to persistent volume configuration, backup procedures, and the interaction between pod scheduling and storage locality. Many teams find it simpler to run stateful infrastructure (databases, object storage) outside the cluster on managed services, and run only stateless application logic in Kubernetes. This separation of concerns reduces operational risk at the cost of slightly more complex networking between layers.

Bottom Line: Kubernetes is a powerful platform that rewards careful configuration and punishes carelessness in specific, predictable ways. We recommend teams moving to production Kubernetes treat resource limits, health probes, and centralized logging as non-negotiable from day one — not as refinements to add later. The arXiv research on Kubernetes security and deployment practices consistently shows that the gap between a cluster that works in demos and one that operates reliably under real conditions is almost always a configuration gap, not a capability gap. Start with strong defaults, instrument everything, and build operational confidence incrementally.

Disclaimer: This article is for informational purposes only. Technology landscapes change rapidly; verify information with official sources before making technical decisions.