Designing Kubernetes for stateful workloads

Start with workload behaviour

Stateful workloads introduce requirements that cannot be answered by looking at pod replicas alone. Data durability, write behaviour, latency, recovery time, consistency and failure handling all shape the architecture.

Before choosing a storage platform, capture the application's requirements and how it responds to node, network, storage and zone-level failures.

Treat storage as part of the application

Persistent volumes, storage classes, scheduling rules, topology and application-level replication interact. A storage layer may protect data while the application remains unavailable, or application replicas may exist without providing independent failure domains.

• Document expected data loss and recovery objectives.
• Test volume attachment, detach and rescheduling behaviour.
• Confirm application consistency during interruption.
• Observe capacity, latency, throughput and filesystem health.
• Exercise backup restoration and disaster recovery procedures.

Test failure, not just deployment

A successful deployment confirms that the happy path works. It does not establish that the system will recover safely after node loss, storage errors, capacity exhaustion or an unsuccessful upgrade.

Validate those cases in a controlled environment and record the measured behaviour. Do not treat assumed recovery times as proven service objectives.