Production Engineering — Boundaries, Performance, Recovery, and Delivery in 14 Steps

A feature is not finished when it appears on a screen. It is finished when retries, partial failure, data changes, delivery, and incident recovery keep the same meaning. This course applies production principles to example service boundaries and turns them into evidence shared by product, design, engineering, users, and operations.

What you will be able to do

• turn requirements into success, in-progress, partial-failure, and failure states
• separate HTTP retry keys from request correlation IDs
• design tables, indexes, and pools from real queries and time budgets
• define Timeout, Unknown, Retry, and Reconcile boundaries for providers, Kafka, AI, and notifications
• verify authentication, authorization, audit, and upload integrity together
• make RAG source, search copies, permissions, and quality converge
• operate expand/contract migrations, SHA images, readiness, smoke, and rollback
• close completion with bounded RED metrics, tests, and SLO evidence
• connect partial failure, recovery, and static-client boundaries into cross-service evidence
• operate protobuf gRPC contracts and WebRTC signaling/media boundaries behind a cost guard

The flow

Connecting a feature to completion evidence

1Requirements and contracts

Define user state, HTTP failures, and idempotency first.

2Data and external boundaries

Make database performance, upstream failures, authorization, and files recoverable.

3Cost and change

Verify AI/RAG cost gates and migration stop-and-restore conditions.

4Deployment and evidence

Combine release, Compose health, observability, and public smoke into one completion decision.

Steps

1. Turn requirements into states and evidence
2. Fix HTTP retries and request correlation
3. Design tables, indexes, and pools from query budgets
4. Handle Timeout, Unknown, Retry, and Reconcile
5. Make authentication, authorization, and audit one boundary
6. Validate uploads and publish them atomically
7. Bind AI providers and RAG to permission and quality
8. Change schemas with expand/contract and idempotent backfills
9. Bind a verified commit to a release and image
10. Operate Compose, readiness, reverse proxy, and rollback
11. Follow one request with RED, correlation, and SLOs
12. Verify a change with five perspectives and deployment smoke
13. Close cross-service partial failure, recovery, and platform boundaries
14. Introduce gRPC and WebRTC behind an explicit cost guard

Prerequisites: experience with architecture-patterns, data-engineering-postgres, or quality-and-testing, plus basic PostgreSQL and HTTP.