Systems and deployment
Eleven weeks on keeping a model useful once real traffic reaches it. Serving and latency, an evaluation harness you trust, observability that says why, and what to do when it pages you.
- Length
- 11 weeks
- Time
- 4 h a week
- Enrolled
- 1,286
- Next cohort
- January 2027 cohort
- Written and marked by
- Jonas Reiter Lead instructor, Systems
Four modules, one assignment each
Every module ends with work that is read by the person who wrote it. The assignments build on each other, so the order is fixed.
- 01
Serving and latency
Batching, caching, and the p99 nobody measured. Designing for the slowest request rather than the average one.
- 02
Evaluation harnesses
A harness that runs on every change and fails for the right reasons. Golden sets, drift, and the metric that actually correlates with users.
- 03
Observability in production
Logging what the model saw, not just what it said. Tracing a bad output back to the input that caused it, in minutes.
- 04
Incident response
The runbook for a model that got worse overnight. Rollback, kill switches, and the write-up that stops it happening twice.
Who it is for
Backend and platform engineers who have inherited a model in production, or are about to. You know how to run services; what is new is a component whose behaviour changes when the data does, and whose failures do not throw exceptions.
It assumes the operational basics. If you have not run a service with users, take the systems module of the LLM track first.
How the eleven weeks run
Two taught hours a week, most of them at a terminal. Two hours on your own on the fortnightly assignment, each of which adds a layer to the same service: it is served in week two, evaluated in week four, observed in week six, and broken on purpose in week nine.
The final brief is a partner company’s live system, or your own. You are reviewed by a systems practitioner and a governance instructor, because the second question after “does it work” is always “who is accountable when it does not”.
