Best for
- The pipeline design about to be built — reviewed while changes are cheap
- The existing pipeline that fails weekly, audited for why
- Teaching the team what production-grade means in data engineering
What you give it
- The design (or the existing pipeline's code and its incident history)
What you get back
- Findings per reliability dimension: where re-runs double-count, where failures strand state, where late data silently lies
- Each finding with its failure story: the concrete Tuesday on which this design hurts you, and the fix
- The verdict: ship it, fix-then-ship, or redesign — with the fixes ranked by incident-prevention value
How it works
- Walks the design through the standard disasters: the mid-run crash, the double trigger, the late batch, the restated history, the malformed row, the full backfill — asking what this design does in each.
- Checks idempotency structurally: not 'we retry carefully' but 'the write pattern makes re-runs harmless by construction'.
- Prices each finding by its incident: what pages whom, what numbers lie to whom, how long recovery takes.
- Reviews the recomputability spine: raw retained, transforms deterministic, backfills defined — the properties that turn failures into retries.
Example
You: Review the design for our new revenue aggregation pipeline before we build it.
Result: The review's findings: the append-based load double-counts on any retry (the design's worst flaw — swapped for keyed upserts; idempotency is now structural), the daily window closes at midnight but payments arrive up to 48h late (the silent-lie finding: added watermark handling and a reprocessing window, plus the 'numbers may restate for 48h' note for finance), no quarantine path for malformed source rows (one bad row would have stalled the whole run), and the backfill procedure undefined (designed now, tested once — not invented during the incident). Built to the revised design: four months in production, zero data incidents.
Limits — please read
- A design review catches structural flaws; implementation bugs still need tests (the review says which tests matter most).
- Source-system lies (late, duplicated, restated data) are modelled from what you know of the sources; unknown sources get conservative assumptions, stated.
- The ship/redesign verdict is advice with reasons; the deadline trade-off stays yours.