Field notes
Follow the decisions behind a trusted result.
Start with a messy event, inspect the decision that changes its path and verify the result. Six field notes so far, each run on small synthetic data you can inspect.
- 01Flagship, runnablePipeline, Modelling, Quality6 min
A daily orders pipeline you can run in DuckDB
Start with a messy delivery and follow each decision to a trusted current order. Inspect validation, replay handling and correction before you run it yourself in your browser.
- 02Companion noteReliability, Pipeline7 min
An idempotent pipeline counts each order once
When delivery retries, the same event can arrive again or disagree with its earlier payload. Trace how the pipeline recognises a safe repeat, rejects a conflict and protects the committed result.
- 03Companion noteModelling, Reliability7 min
Late-arriving data can revise an earlier day
A correction can arrive after an order has changed yesterday's total. See how the pipeline chooses the winning state, removes the old contribution and adds the corrected one.
- 04Pattern noteReliability, Pipeline7 min
Re-run a backfill without doubling the total
Backfill a daily warehouse slice safely: compare append-only SQL with replace-by-slice and watch a replay keep the same total.
- 05Pattern noteQuality, Pipeline7 min
Schema drift should fail loudly, not quietly
Schema drift, run in DuckDB: four column changes go through a permissive loader and a data contract, showing which rows vanish, mis-type or stop.
- 06Pattern noteModelling9 min
dbt snapshots and SCD type 2: what history misses
See a dbt-style snapshot table joined three ways: the current row rewrites history, every version multiplies revenue and an as-of join gets both right.