CHAPTER 15 STUDIO

Orchestration is just materializing a graph, in the right order.

Cron runs scripts on a clock. An orchestrator runs an asset graph: it knows what depends on what, builds in dependency order, skips work already done, and tells you exactly what breaks when one node fails. These three labs make each of those ideas visible. They are the browser face of a 150-line pure-Python orchestrator built from scratch in Chapter 15.

Open the labs Open chapter folder

9 notebooks in the chapter

3 interactive labs

~150 lines of pure Python

Why a graph, not a schedule

Dependency order is the whole game.

A schedule says "run at 2am". A graph says "build positions after price_history, which comes after market_data". The orchestrator topologically sorts the graph and materializes in that order, so nothing is ever built on a missing input. This is the asset model Dagster is built on, and the same DAG Chapter 12 used for lineage.

Why blast radius matters

One failure, a measured cost.

When an asset fails, its downstream descendants cannot be trusted. The set of those descendants is its blast radius, the exact descriptor Chapter 12 built to rank lineage risk. It is also the number that should decide your on-call priority: fail the source and everything breaks; fail a leaf and nothing downstream does.

LAB 1: ASSET GRAPH MATERIALIZER

Click Materialize and watch the wave sweep the graph in dependency order.

This is the trading platform's data flow as an asset graph. Hit Materialize: each node goes idle, then running, then materialized, but only once all of its upstreams are done. A node cannot jump ahead of its dependencies. That ordering, computed by a topological sort, is the core of every orchestrator.

Click MATERIALIZE to run the graph. RESET to re-arm. ready

Step speed

650 ms per node

Materialized 0 / 7

Currently building --

idle running materialized

Built from notebook 15.1. The order is a Kahn topological sort: start from sources, peel inward, never run a node before its upstreams.

LAB 2: BACKFILL AND PARTITIONS

Fill the missing partitions. Then run it again and watch it do nothing.

Each row is an asset; each column is a day. Filled cells are already materialized. Pick a date range and click Backfill: only the missing cells fill, in dependency order, top to bottom per day. Run the same backfill a second time and every cell is skipped. That no-op is idempotency, the property cron never had.

Set the range, then click BACKFILL. RESET randomizes the starting state. ready

Backfill from day

day 1

Backfill to day

day 12

Materialized this run 0

Skipped (already done) 0

Coverage 0%

Built from notebook 15.2. A materialization log records every (asset, day) pair, so a rerun over a complete range is a guaranteed no-op.

LAB 3: FAILURE BLAST RADIUS

Fail a node. Watch everything downstream go dark.

Same graph as Lab 1. Click any node to fail it. It turns red; its downstream descendants turn grey, because an orchestrator will not build an asset on a failed input. The panel reports the blast radius: the count and names of everything skipped. Toggle retries to see a transient failure absorbed before it propagates.

Click a node to fail it. RESET to recover the graph. click a node

Blast radius 0

click a node to fail itidle

ok failed skipped

Built from notebook 15.5. The blast radius equals the graph descendants of the failed node, the Chapter 12 lineage-risk descriptor, computed live.

CHAPTER 15 NOTEBOOK PATH

Each lab has notebooks behind it.

The studio is the surface. The chapter builds the orchestrator from scratch, points it at the repo's own dbt project and trading platform, maps it to Dagster, and names the failure modes.

15.0 - 15.1

From cron to a tiny asset graph

The four failures of cron. Then `AssetGraph`, topological materialization, and cycle detection in pure Python.

15.2 - 15.3

Partitions, backfills, sensors

Idempotent backfills over a date range. A polling sensor and a freshness policy on a simulated clock.

15.4

The repo's dbt project as a graph

Parse the real `dbt/dbt_dq` models for ref() edges and orchestrate them. Lineage and orchestration, the same DAG.

15.5

Failure, retries, blast radius

Skip-on-failure, a retry policy, and the Chapter 12 blast-radius descriptor applied to a live run.

15.6 - 15.7

Dagster bridge and the capstone

Every toy concept mapped to the real Dagster API. Then the trading platform's data, orchestrated by day.

15.8

When the schedule lies

Silent partial failure, sensor storms, backfill thundering herd, and retries masking a data bug. Read before you ship.