You are a Data Engineer at Rootsmarkt, a grocery chain with twelve stores.

Every night, a supplier platform called SupplyHub delivers the day's sales file. A pipeline picks it up, cleans it, aggregates it, and loads daily_revenue. At 07:00 Finance opens a dashboard built on that table.

On 2026-03-01 the dashboard showed €0.00 for every store.

The pipeline log was entirely green.

Over the next four hours you will rebuild that pipeline twice — once in Apache Airflow, once in Dagster — and work out what the system should have known.

Do this days before the course, not on the morning. If something is going to go wrong, it should go wrong while there is time to fix it. Budget 30 minutes, most of it waiting for a Docker image.

Six steps, in this order. Each one tells you how to know it worked.

Windows: run everything in PowerShell — not in WSL, not in Git Bash. The commands below are identical on both platforms; they are program invocations, not shell syntax. WINDOWS-SETUP.md has the full walkthrough.

Run the setup commands from the repository directory:

cd orchestration-rootsacademy-2026

You need three things: Docker Desktop, uv, and the Astro CLI.

macOS

brew install --cask docker          # then OPEN it, and leave it running
brew install uv

Then the Astro CLI. Download it directly — the version is pinned for this course, and the download is the only way to get exactly it:

curl -sSLo /tmp/astro.tgz https://github.com/astronomer/astro-cli/releases/download/v1.42.1/astro_1.42.1_darwin_arm64.tar.gz
shasum -a 256 /tmp/astro.tgz
tar -xzf /tmp/astro.tgz -C /tmp astro
sudo mv /tmp/astro /usr/local/bin/astro

The shasum line must print:

87c25a9652b420f47c067a80671cdf48bfc6607f2dc13d198a6f75addc34c026

On an Intel Mac use astro_1.42.1_darwin_amd64.tar.gz instead, and expect 5346b505c5bb63dbe96092cc385716cc3f0106519df3c8e49e6b647498c6366d.

No sudo? Put it somewhere you own instead — mkdir -p ~/.local/bin && mv /tmp/astro ~/.local/bin/ — and make sure that directory is on your PATH.

Homebrew works too, but read this first: brew install astronomer/tap/astro@1.42.1 also installs Podman, because the formula recommends it. That is harmless — the Astro CLI looks for docker before podman — but it is ~100MB you did not ask for. Do not add --without-podman: current Homebrew rejects it outright with invalid option. And do not use plain brew install astro, which is a different, unpinned version.

Then pull the Airflow image now rather than on the day:

docker pull astrocrpublic.azurecr.io/runtime:3.3-2

Windows (PowerShell) — no winget and no admin rights needed, except for one step:

1. Enable WSL 2. This is the only part needing administrator rights, and it is one-time per machine. In an admin PowerShell, then reboot:

   wsl --update
   wsl --install --no-distribution

   If you cannot elevate, ask IT now — nothing else needs admin, but nothing works without this.

2. Install Docker Desktop, per-user, no admin: download it, then either double-click it or run & "$HOME\Downloads\Docker Desktop Installer.exe" install --user. Open it and leave it running.

3. From the repository directory, run:

   powershell -ExecutionPolicy Bypass -File .\setup-windows.ps1

   It installs uv, downloads the pinned Astro CLI, verifies its checksum against Astronomer's published value, puts it on your PATH and pulls the Airflow image. Then open a new terminal so the PATH change applies.

   -ExecutionPolicy Bypass applies to that one run only. It is there because managed laptops block scripts by default.

If scripts are blocked outright — common on managed laptops — the same steps by hand are in WINDOWS-SETUP.md, along with the winget shortcut if you happen to have it.

✓ Done when all three report a version:

docker info --format "{{.ServerVersion}}"     # a number, not an error
uv --version
astro version                                 # 1.42.1

If it fails

• astro version reports something other than 1.42.1 → on macOS you have a different Astro on your PATH. which astro tells you where it is; the download above puts 1.42.1 in /usr/local/bin.
• astro not found on Windows → you did not open a new terminal after the script added it to PATH.
• docker info errors → Docker Desktop is installed but not running. Open it.
• wsl errors on Windows → Step 1.1 has not been done, and it needs admin.

uv sync

This creates .venv/ and installs Dagster, the roots CLI and the sealed processing package. It does not install Airflow — that lives only inside the Docker image, deliberately.

✓ Done when

uv run roots --help

lists commands including join, doctor, verify and hint.

If it fails — a resolution error here usually means a stale .venv. Delete it and re-run.

Your instructor gives you a join code — six characters, like RMEETP.

uv run roots join RMEETP

You do not create .env yourself. This command creates both files you need:

They hold the same information in two shapes because two different runtimes need it two different ways. Never edit either by hand — if something is wrong, run roots join again.

If your instructor also gave you a lab address, add it:

uv run roots join RMEETP --lab-url http://192.168.1.50:8090

Usually you will not need to — the address is normally already built into the repository you cloned.

✓ Done when config/team.yaml and .env both exist, and the command printed joined as team-NN followed by an Airflow connection recipe.

If it fails

• could not join → the lab server is unreachable. Check the address with your instructor; you may need --lab-url.
• 401 or token does not match → the join code was mistyped. They are case-insensitive but otherwise exact.

uv run --env-file .env roots doctor

Note the --env-file .env, and use it from now on. Step 4 created that file; every command after this point needs the values in it, and that flag is what loads them. Forget it and things fail confusingly — Dagster loses its event log, the warehouse becomes unreachable. Just always include it.

roots doctor checks versions, Docker, ports, both runtimes, the lab server and the warehouse, and tells you exactly what to fix.

✓ Done when there are no red FAIL lines. Yellow .... lines are fine.

Do not go to Step 6 until this is green. This is the entire reason for doing setup early.

Three things run at once, so you need three terminal windows, all in the repository directory. Leave all three running for the whole course.

Terminal 1 — Airflow

astro dev start

The first run takes a few minutes; later ones are fast. Read the URL it prints — usually http://localhost:8080, but Astro picks a different port when 8080 is busy. Log in with admin / admin.

Terminal 2 — Dagster

uv run --env-file .env dg dev --target-path dagster

Opens on http://localhost:3000. The asset graph will be empty. That is correct — building it is mission D1.

Terminal 3 — the progress daemon

uv run --env-file .env roots watch

Re-checks your work every 20 seconds and updates the instructor's dashboard. Start it and forget it.

✓ Done when both UIs load in a browser and roots watch is printing.

uv run --env-file .env roots airflow-conn    # the values mission A1 needs

Then open missions/A1.md and start.

CONCEPTS.md          Mental models for Airflow and Dagster. Keep it open.
missions/            START HERE. A1 -> A5, then D1 -> D4.
dags/                Your Airflow pipeline. You edit this.
dagster/src/rootsmarkt/defs/
                     Your Dagster assets. You edit this.

packages/rootsmarkt-processing/
                     SEALED. All the data processing, written and tested.
                     Read it freely; you should not need to change it.
config/team.yaml     Your team's roster, credentials, revenue bounds.
data/                Deliveries you have fetched, and cleaned output.
hints/ checkpoints/  Read via `roots hint` and `roots checkpoint`.

Data processing reads, transforms, validates, aggregates and writes data. It is already written for you, in packages/rootsmarkt-processing/.

Orchestration decides when something runs, represents dependencies, waits for conditions, coordinates systems, retries work, records execution state, reacts to failures, and exposes what happened.

That is what you will write.

Every time you add a line today, you should be able to say which of the two it is. Some lines are genuinely on the boundary — those are the interesting ones, and the missions point them out.

You are not expected to memorise Airflow or Dagster syntax. You are expected to leave able to say "this is orchestration because…" and "I would represent this differently in Airflow and Dagster because…".

All of these take the uv run --env-file .env prefix.

roots verify a2          # check one mission now, and report it
roots status             # what this machine thinks you have earned
roots hint a2            # next hint. Run again for the one after.
roots checkpoint a2      # take a known-good state and keep moving
roots submit a2 ...      # evidence submissions (A2, D3, D4)
roots airflow-conn       # the exact Airflow connection values for A1
roots doctor             # when something is wrong and you do not know what
roots online --lab-url … # if the lab moves. Your instructor gives you the address.

Use hint and checkpoint freely. There is no penalty and nobody to ask. Losing forty minutes to one mission costs you the rest of the course, which is a much worse outcome than taking a hint.

Row counts, delivery IDs, store rosters and the plausible revenue range all differ per team, derived from your team ID.

If your numbers do not match the team beside you, nothing is broken. You are both right. It also means a copied answer will not verify — not because anyone is policing it, but because it is literally about different data.

roots doctor first, always. It names the fix.

If the lab server becomes unreachable, your work is not lost: milestones are recorded locally in .roots/ and sent when it comes back. If your instructor moves the lab to another machine, they will give you an address:

uv run roots online --lab-url http://<address>:8090
uv run --env-file .env roots doctor

Everything you have already fetched and loaded stays valid — the delivery data is identical whichever machine serves it.