5 AI agents collaborate in real-time via the @Coordinator fleet abstraction. A CEO coordinator manages 4 specialist agents, delegating tasks via the A2A protocol and synthesizing results into an executive briefing streamed to the browser over WebTransport/HTTP3.

This sample demonstrates:

• @Coordinator + @Fleet for multi-agent orchestration
• AgentFleet API for sequential and parallel agent dispatch
• @Agent + @AgentSkill for headless specialist agents
• Agent Activity Streaming — real-time agent-step events (thinking/completed) streamed to the browser via StreamingActivityListener
• Coordination Journal with rendered markdown tables showing the full execution graph
• Governance policy plane — @AgentScope on the coordinator, PolicyAdmissionGate.admit at user input, GovernanceFleetInterceptor at every cross-agent dispatch, signed CommitmentRecords on the journal. See § Governance.
• Plan-and-verify guardrails — Atmosphere's verifier statically checks the team's plan before any agent runs (cross-agent taint, an SMT budget bound, and a research-before-finance ordering automaton), plus a fail-closed human-in-the-loop gate on the consequential action. See § Plan-and-Verify.
• Result Evaluation — dual evaluators (SanityCheckEvaluator + LlmResultEvaluator) auto-score agent responses with EVAL rows in the journal
• SQLite Checkpoints — CheckpointingCoordinationJournal persists coordination state to atmosphere-checkpoints.db
• Skill files from GitHub — skill: prefix loads prompts from atmosphere-skills with SHA-256 integrity verification
• WebTransport over HTTP/3 — self-signed cert auto-discovery via /api/webtransport-info
• Admin Control Plane — live dashboard at /atmosphere/admin/ with kill-switch, hot-reload, OWASP matrix, agt verify export
• Sample runtime toggles: ADK (default), Embabel, Spring AI, and LangChain4j. Atmosphere has twelve contract-tested AgentRuntime adapters; this sample ships ready-made dependency toggles for these four.

• Java 21+
• A Gemini API key (set GEMINI_API_KEY env var). Demo mode works without one.

export GEMINI_API_KEY=your-key-here
./mvnw spring-boot:run -pl samples/spring-boot-multi-agent-startup-team

Open http://localhost:8080/atmosphere/console and type a prompt like: "Analyze the market for AI developer tools"

Open http://localhost:8080/atmosphere/admin/ to see the admin dashboard with live event stream, all 5 agents, fleet topology, and operational controls.

User message (WebSocket)
  |
  v
@Coordinator "ceo" with @Fleet of 4 agents
  |
  |-- Step 1: fleet.agent("research-agent").call("web_search")       [sequential]
  |
  |-- Step 2: fleet.parallel(                                         [parallel]
  |       fleet.call("strategy-agent", "analyze_strategy"),
  |       fleet.call("finance-agent", "financial_model"))
  |
  |-- Step 3: fleet.agent("writer-agent").call("write_report")       [sequential]
  |
  |-- Step 4: fleet.journal().formatLog()                            [observability]
  |
  v
session.stream(synthesisPrompt) --> Gemini LLM --> streams to browser

The AgentFleet handles transport automatically: local agents are invoked directly (no HTTP), remote agents use A2A JSON-RPC over HTTP. The developer writes only orchestration logic.

src/main/java/.../a2astartup/
  A2aStartupTeamApplication.java   # Spring Boot entry point
  CeoCoordinator.java              # @Coordinator with @Fleet (the orchestrator)
  ResearchAgent.java               # @Agent: web search via the built-in web_search tool
  StrategyAgent.java               # @Agent: SWOT analysis
  FinanceAgent.java                # @Agent: financial modeling
  WriterAgent.java                 # @Agent: report synthesis
  CheckpointConfig.java            # SQLite-backed CoordinationJournal via ServiceLoader
  StartupPlanRuntime.java          # Fallback when no API key
  LlmConfig.java                   # LLM settings from env vars

src/main/resources/
  application.yml                  # Atmosphere + LLM config
  META-INF/services/
    ...ResultEvaluator             # SanityCheckEvaluator + LlmResultEvaluator
    ...CoordinationJournal         # CheckpointConfig (SQLite-backed journal)

Skill files are loaded from atmosphere-skills at runtime via skill:startup-ceo, skill:startup-research, etc. No local prompt files — the PromptLoader fetches from GitHub on first run and caches to ~/.atmosphere/skills/.

The @Coordinator annotation registers this class as a fleet manager. @Fleet declares which agents belong to the fleet. The AgentFleet is injected into the @Prompt method automatically. The skill: prefix loads the CEO persona from the atmosphere-skills GitHub repo.

@Coordinator(name = "ceo",
    skillFile = "skill:startup-ceo",
    description = "Startup CEO that coordinates specialist A2A agents")
@Fleet({
    @AgentRef(type = ResearchAgent.class),
    @AgentRef(type = StrategyAgent.class),
    @AgentRef(type = FinanceAgent.class),
    @AgentRef(type = WriterAgent.class)
})
public class CeoCoordinator {

    @Prompt
    public void onPrompt(String message, AgentFleet fleet, StreamingSession session) {
        // Wire per-session activity streaming — clients see agent-step events in real time
        fleet = fleet.withActivityListener(new StreamingActivityListener(session));

        // Step 1: Research (sequential)
        var research = fleet.agent("research-agent").call("web_search",
                Map.of("query", message, "num_results", "3"));

        // Step 2: Strategy + Finance (parallel)
        var results = fleet.parallel(
                fleet.call("strategy-agent", "analyze_strategy",
                        Map.of("market", message, "research_findings", research.text())),
                fleet.call("finance-agent", "financial_model",
                        Map.of("market", message)));

        // Step 3: Writer synthesis
        var report = fleet.agent("writer-agent").call("write_report",
                Map.of("title", message, "key_findings", research.text()));

        // Step 4: CEO LLM synthesis
        session.stream("Write an executive briefing based on: " + report.text());
    }
}

The StreamingActivityListener emits agent-step events to the WebSocket as each agent transitions through Thinking -> Completed states. The JournalingAgentFleet (wired via CheckpointConfig) auto-evaluates each result with both SanityCheckEvaluator (word count, structure) and LlmResultEvaluator (Gemini judge via AgentRuntime.generate()). EVAL rows appear in the coordination journal with evaluator name, score, reason, and PASS/FAIL status.

Specialist agents are plain @Agent classes with @AgentSkill methods. They don't know they're in a fleet — the coordinator dispatches to them via the A2A protocol.

@Agent(name = "research-agent",
    skillFile = "prompts/research-skill.md",
    description = "Web research agent",
    endpoint = "/atmosphere/a2a/research")
public class ResearchAgent {

    @AgentSkill(id = "web_search", name = "Web Search",
        description = "Search the web for market data and news")
    @AgentSkillHandler
    public void webSearch(TaskContext task,
            @AgentSkillParam(name = "query") String query,
            @AgentSkillParam(name = "num_results") String numResults) {
        task.updateStatus(TaskState.WORKING, "Searching: " + query);
        // Atmosphere's built-in web_search tool: pluggable engine, fail-closed offline.
        var results = WebSearchSupport.shared()
                .search(WebSearchQuery.of(query, Integer.parseInt(numResults)));
        task.addArtifact(Artifact.text(results.toModelText()));
        task.complete("Found " + results.results().size() + " result(s)");
    }
}

The web_search engine is fail-closed: with no org.atmosphere.ai.websearch.endpoint configured it returns a clear "not configured" brief without touching the network, so the sample runs offline out of the box. Point that property at a JSON search endpoint (a self-hosted metasearch instance or a hosted JSON search API), and set org.atmosphere.ai.websearch.apiKey if the endpoint needs a credential, to get live results. Swap in an alternative backend by putting a WebSearchEngine service on the classpath.

Agent personas are loaded from the atmosphere-skills GitHub repo via the skill: prefix. On first run, PromptLoader.loadSkill() fetches each skill file from GitHub, verifies its SHA-256 hash against registry.json (supply chain protection), and caches it to ~/.atmosphere/skills/.

@Agent(name = "research-agent",
    skillFile = "skill:startup-research",  // loaded from GitHub
    endpoint = "/atmosphere/a2a/research")

To use a local skill file instead, drop the skill: prefix:

@Agent(name = "custom", skillFile = "prompts/custom.md")  // classpath only

The search order: classpath -> disk cache (~/.atmosphere/skills/) -> GitHub raw. Set atmosphere.skills.offline=true for air-gapped environments.

Atmosphere has twelve contract-tested AgentRuntime adapters. This sample ships ready-made dependency toggles for five of them, and only one is active at a time — swap by editing the dependencies in pom.xml:

To switch to Embabel:

1. Comment out the active Koog dependency in pom.xml
2. Uncomment Embabel dependencies
3. Run with: ./mvnw spring-boot:run -pl samples/spring-boot-multi-agent-startup-team

The console will show Runtime: embabel (or whichever runtime is active).

The CoordinationJournal records every dispatch, completion, evaluation, and routing decision. It renders as a markdown table in the browser via remark-gfm:

Two evaluators run automatically via ServiceLoader:

• SanityCheckEvaluator — hardcoded baseline (word count, error keywords, structure). No API key needed. Works in CI.
• LlmResultEvaluator — calls the active AgentRuntime (Gemini, etc.) as an LLM-as-judge. Prompt loaded from atmosphere-skills/llm-judge/SKILL.md. Uses AgentRuntime.generate() for synchronous one-shot evaluation.

Evaluations run asynchronously on a serialized virtual thread executor to avoid rate-limiting LLM APIs. Results stream to the client in real time via AgentActivity.Evaluated -> StreamingActivityListener -> AiEvent.AgentStep("eval", ...).

Where the Coordination Journal records the dispatch graph, the session tape records each agent's actual AI event stream — and lets you replay the whole team session deterministically, with no model in the loop. It is enabled in this sample (atmosphere.ai.tape.* in application.yml, on the SQLite store via the atmosphere-checkpoint dependency).

When the CEO fans out, CeoCoordinator stamps its own run id onto the dispatch:

// Link every specialist's tape run to this coordinator's run id.
fleet = fleet.withParentRun(session.runId().orElse(null));

Each specialist dispatched over A2A inherits that id and records it as parentRunId on its own tape run — so a single team request produces a tree: the CEO run plus one run per specialist. Because the specialists here are tool-agents (@AgentSkillHandler methods that return directly, never touching the LLM pipeline), each dispatch is recorded as a single completed run — the tree is complete whether children are LLM- or tool-backed.

Open the Atmosphere Console (/atmosphere/console/), go to the Tape tab, and click ▶ Replay on the coordinator run:

6 run(s) in this coordination — 1 coordinator + 5 agent(s), linked by parentRunId.
  COORDINATOR  <ceo run>      ▸ user prompt → executive briefing
  AGENT        web_search     ↳ parent <ceo run>
  AGENT        analyze_strategy
  AGENT        financial_model
  AGENT        write_report   (Market Analysis)
  AGENT        write_report   (Risk Assessment)

or hit the gated admin endpoint directly:

curl -s localhost:8080/api/admin/tape/runs/<ceoRunId>/replay | jq
# -> { present, runCount, root, children[] } — each reconstructed from the tape

See the Session Tape & Replay tutorial for the full story.

The CheckpointConfig class wires CheckpointingCoordinationJournal with SqliteCheckpointStore. Coordination state persists to atmosphere-checkpoints.db and survives JVM restarts.

# After a request, verify checkpoints on disk:
sqlite3 atmosphere-checkpoints.db "SELECT COUNT(*) FROM checkpoints;"

This sample applies the full governance policy plane. GovernanceConfig publishes a policy chain at boot; CeoCoordinator evaluates it at @Prompt entry AND on every cross-agent dispatch.

# Who's enforcing right now?
curl http://localhost:8080/api/admin/governance/policies
# Returns 4 policies with sha256 digests + armed/timed/dry-run flags

# Full health snapshot
curl http://localhost:8080/api/admin/governance/health

# Send an off-topic prompt through the MS-compatible /check endpoint
curl -X POST http://localhost:8080/api/admin/governance/check \
     -H 'Content-Type: application/json' \
     -d '{"agent_id":"ceo","action":"prompt","context":{"message":"write_code in python"}}'
# → {"allowed":false,"matched_policy":"dispatch-deny","evaluation_ms":1.14}

# Break-glass — arm the kill switch
curl -X POST http://localhost:8080/api/admin/governance/kill-switch/arm \
     -H 'Content-Type: application/json' \
     -d '{"reason":"incident-42","operator":"oncall"}'

# Compliance export (cross-vendor agt verify schema)
curl http://localhost:8080/api/admin/governance/agt-verify | jq '.summary'
# → OWASP 9/1 covered/not-addressed, EU_AI_ACT 4/1, HIPAA 3/1/1, SOC2 3/2

Every decision streams into the admin Decisions tab — expand an entry to see the matched policy, reason, and redaction-safe context snapshot.

The combination below isn't possible in any other JVM AI framework:

• Streaming transport + governance: decisions flow through the same WebSocket/SSE the UI uses. Admin console sees policy denies as they happen.
• Per-dispatch enforcement: GovernanceFleetInterceptor gates every coord→specialist hop — a coordinator mistakenly dispatching "write Python" to the research agent gets denied at the fleet boundary, not just at the user-facing entry.
• Signed audit trail over the same transport: the admin Commitments tab renders Ed25519-signed CommitmentRecords as they land on the journal.

Governance gates each dispatch; the verifier checks the plan. This sample wires Atmosphere's atmosphere-verifier — the native-Java implementation of Erik Meijer's "Guardians of the Agents" pattern (CACM, Jan 2026) — over the team's consequential actions, modelled as tools in StartupTools. One Policy enforces three properties:

The same policy drives three integration points:

1. Live plan verification (before any agent runs). CeoCoordinator.onPrompt calls planAndVerify.verify(plan) right after admission and before the first dispatch. A plan that would leak financials, over-commit, or skip research is refused — no specialist agent is dispatched — and a verify_plan card shows the verdict in the console.
2. Fail-closed approval gate (Approach 2). The CEO's consequential commit_budget runs through a GatedToolDispatcher + ApprovalGate at execution time — defense in depth on top of the static proof. Set startup.approvals.auto-approve=false to see the commit denied even on a verified plan (the tool never fires).
3. Console Validation tab. VerifierExampleSource surfaces four one-click goals at /atmosphere/console/ → Validation: one passes, and one each is refused by taint, SMT, and the automaton.

Analyze the market for AI fitness apps …          → VERIFIED (research→model→report→publish→commit)
Publish our confidential financial model …        → REFUSED (taint)
Commit the full requested budget …                → REFUSED (SMT: amount ≤ runway unprovable)
Skip research and jump straight to the model …    → REFUSED (automaton)

The headline guarantee carries over from a single agent to a whole team: no agent runs, and no tool fires, for a plan the verifier refuses. Regression coverage is in StartupTeamVerifierTest.

The team's intended work compiles to a flat JSON Workflow — the same AST the chain reasons over. In the sample a deterministic StartupPlanRuntime emits it so the demo runs without an API key; a real deployment swaps in any AgentRuntime (Spring AI, LangChain4j, ADK, …) and the PlanAndVerify contract is identical. Each step is a tool call; @binding references (SymRefs) thread one step's result into a later step's argument and are resolved only after the whole plan passes:

{ "goal": "Analyze the market and brief the board", "steps": [
  { "toolName": "web_search",      "arguments": { "query": "target market" },        "resultBinding": "research" },
  { "toolName": "financial_model", "arguments": { "market": "…", "tam_estimate": "15" }, "resultBinding": "financials" },
  { "toolName": "analyze_strategy","arguments": { "research": "@research" },          "resultBinding": "strategy" },
  { "toolName": "write_report",    "arguments": { "key_findings": "@strategy" },      "resultBinding": "report" },
  { "toolName": "check_runway",    "arguments": {},                                   "resultBinding": "runway" },
  { "toolName": "commit_budget",   "arguments": { "amount": "@runway" },              "resultBinding": "receipt" },
  { "toolName": "publish_to_board","arguments": { "body": "@report" },               "resultBinding": "published" }
] }

This plan passes: @report is derived from @strategy/@research, never from @financials, so nothing confidential reaches the board; commit_budget binds @runway against the ref(runway) bound; and web_search runs before the model.

• Taint — forward dataflow. The TaintVerifier walks the steps keeping a Map<binding, Set<sourceTool>>. financial_model is a rule source, so its @financials binding is tainted; publish_to_board.body is the @Sink. The leak plan binds body: "@financials" → tainted binding reaches the forbidden sink → refused. Taint propagates transitively and through both arms of a conditional, so a summary-of-a-summary still carries the taint.
• SMT — proof by refutation. For commit_budget.amount <= ref(runway) the solver asserts the negation and checks satisfiability. Benign binds amount: "@runway" — the same symbol on both sides, so runway > runway is UNSAT (proven). The over-budget plan binds amount: "@requested" (an unrelated symbol from request_budget) — requested > runway is SAT, a concrete counterexample → refused.
• Automaton — symbolic execution over a state set. Starting at research_pending, financial_model / analyze_strategy have a transition to an error state; web_search transitions to researched (from which the models are unconstrained). The skip-research plan calls financial_model first → error state reachable → refused.

// StartupVerifierConfig: one Policy, single-sourced from annotations
new Policy("startup-team", allowedTools,
        SinkScanner.scan(StartupTools.class),           // @Sink   -> taint rules
        List.of(researchBeforeFinanceAutomaton()),      // ordering
        Set.of("treasury"),                              // granted capabilities
        CapabilityScanner.scan(StartupTools.class))      // @RequiresCapability
    .withNumericInvariants(List.of(new NumericInvariant(
        "commit_budget", "amount", Op.LE, new RefBound("runway"))));   // SMT

// CeoCoordinator.onPrompt — Approach 1: verify before any agent is dispatched
Workflow plan = planAndVerify.plan(message);
if (!planAndVerify.verify(plan).isOk()) { session.error(...); return; }

// CeoCoordinator — Approach 2: fail-closed gate on the consequential action
new GatedToolDispatcher(new RegistryToolDispatcher(registry), approvalGate)
    .dispatch("commit_budget", Map.of("amount", "50000"));  // throws if denied

The policy runs in ControlFlowMode.LINEAR_ONLY (the default): plans are flat, so the proof covers the single sequence that runs. Switching to BRANCHING admits ConditionalNode plans, and every verifier — the structural checks and the SMT layer — descends into both arms (the runtime predicate is never trusted to keep an unsafe arm from firing). Full algorithms — taint dataflow, the subset-construction automaton, the SMT encoding, and the Condition guard grammar — are in the Plan-and-Verify tutorial.

The atmosphere-admin dependency enables a real-time management dashboard at /atmosphere/admin/:

• Dashboard — live counters (5 agents, 12 broadcasters), WebSocket event feed, connected resources
• Agents — all 5 agents listed: CEO coordinator (v1.0.0) + 4 headless specialists with a2a/mcp protocol badges
• Journal — query coordination events by coordination ID or agent name
• Control — broadcast messages to any agent, disconnect clients, cancel A2A tasks, with full audit trail

The event stream uses Atmosphere's own WebSocket transport — the admin dashboard eats its own dog food.

# REST API
curl http://localhost:8080/api/admin/overview     # system snapshot
curl http://localhost:8080/api/admin/agents        # all 5 agents
curl http://localhost:8080/api/admin/broadcasters   # 12 active broadcasters

The sample starts a Reactor Netty HTTP/3 sidecar on port 4446 with a self-signed ECDSA certificate. The frontend auto-discovers the transport via /api/webtransport-info (returns port + SHA-256 certificate hash) and connects with serverCertificateHashes. Falls back to WebSocket if WebTransport is unavailable.

WebTransportWiringE2ETest boots this application and proves the transport is genuinely wired: the HTTP/3 server binds a real UDP port, /api/webtransport-info advertises runtime-confirmed state only, the Alt-Svc advertisement is present in the servlet chain, and the WebTransportProcessor SPI resolves the starter's real processor (not the no-op fallback). Browser-level coverage lives in modules/integration-tests/e2e/webtransport*.spec.ts, but note what those specs do and do not assert. They cover the /api/webtransport-info discovery endpoint, transport negotiation, the fallback badge contract (data-transport / data-via-fallback), and a message round-trip over whichever transport was negotiated. They do not hard-assert that the connection is WebTransport: each one passes on the WebSocket fallback, and they skip entirely when netty-codec-http3 is off the classpath. An HTTP/3-only browser round-trip is not gated in CI today — verify it by reading data-transport and data-via-fallback, not just the word "Connected". separately by the Playwright specs in modules/integration-tests/e2e/webtransport*.spec.ts.

When you send a prompt, the console shows:

1. Agent Activity Events — agent-step frames stream in real time: "Agent 'research-agent' is thinking...", "completed in 336ms"
2. AGENT COLLABORATION — tool cards for each agent with expandable results (rendered as markdown via remark-gfm)
3. Coordination Journal — full execution table with EVAL rows showing evaluator name, score, reason, and PASS/FAIL
4. CEO Briefing — the LLM-generated executive summary with GO/NO-GO recommendation, key risks, and next steps
5. Agent Status Bar — bottom bar with per-agent status (thinking/completed) and green checkmarks

This sample includes atmosphere-interactions, so the Atmosphere Console (/atmosphere/console/) gains an Interactions tab over POST/GET /api/interactions. Launch a turn in the background and the detached run returns immediately — the natural fit for a long-running multi-agent task you kick off rather than hold a connection open. While it runs, the Console subscribes to the per-interaction stream (/atmosphere/interactions-stream?id=<id>) over WebSocket and renders each durable steps[] entry live as the agents produce it, falling back to polling the GET /api/interactions/{id} snapshot if the socket cannot open. Finished interactions can be continued, chaining context via previous_interaction_id.

The mutating endpoints are default-deny (Correctness Invariant #6); for this local demo application.yml sets atmosphere.interactions.http-write-enabled=true and demo-principal: demo-user — never enable either in production.