Run lifecycle

A workflow run starts with a UI command addressed to a run ID. Desktop passes it to orchestration, which hosts InteractiveEngine, connects providers and tools, persists checkpoints, and returns state changes to the UI.

Terms: glossary. Ownership: architecture contract. Each table names the source files for its part of the run.

1. Runtime boundaries

flowchart TB
    subgraph UI["ui — presentation"]
        Editor["Editor / Chat / Canvas"]
    end
    subgraph Desktop["desktop — Tauri IPC"]
        Cmd["invoke handlers"]
        Bridge["run-state event bridge"]
    end
    subgraph Orch["orchestration — run host + app state"]
        Backend["AppBackend"]
        Coord["RunCoordinator"]
        Drive["drive_interactive_workflow"]
        ToolPort["ToolPortImpl"]
    end
    subgraph Engine["engine — execution semantics"]
        IE["InteractiveEngine"]
        Ports["AiPort / ToolPort traits"]
    end
    subgraph Prov["providers — LLM transport"]
        AiClient["create_provider → AiClient"]
    end

    Editor -->|invoke| Cmd
    Cmd --> Backend --> Coord --> Drive
    Drive --> IE
    IE --> Ports
    Ports --> AiClient
    IE --> ToolPort
    Drive -->|RunTelemetry| Bridge -->|run-state| Editor

Concern	Rule owner	Adapter / host
Workflow graph validity, execution layers, turn semantics	engine (`graph/`, `execution/`)	orchestration calls `validate_workflow` before run
Tool approval policy (`ApprovalMode`)	engine (`tools/config.rs`)	orchestration filters tool catalog in `tool/registry.rs`
When to pause (`NeedsInteraction`)	engine (`InteractiveEngine::run`)	orchestration maps to `RunTelemetry` + session actions
Tool/subagent I/O, cwd, artifacts	orchestration (`run/execution/tool_port.rs`, `adapters/tool_impl/`)	implements `ToolPort`
LLM wire format, streaming, `AgentError`	providers (`client.rs`, `rig_adapter/`)	implements `AiPort`
Run session, checkpoints, trace projection	orchestration (`run/coordinator/`, `run/state.rs`)	desktop bridges to UI
Presentation, derived UI state	ui	never imports engine/providers
IPC transport	desktop	maps invoke ↔ `AppBackend`

Invariant: only orchestration/src/run/execution/ may construct InteractiveEngine (arch check enforced).

2. Run lifecycle

sequenceDiagram
    participant UI as ui
    participant DT as desktop
    participant BE as AppBackend
    participant RC as RunCoordinator
    participant DR as drive_interactive_workflow
    participant EN as InteractiveEngine
    participant PR as providers (AiPort)
    participant TP as ToolPortImpl

    Note over UI: Author / load (pre-run)
    UI->>DT: save_workflow / load workflows
    DT->>BE: WorkflowCatalog CRUD
    BE->>BE: merge app + project stores

    Note over UI: Start run
    UI->>DT: start_run(workflow, settings, projectId, key, entrypoint?)
    DT->>BE: start_run
    BE->>BE: resolve workspace + probe write access
    BE->>RC: start_run(execution cwd, run root)
    RC->>RC: validate_workflow
    RC->>RC: resolve_execution_cwd
    RC->>RC: prepare_workflow_run (provider router, snapshots)
    RC->>RC: create RunRecord, spawn drive task
    DR->>EN: InteractiveEngine::new / from_checkpoint
    DR->>EN: run(ai_adapter, tool_port, cancel_token)

    loop Until terminal or pause
        EN->>PR: AiPort::invoke_stream(AgentRequest)
        PR-->>EN: AgentTurnOutcome | AgentError
        EN->>TP: ToolPort::execute_batch (auto-approved tools)
        TP-->>EN: ToolBatchOutput
    end

    alt NeedsInteraction
        EN-->>DR: EngineRunResult::NeedsInteraction
        DR->>BE: RunTelemetry (await input/approval/retry)
        BE->>DT: apply_execution_event → emit run-state
        DT->>UI: run-state (WorkflowRunState)
        UI->>DT: submit_user_input / submit_tool_approval / retry_node
        DT->>EN: on_human_input / on_tool_decision / retry_node
        DR->>EN: run(...) again
    else Completed / Failed / Cancelled
        EN-->>DR: terminal EngineRunResult
        DR->>BE: Finished | Error | Aborted
        BE->>DT: final run-state
    end

Phase	Entry	Key symbols	Owner
Author / load	Editor save, bootstrap	`WorkflowCatalog::load_all`, `save_all`	orchestration
Validate	Run start, engine construct	`validate_workflow` → `execution_layers`	engine
Prep	`start_run`	`prepare_workflow_run`, `resolve_callable_agent_snapshots`	orchestration
Drive	spawned tokio task	`drive_interactive_workflow`, `wire_run`	orchestration
Pause / resume	`NeedsInteraction`	`emit_interaction_pause`, `await_interaction_actions`	orchestration wraps engine
Complete / fail	`EngineRunResult` terminal	`Finished(RunReport)`, `NodeFailed`, `Cancelled`	engine decides; orchestration projects

3. State machines

3.1 InteractiveEngine / EngineRunResult

Source: crates/engine/src/execution/interactive_engine/mod.rs → EngineRunResult, InteractiveEngine::run

stateDiagram-v2
    [*] --> Running: run() entered
    Running --> Running: dispatch AI + tools (FuturesUnordered)
    Running --> NeedsInteraction: idle + inputs/approvals/retryables
    Running --> Completed: all layers done
    Running --> Failed: terminal_error / NoRunnableNodesInLayer
    Running --> Cancelled: cancel_token
    NeedsInteraction --> Running: on_human_input / on_tool_decision / retry_node
    Completed --> [*]
    Failed --> [*]
    Cancelled --> [*]

`EngineRunResult` variant	Meaning
`NeedsInteraction { inputs, approvals, retryables }`	Batch pause; run not terminal
`Completed(RunReport)`	All nodes finished; aggregated outputs + counters
`Failed(RunError)`	Non-recoverable run failure
`Cancelled`	User/system cancel via `CancellationToken`

Pause payloads:

Struct	Fields	When
`EngineAwaitInput`	`node_id`, `label`, `context`, `is_initial`	Explicit `NeedsUserInput` request
`EngineAwaitApproval`	`approval_id`, `node_id`, `label`, `tool_calls`	Batch has `requires_approval == true`
`EngineRetryableNode`	`node_id`, `label`, `error`, `interrupted`	Transient exhaustion, user interrupt, or recoverable node error

3.2 Per-node turn (harness vs executable batching)

Sources: ports/outbound.rs → AgentTurnOutcome; providers/src/mapping/mod.rs → all_tool_specs / resolve_tool_turn_outcome; interactive_engine/completion.rs

stateDiagram-v2
    [*] --> Ready: build_request (unified catalog)
    Ready --> AwaitingInput: NeedsUserInput / Message (conversational)
    Ready --> PendingTools: ToolCalls (executable only)
    PendingTools --> RunningTools: auto-allow OR approved
    RunningTools --> Ready: on_tool_results / queued live user message
    Ready --> Completed: AgentTurnOutcome#58;#58;Completed
    Ready --> Failed: non-retryable AgentError
    Ready --> Backoff: AgentError#58;#58;Transient
    Ready --> Ready: MixedToolTurn retry feedback
    Backoff --> Ready: retry_after elapsed
    Completed --> [*]
    Failed --> [*]

Unified tool catalog (every turn):

Kind	Model-facing tools
Harness	`openflow_submit_node_output`, optional `openflow_request_user_input`, optional `openflow_ask_user_question`
Executable	Catalog tools from registry (+ `ToolPort::augment_request`)

A batch holds exactly one harness tool or one or more executable tools. Mixing them produces AgentError::MixedToolTurn.

AgentTurnOutcome routing (completion.rs):

Outcome	Effect
`Completed`	Store `NodeRunOutput`, clear recovery state, advance layer when all siblings done
`ToolCalls`	`apply_tool_calls` → approval policy → `pending_tool_batches`
`NeedsUserInput`	Conversational: pause (valid human-facing message or structured question). Autonomous: auto-continue nudge
`Message`	Direct chat (`conversationMode`): end turn and await next message. Workflow: auto-continue nudge (max streak)

Provider requests follow the same split: workflow nodes require a tool call; conversationMode nodes use automatic tool choice, so plain text is a valid outcome.

While a node is running model or tool work, ExecutionAction::ProvideInput goes to the engine's user-message queue. The engine appends the message to the node transcript, and the next AgentRequest includes it. Paused input, approval, and retry actions keep their own interaction handlers.

3.3 ToolCallStatus lifecycle

Source: crates/engine/src/tools/config.rs → ToolCallStatus

ToolCallStatus is used for UI and orchestration projection (events.rs). The engine loop does not use it.

stateDiagram-v2
    [*] --> Proposed: ToolCallProposed
    Proposed --> AwaitingApproval: ApprovalMode requires prompt
    Proposed --> Running: auto-allow
    AwaitingApproval --> Running: ToolApproved
    AwaitingApproval --> Blocked: ToolDenied / read_only blocked
    Running --> Completed: success
    Running --> Failed: error result
    Running --> Aborted: run Aborted
    Blocked --> [*]
    Completed --> [*]
    Failed --> [*]
    Aborted --> [*]

3.4 Orchestration session / RunStatus

Sources: run/persistence.rs → RunStatus; run/state.rs → WorkflowRunState, AgentStatus

stateDiagram-v2
    [*] --> Running: start_run / resume
    Running --> Paused: checkpoint (input / approval / retry)
    Running --> Stopped: user stop
    Running --> Completed: Finished
    Running --> Failed: NodeFailed / terminal error
    Paused --> Running: continue_run / resume_durable_run / submit_*
    Stopped --> Running: continue_run (in-session checkpoint)
    Completed --> [*]
    Failed --> [*]

Live flags on WorkflowRunState: active, awaiting_node_ids, pending_approvals, status_by_node, chat_logs, run_trace.

3.5 Subagent status

Source: engine/tools/config.rs → SubagentStatus; telemetry SubagentsDeclared / SubagentStarted / SubagentCompleted / SubagentFailed

stateDiagram-v2
    [*] --> Declared: openflow_declare_subagents
    Declared --> Active: openflow_call_subagent → SubagentStarted
    Active --> Completed: subagent turn machine Done
    Active --> Failed: SubagentFailed
    Completed --> [*]
    Failed --> [*]

Subagent turns run inside a parent CALL_SUBAGENT tool execution (subagent_runtime.rs) and are not peers of the nodes in a DAG layer.

4. Validation, pauses, and recovery

Gate	Trigger	Owner	Blocks?	Resume API	Failure mode	Source
Workflow validation	Empty graph, duplicate ids, dangling edge, cycle	engine	Yes (start)	Fix workflow	`WorkflowValidationError`	`graph/validation.rs` → `validate_workflow`
Provider / key readiness	Any referenced provider lacks resolvable credentials	orchestration	Yes (start)	Settings / transient key	Backend error to UI	`settings/provider.rs` → `resolve_provider_config`
MCP setup	Enabled server cannot connect, list tools, or register a tool	orchestration	No	Fix, test, or disable server in Settings	Server/tool skipped; system chat warning	`adapters/mcp/mod.rs` → `run/execution/drive/setup.rs`
`request_user_input`	Model calls `openflow_request_user_input` with one free-text question or `openflow_ask_user_question` with valid structured questions	engine	Yes (pause)	`submit_user_input`	Invalid request → nudge then fail	`completion.rs`, `allow_user_input` on `AgentRequest`
Tool approval (`ApprovalMode`)	Write-tier tool under `write` / `always_ask`	engine policy; orch catalog	Yes (pause)	`submit_tool_approval` → `on_tool_decision`	Deny → synthetic denied `ToolResult`	`tools/config.rs`, `completion.rs`
`read_only` tool exposure	Write-tier tool not in catalog	orchestration	Yes (model can't call)	Switch `ApprovalMode`	Tool unavailable to model	`tool/registry.rs` → `is_read_only`
Retryable node / `RetryPolicy`	`AgentError::Transient`, interrupt, recoverable fail	engine	Yes (pause)	`retry_node`	Exhaust → `failed_nodes` / run fail	`completion.rs`, `RetryPolicy` in `WorkflowSettings`
Cancel	User stops an addressed run	orchestration + token	Yes (terminal)	New run / continue	`EngineRunResult::Cancelled`	`CancellationToken`, `stop_run(run_id)`
Structured output / completion protocol	Invalid `openflow_submit_node_output` JSON	engine	No (nudge retry)	Auto (≤3 nudges)	Then `failed_nodes`	`AgentError::MalformedSubmitOutput`
DAG layer readiness	Upstream node lacks output	engine	No (wait)	Auto when upstream completes	N/A	`is_node_blocked`, `gather_call_ai_actions`
Checkpoint restore	User stop / durable resume	orchestration	N/A	`continue_run` / `resume_durable_run`	Hash / stale node mismatch	`checkpoint.rs`, `run_checkpoint_store.rs`
Callable agent snapshot	Unknown agent id at run start	engine	Yes (prep)	Fix workflow/agents	Prep error	`resolve_callable_agent_snapshots`

Provider resolution order: node AgentNodeConfig.provider_id → workflow WorkflowSettings.provider_id → active settings provider. Run prep creates clients for every referenced provider. Requests never fail over across providers.

API key resolution order (resolve_api_key): transient input panel → stored ProviderProfile.api_key → provider env var. Bedrock uses AWS profile/region, not API keys.

5. Data flowing across seams

flowchart LR
    W["Workflow + Node + AgentNodeConfig"]
    NI["NodeInvocationContext"]
    AR["AgentRequest"]
    WIRE["Provider wire JSON/SSE"]
    ATO["AgentTurnOutcome"]
    TC["ToolCall / ToolResult"]
    RT["RunTelemetry"]
    DTO["WorkflowRunState"]

    W --> NI --> AR --> WIRE --> ATO
    ATO --> TC
    ATO --> RT
    TC --> RT
    RT --> DTO

Seam	Type	Important fields
Workflow → invocation	`NodeInvocationContext`	upstream outputs and handoff refs, `entrypoint_text`, `shared_context`, callable snapshots
→ `AgentRequest`	`build_agent_request`	`provider_id`, `model`, reasoning options, `fast_mode`, `system_messages`, `task_prompt`, `input` JSON, `output_schema`, `tool_config` (approval + structured-input capability), `available_tools`, `allow_user_input`, `transcript`
→ provider	`ProviderRouter` → `AiPort::invoke_stream`	Effective provider, model id, reasoning options, independent fast-mode service tier, phase-specific `available_tools` (`mapping/mod.rs` → `all_tool_specs`)
← provider	`AgentTurnOutcome`	`Completed.output`, optional `Completed.handoff`, `ToolCalls.tool_calls`, control transitions
Tool execution	`ToolPort::execute_batch`	`ToolBatchOutput` → `on_tool_results`
→ UI	`RunTelemetry` / `ExecutionEvent`	`ChatMessageDelta`, `NodeAwaitingInput`, `ToolApprovalRequested`, `NodeCompleted`, `Finished`
Projection	`apply_event_to_run_state`	Mutates `status_by_node`, `chat_logs`, `pending_approvals`, `outputs`, `handoffs`

Orchestration wraps the run-scoped provider router with AiInvocationAdapter. After a valid submit, the adapter writes the configured Markdown or JSON handoff under the run root, attaches its immutable run:// manifest, then emits completion and usage events.

6. Persistence map

Data	Location	Writer	Notes
App workflows	`{data_local}/openflow/workflows.json`	`app_workflow_store`	Unassigned workflows only on save
Chats	`{data_local}/openflow/chats.json`	`chat_store`	Chat identity, title, project/runtime config, timestamps, and durable run ID; separate from workflow catalog
Project workflows	`{project}/.flow/workflows/{id}.workflow.json`	`project_workflow_store`	Wins on ID collision at load
App-managed execution folders	`{data_local}/openflow/workspaces/{workflows\|chats}/{id}`	workspace resolver	Isolated cwd for runs without a project
Settings + API keys	`{data_local}/openflow/settings.json`	`settings_store`	Plaintext keys
Projects registry	`{data_local}/openflow/projects.json`	`project_store`	Bindings + assigned workflow ids
Saved agents (`CallableAgent`)	`{data_local}/openflow/agents.json`	`agent_store`	Snapshotted at run start
Run record	`{run_root}/{run_id}/run.json`	`RunCoordinator`	`RunRecord` + `workflow_snapshot`
Checkpoints	`{run_root}/{run_id}/checkpoints/NNNN.json`	`run_checkpoint_store`	Engine + `WorkflowRunState` projection
Artifacts	`{run_root}/{run_id}/artifacts/`	`ArtifactStore`	Large tool outputs
Node handoffs	`{run_root}/{run_id}/handoffs/{node_id}/HANDOFF.md` or `HANDOFF.json`	`HandoffStore`	Canonical node result; SHA-256 manifest travels through run state and downstream input
In-session checkpoint	`RunSession.engine_checkpoint`	drive lifecycle	User stop → Continue (no disk required)

Merge rule: WorkflowCatalog::load_all loads the app store, then overlays project-discovered workflows (project wins on the same WorkflowId).

7. Concurrency & cancellation

Mechanism	Behavior	Source
Top-level runs	`RunRegistry` addresses sessions by `run_id`, permits one active run per workflow, and targets 64 in-process sessions by evicting inactive sessions first; active sessions are never evicted	`run/registry.rs`
Provider budget	At most 8 active provider invocations; excess calls await a permit	`run/resources.rs`
Tool budget	At most 16 executable tools across runs; excess calls await a permit	`run/resources.rs`, `tool_port.rs`
Same-cwd mutation	One mutation-capable run per canonical execution cwd	`run/resources.rs`, `execution/mod.rs`
Layer parallelism	Same-layer ready nodes dispatch concurrently via `FuturesUnordered`	`interactive_engine/mod.rs`
Completion ordering	AI/tool completions applied serially as they finish	`in_flight.next()` in `run()`
Cross-layer	Strict: layer N+1 after all layer N outputs	`current_layer_complete`
Tool batch parallelism	`ToolConcurrency::Shared` tools spawn in parallel inside batch	`tool_port.rs`
Exclusive tools	Semaphore permit before spawn	`tool/registry.rs`, `tool_port.rs`
Subagents	Nested in parent tool; not layer-parallel	`subagent_runtime.rs`
Cancel	`CancellationToken` checked while waiting for a mutation lease and during `run()`	addressed `stop_run(run_id)`
Event coalescing	~30ms independently per active run	`desktop/run_event_bridge.rs`
Blocking I/O	Filesystem/subprocess on `spawn_blocking`	orchestration adapters

Headless vs interactive: both call drive_interactive_workflow with the same InteractiveEngine. Headless (run_workflow_headless) uses scripted ExecutionAction queues instead of the UI. Without a coordinator, checkpoints are not durably persisted.

8. Worked scenarios

A. Linear 2-node readiness happy path

UI: start_run with entrypoint: null.
validate_workflow → layers [ [A], [B] ].
InteractiveEngine::new → layer 0: A has all required inputs → gather_call_ai_actions → AiPort::invoke_stream.
Completed(AgentTurnSuccess) → output stored → layer advances.
B gets the upstream map in AgentRequest.input → completes → EngineRunResult::Completed(RunReport).
RunTelemetry::Finished(report) → WorkflowRunState.active = false.

B. Node pauses for user input, resume

A node with request_user_input: true starts when its upstream inputs are ready.
The model calls openflow_request_user_input or openflow_ask_user_question → awaiting_nodes → run() returns NeedsInteraction { inputs: [EngineAwaitInput { ... }], ... }.
Orchestration emits NodeAwaitingInput → UI composer enabled.
User: submit_user_input accepts the text, removes that node's actionable structured request from WorkflowRunState, appends the user transcript message, and forwards to engine.on_human_input(node_id, text). Awaiting/status transitions still follow execution events.
Next run() → AI invoke with the user message in the transcript.

C. Tool needs approval, approve, continue

Model returns AgentTurnOutcome::ToolCalls with a write-tier tool under ApprovalMode::Write.
apply_tool_calls sets requires_approval: true on the batch.
NeedsInteraction { approvals: [EngineAwaitApproval] } → ToolApprovalRequested.
User: submit_tool_approval(allow: true) → on_tool_decision clears the approval flag.
take_ready_tool_batches → ToolPort::execute_batch inside run().
on_tool_results → next model turn with the unified tool catalog.

D. Write tool blocked under `read_only`

NodeToolConfig.approval_mode = ReadOnly.
ToolRegistry exposes only read-tier tools (read, search, …); write tools omitted.
The model cannot invoke bash / write. Unavailable tools never reach the ToolDecision path, even through a policy edge case.

E. Node fails / retry / exhaust

Provider returns AgentError::Transient → backoff via WorkflowSettings.retry_policy.
Retry streak exhausted → failed_nodes → EngineRetryableNode in NeedsInteraction.
UI: retry_node → engine.retry_node clears failure state.
Non-retryable error → Failed(RunError::NodeFailed) → run terminal.

F. Cancel mid-run

UI: stop_run → cancel token + snapshot_and_abort.
In-flight work sees the cancel at the next run() loop check → EngineRunResult::Cancelled.
RunTelemetry::Aborted → projection marks nodes/tools stopped/aborted.

G. Subagent call

Parent turn calls openflow_declare_subagents → SubagentsDeclared.
openflow_call_subagent → SubagentStarted; nested state machine in subagent_runtime.rs.
Subagent uses the unified catalog (allow_user_input: false) and tool batches until Done.
SubagentCompleted / SubagentFailed → parent receives ToolResult.

H. Headless acceptance path

Test calls run_workflow_headless(workflow, ...).
Same wire_run + drive_interactive_workflow; manual inputs/approvals dequeued from script.
Returns WorkflowRunSnapshot or an error; emits no Tauri run-state events.

9. Invariants & non-goals

Engine guarantees

Valid workflows have acyclic execution layers; validate_workflow is the gate.
A tool batch never mixes harness and executable tools (AgentError::MixedToolTurn).
NeedsInteraction is the only pause surface from run(); orchestration must not invent parallel pause semantics.
Node outputs propagate downstream only after Completed outcome stores structured output.

Orchestration must not

Reimplement approval rules (only project ToolCallStatus, filter catalogs).
Branch on concrete provider types (use Box<dyn AiPort>).
Construct InteractiveEngine outside run/execution/.

Providers must not

Decide tool approval, DAG advancement, or run lifecycle.
Offer a human-input harness when allow_user_input == false.
Persist workflow or session state.

Boundary violations to avoid: UI importing engine types directly; desktop calling orchestration internals; engine performing filesystem/HTTP I/O.

10. Appendix

A. Key file index

Path	Role
`crates/ui/src/api.ts`	IPC seam; `start_run`, `submit_*`, `listenToRunState`
`crates/ui/src/context/appProvider/useRunSession.ts`	Run / continue / kickoff handlers
`crates/desktop/src/commands/run.rs`	Tauri command wiring
`crates/desktop/src/run_event_bridge.rs`	`RunTelemetry` → `run-state`
`crates/orchestration/src/backend/runs.rs`	`AppBackend` run entry
`crates/orchestration/src/run/coordinator/mod.rs`	Session lifecycle
`crates/orchestration/src/run/coordinator/session.rs`	`prepare_workflow_run`
`crates/orchestration/src/run/prep.rs`	Reasoning defaults, execution prep
`crates/orchestration/src/run/execution/drive/mod.rs`	Main drive loop
`crates/orchestration/src/run/execution/drive/interaction.rs`	Pause/resume host
`crates/orchestration/src/run/execution/events.rs`	`apply_event_to_run_state`
`crates/orchestration/src/run/execution/tool_port.rs`	`ToolPort` impl
`crates/orchestration/src/run/execution/ai_adapter.rs`	Streaming + telemetry wrapper
`crates/orchestration/src/run/execution/headless.rs`	Acceptance runner
`crates/orchestration/src/workflow/catalog.rs`	Workflow merge/split
`crates/orchestration/src/settings/provider.rs`	Key resolution
`crates/engine/src/execution/interactive_engine/mod.rs`	`InteractiveEngine`, `EngineRunResult`
`crates/engine/src/execution/interactive_engine/completion.rs`	Turn outcome routing
`crates/engine/src/execution/node_invocation.rs`	`AgentRequest` assembly
`crates/engine/src/execution/subagent_runtime.rs`	Subagent builtins
`crates/engine/src/graph/validation.rs`	Graph gates
`crates/engine/src/ports/outbound.rs`	Port contracts
`crates/engine/src/execution/telemetry.rs`	`RunTelemetry`
`crates/providers/src/lib.rs`	`create_provider`
`crates/providers/src/client.rs`	`AiClient: AiPort`
`crates/providers/src/mapping/mod.rs`	Tool specs + outcome parsing

B. Enum cheat sheet

Enum	Variants	File
`EngineRunResult`	`NeedsInteraction`, `Completed`, `Failed`, `Cancelled`	`interactive_engine/mod.rs`
`AgentTurnOutcome`	`Completed`, `ToolCalls`, `NeedsUserInput`, `Message`	`ports/outbound.rs`
`AgentError`	`Transient`, `Permanent`, `Failed`, `MalformedSubmitOutput`, `MixedToolTurn`, `Interrupted`	`ports/outbound.rs`
`ApprovalMode`	`ReadOnly`, `AlwaysAsk`, `Write`, `Yolo`	`tools/config.rs`
`ToolDecision`	`AutoAllow`, `Prompt`, `Deny` (reserved)	`tools/config.rs`
`ToolCallStatus`	`Proposed`, `AwaitingApproval`, `Running`, `Completed`, `Blocked`, `Failed`, `Aborted`	`tools/config.rs`
`SubagentStatus`	`Declared`, `Active`, `Completed`, `Failed`	`tools/config.rs`
`RunStatus`	`Running`, `Paused`, `Stopped`, `Completed`, `Failed`	`run/persistence.rs`
`RunCheckpointReason`	`AwaitingInput`, `AwaitingToolApproval`, `AwaitingRetry`, `UserStopped`, `Completed`, `Failed`	`run/persistence.rs`
`AgentStatus`	`Idle`, `Queued`, `Started`, `AwaitingInput`, `AwaitingToolApproval`, `RunningTool`, `Completed`, `Failed`, `Interrupted`, `Stopped`	`run/state.rs`
`TraceStatus`	`Queued`, `Running`, `Paused`, `Failed`, `Stopped`, `Completed`	`run/state.rs`
`WorkflowValidationError`	`EmptyWorkflow`, `DuplicateNodeId`, `DuplicateEdgeId`, `MissingEndpoint`, `SelfEdge`, `Cycle`, `InternalConsistency`	`graph/validation.rs`

RunTelemetry variants: NodeQueued, NodeStarted, ChatMessage, ChatMessageDelta, NodeAwaitingInput, ToolCallProposed, ToolApprovalRequested, ToolApproved, ToolDenied, ToolStarted, ToolRetrying, ToolUpdated, ToolCompleted, ToolArtifactCreated, FileChanged, EditBatchRecorded, NodeCompleted, NodeFailed, NodeInterrupted, NodeErrored, Finished, Aborted, Error, SubagentsDeclared, SubagentStarted, SubagentCompleted, SubagentFailed, PhaseTimed, UsageReported, AiInvokeFailed.

C. Ambiguities / dead paths / TODOs

Item	Notes
`ToolDecision::Deny`	Reserved in `decision_from_mode`; not emitted today.
Run entry points	The header Run button starts without a starter message (`AppHeader.tsx`). Runs can also start through ⌘/Ctrl+Enter or the chat kickoff composer (`useRunSession.ts`).
`AiInvokeFailed`	Emitted but `apply_event_to_run_state` intentionally no-ops (incident hook).
Stale pause actions	Wrong-node input/approval logged as `"ignored …"` `Error` event; skipped non-fatally.
`ReadOnly` approval matrix	Engine `decision_from_mode` auto-allows all tiers; write blocking is enforced by orchestration tool catalog filtering.

run-persistence.md: durable replay and resume edge cases
callable-agents.md: snapshot rules and subagent model
technical-overview.md: complementary architecture narrative
threading-concurrency.md: runtime topology and layer concurrency
crates/orchestration/tests/workflow_acceptance.rs: headless scenario catalog

Spot-check (2026-07-18): verified EngineRunResult variants (mod.rs:62–72), validate_workflow delegation (validation.rs:64–66), ApprovalMode + decision_from_mode (config.rs:69–77), RunStatus variants (persistence.rs:9–14), and RunTelemetry terminal variants (telemetry.rs:126–128).

Run preparation and dispatch

Runtime path

text

UI
  -> Desktop IPC
    -> Orchestration
      -> Engine
      -> Providers
      -> Tools and storage adapters

The UI sends a start, continue, or resume command through the desktop IPC seam.
Orchestration resolves settings, provider readiness, workflow storage, run root, and execution cwd.
Orchestration creates the provider adapter through providers::create_provider.
The engine validates and advances the workflow.
Agent nodes assemble an AgentRequest: system prompt, upstream input, shared context, tools, and callable-agent snapshots.
The provider returns model output, tool calls, or terminal output.
Tool calls go through ToolPortImpl in orchestration and can pause for approval.
Orchestration projects engine events into run state, trace entries, chat logs, and desktop events.
Durable run data supports history and read-only replay. Paused, stopped, and failed runs can resume when their latest checkpoint is recoverable; completed workflow runs are replay-only. See Run persistence for the saved records and checkpoints.

Ports

Add a port only when a consumer is typed against its interface; otherwise call the concrete type directly. OpenFlow uses:

AiPort and AgentRequest: model invocation
ToolPort: tool and subagent execution
InteractiveEngine::on_human_input and InteractiveEngine::on_tool_decision: resuming interaction pauses
api.ts wrappers: UI-to-desktop calls

Run lifecycle ​

1. Runtime boundaries ​

2. Run lifecycle ​

3. State machines ​

3.1 InteractiveEngine / EngineRunResult ​

3.2 Per-node turn (harness vs executable batching) ​

3.3 ToolCallStatus lifecycle ​

3.4 Orchestration session / RunStatus ​

3.5 Subagent status ​

4. Validation, pauses, and recovery ​

5. Data flowing across seams ​

6. Persistence map ​

7. Concurrency & cancellation ​

8. Worked scenarios ​

A. Linear 2-node readiness happy path ​

B. Node pauses for user input, resume ​

C. Tool needs approval, approve, continue ​

D. Write tool blocked under read_only ​

E. Node fails / retry / exhaust ​

F. Cancel mid-run ​

G. Subagent call ​

H. Headless acceptance path ​

9. Invariants & non-goals ​

10. Appendix ​

A. Key file index ​

B. Enum cheat sheet ​

C. Ambiguities / dead paths / TODOs ​

D. Related implementation ​

Run preparation and dispatch ​

Runtime path ​

Ports ​