Threading and concurrency

Orchestration runs on the Tauri Tokio runtime handle in the desktop app. Each workflow run has one execution task. Within a DAG layer, ready AI calls and tool batches run concurrently via FuturesUnordered, and completions apply to engine state one at a time. The architecture contract defines ownership across the runtime boundaries.

Concurrency constraints:

1. Runtime split outside Tauri: tests or helper entry points can construct AppBackend with a separate owned runtime through AppBackend::new(...).
2. Blocking I/O coverage depends on dispatching filesystem and subprocess work through spawn_blocking.
3. Concurrency is layer-bounded: independent nodes overlap only while they share a ready layer, and later layers still wait.
4. Addressable run sessions share provider/tool budgets and hold one mutation lease per canonical execution cwd.
5. Many sync Tauri commands do filesystem or settings I/O on the command thread pool.

Process & runtime topology

Everything runs in one Tauri desktop process. In production, desktop injects tauri::async_runtime::handle() into AppBackend::with_runtime_handle(), and workflow execution is spawned from that handle.

flowchart TB
    subgraph Process["Single Tauri Process"]
        subgraph WebView["WebView Thread (SolidJS UI)"]
            UI["AppProvider / Canvas / Chat"]
        end

        subgraph TauriRT["Tauri async_runtime"]
            TC["Tauri Commands<br/>(invoke handlers)"]
            EB["Event Bridge Task<br/>(recv execution events)"]
        end

        subgraph BackendRT["Injected Tokio Handle"]
            EX["drive_interactive_workflow<br/>(single task per run)"]
        end

        subgraph SyncPool["Tauri sync command pool"]
            SC["sync fn commands<br/>load/save workflows, settings, etc."]
        end
    end

    UI -->|"invoke()"| TC
    UI -->|"listen('run-state')"| EB
    TC -->|"lookup run_id"| RR[("RunRegistry")]
    EB -->|"apply_execution_event_for(run_id)"| RR
    RR --> RS1[("RunSession A")]
    RR --> RS2[("RunSession B")]
    TC -->|"action_tx.send()"| CH[["Unbounded MPSC"]]
    EX -->|"event_tx.send()"| CH
    CH --> EB
    CH --> EX
    TC -->|"runtime.spawn()"| EX
    UI -->|"invoke() sync"| SC
    SC --> FS[("Disk / Keychain")]

Where each runtime lives

Component	Runtime	File
Workflow execution task	injected tokio::runtime::Handle	crates/orchestration/src/backend/mod.rs, crates/orchestration/src/run/execution/mod.rs
Tauri commands + event fanout	tauri::async_runtime	crates/desktop/src/lib.rs
Sync file commands	Tauri blocking pool	crates/desktop/src/lib.rs

• AppBackend::with_runtime_handle() uses the handle desktop passes in. Tests can instead create an owned tokio::runtime::Runtime and pass its handle to AppBackend::new(...).
• spawn_interactive_workflow_run() (crates/orchestration/src/run/execution/mod.rs) spawns workflow runs.
• start_run() (crates/desktop/src/lib.rs) spawns the event bridge on tauri::async_runtime. The bridge receives execution events, applies them to run state, and emits run-state to the UI.

Issue: runtime split outside the desktop path

On the desktop path, command tasks and run execution share the Tauri runtime handle. Tests or bespoke non-Tauri entry points may still construct AppBackend with a separate owned runtime. That split is useful for local construction but must not become the production path. Keep desktop execution on the injected handle, and reserve owned runtimes for tests or explicit non-Tauri entry points.

Workflow execution model (one task, concurrent layer work)

Each run is a single async task. Inside it, InteractiveEngine::run() drives a FuturesUnordered of ready work: model invocations and tool batches for every node runnable in the layer. Completions apply serially as they arrive. Within one tool batch, ToolPortImpl runs adjacent ToolConcurrency::Shared tools in parallel and serializes ToolConcurrency::Exclusive tools with per-tool semaphores.

stateDiagram-v2
    [*] --> EngineRun

    EngineRun --> Done: Completed
    EngineRun --> Failed: Failed
    EngineRun --> Cancelled: Cancelled
    EngineRun --> AwaitInteraction: NeedsInteraction

    AwaitInteraction --> WaitActions: action_rx.recv().await
    WaitActions --> EngineRun: ProvideInput / ToolApproval / RetryNode

    Done --> [*]
    Failed --> [*]
    Cancelled --> [*]

The host loop, drive_interactive_workflow() (crates/orchestration/src/run/execution/drive/mod.rs), calls InteractiveEngine::run(). The engine invokes AiPort and ToolPort internally until it completes, fails, or returns NeedsInteraction.

Layer parallelism (implemented)

execution_layers() groups nodes by dependency depth. For the active layer, InteractiveEngine schedules every ready AI call and tool batch into one FuturesUnordered pool (crates/engine/src/execution/interactive_engine/mod.rs). Independent nodes in a layer overlap in wall time; state updates stay single-threaded on completion.

Subagent waits

A subagent runs inside its parent's tool batch and shares ai, tool_runner, and channels. The parent waits for its result. Other ready nodes in the layer continue through the engine's FuturesUnordered pool, so the subagent does not block the whole run.

Blocking I/O coverage

ToolRunner and RunCoordinator use spawn_blocking for search/find/write/edit/patch, local reads, preview/git/revert, execution cwd resolution, and large artifact spills. Desktop and orchestration share one injected Tokio Handle (the Tauri runtime in production).

ToolRunner::execute is async. Keep concrete local filesystem and subprocess work inside blocking tasks:

flowchart LR
    subgraph AsyncTask["async task on Tokio worker"]
        EX["tool_runner.execute().await"]
    end

    subgraph Blocking["Runs on blocking thread"]
        FS["fs::read_to_string"]
        WD["WalkDir tree walks"]
        CMD["Command::output (ast-grep)"]
        REG["Regex scan entire files"]
        HTTP["HTTP clients stay async"]
    end

    EX --> FS
    EX --> WD
    EX --> CMD
    EX --> REG
    EX --> HTTP

Sync work in crates/orchestration/src/tool/dispatch.rs and crates/orchestration/src/run/coordinator/mod.rs:

• fs::read_to_string, fs::read_dir, fs::metadata
• WalkDir over large trees
• Command::new("ast-grep").output() (sync subprocess wait)
• search(), which reads every file and runs the regex on every line

read_url uses async HTTP. Local filesystem and subprocess tools should keep using spawn_blocking.

Impact

A large search or find call can still consume blocking-pool capacity:

1. The run task waits for the blocking work to finish.
2. Other blocking tasks can queue behind long file scans or subprocess calls.
3. UI commands that also need blocking-pool work can feel delayed.

Tauri command threading

Most commands are synchronous fn. Only a handful are async fn; the rest block a Tauri worker. Sync commands call straight into orchestration stores (crates/orchestration/src/adapters/storage/).

flowchart TB
    UI["UI invoke()"] --> Router{Tauri command type}

    Router -->|async fn| AsyncCmd["start_run, submit_user_input,<br/>submit_tool_approval, bootstrap_app"]
    Router -->|sync fn| SyncCmd["load_workflows, save_workflow,<br/>save_settings, provider key ops, etc."]

    AsyncCmd --> TauriAsync["tauri async_runtime"]
    SyncCmd --> SyncThread["blocking thread pool"]

    SyncCmd --> Disk["fs::read/write workflows.json"]
    AsyncCmd --> Mutex["lock RunSession"]

Impact

• Saving workflows during an active run competes for disk I/O on a blocking pool thread. This is usually fine, but large files can delay command responses.
• Provider API key load/save reads and writes settings.json synchronously.
• The UI awaits invoke(), so long sync commands make the app feel frozen.

Run session locking & channels

Each run keeps its lifecycle state behind its own mutex:

sequenceDiagram
    participant EX as Execution Task<br/>(injected runtime)
    participant CH as MPSC channels
    participant EB as Event Bridge<br/>(tauri runtime)
    participant UI as UI commands<br/>(tauri runtime)
    participant RS as Mutex RunSession

    EX->>CH: event_tx.send(NodeStarted)
    CH->>EB: recv event
    EB->>RS: lock().await
    EB->>RS: apply_event_to_run_state
    EB->>RS: unlock
    EB->>UI: emit("run-state")

    Note over EX: blocks on action_rx

    UI->>RS: lock().await (submit_user_input)
    UI->>CH: action_tx.send(ProvideInput)
    UI->>RS: unlock
    CH->>EX: recv ProvideInput
    EX->>EX: resume run loop

Issues

Issue	Detail
Per-run mutex	Events and controls for one run contend on that run's RunSession; independent runs do not share the lock
Clone on hot path	apply_execution_event clones the full Workflow and WorkflowRunState on each event
Unbounded channels	unbounded_channel() gives no backpressure, so fast event bursts can grow memory
Shared capacity	Active runs across different workflows may exceed the 64-session retention target; provider calls and executable tools wait on shared process budgets
Same-cwd mutations	Write-capable runs sharing a canonical cwd serialize for the whole run; read-only runs and different roots can overlap

UI ↔ backend boundary

The UI is single-threaded (SolidJS in WebView) and spawns no threads. Slow backend work shows up as delayed IPC responses or event delivery:

flowchart LR
    subgraph WebView
        Solid["SolidJS signals<br/>runState, chatLogs"]
    end

    subgraph Rust
        Emit["emit('run-state')"]
        Invoke["invoke commands"]
    end

    Solid -->|"await invoke"| Invoke
    Emit -->|"Tauri event"| Solid

    Invoke -.->|"can block on<br/>sync commands"| Solid
    Emit -.->|"one event per<br/>execution step"| Solid

useRunSession (via context/appProvider/) listens for run-state events and updates run signals. Every execution step (node start, tool call, chat message) emits a full state snapshot, so high-frequency tool loops mean many serializations and IPC round-trips.

Tool concurrency controls

ToolConcurrency lives in the engine tool model (crates/engine/src/tools/config.rs). crates/orchestration/src/tool/registry.rs registers each builtin tool as Shared or Exclusive.

• Per run, ToolPortImpl runs adjacent shared tools in parallel. Each run keeps its own per-tool tokio::sync::Semaphore for ToolConcurrency::Exclusive tools and acquires it before execution (crates/orchestration/src/run/execution/tool_port.rs). This keeps tools such as write, edit, bash, and apply-patch from running concurrently with the same tool name inside a run.
• Across runs, all runs acquire process-wide permits in run/resources.rs: by default 8 concurrent provider invocations and 16 concurrent executable tools. Work beyond these budgets queues.
• For sessions, different workflows may run concurrently, but one workflow owns at most one active run; RunRegistry rejects a second. The registry targets 64 in-process sessions by evicting the oldest inactive session first. It never evicts active sessions, so the count may temporarily exceed 64. Durable run records remain on disk.

Issue severity matrix

Priority	Issue	Symptom
P0	Long blocking filesystem or subprocess tasks	UI commands can wait behind search/find/ast-grep on large repos
P1	Non-Tauri owned runtime path	Extra runtime split if production code accidentally constructs AppBackend around its own runtime
P1	Sync Tauri commands for persistence	Invoke hangs on save/load during heavy I/O
P2	Subagents in nested loop, same task	Parent node frozen while subagent runs
P2	Unbounded event channel	Memory growth on chatty runs
P3	Shared budget sizes are fixed defaults	Provider-specific quotas cannot be tuned independently yet

Recommended directions (not implemented)

None of these is current behavior; each needs implementation and measurement.

A. Keep production on one runtime

• Continue injecting tauri::async_runtime::handle() in desktop.
• Scope any non-Tauri AppBackend construction to tests or explicit helper entry points.

B. Keep blocking work off async workers

// Pattern for local filesystem and subprocess work.
tokio::task::spawn_blocking(move || {
    // fs, WalkDir, Command::output
})
.await?

C. Tune layer concurrency (already parallel)

• Ready nodes in a layer already share one FuturesUnordered pool.
• Follow-ups: fairer permit ordering under load, or isolating per-node cancellation without draining siblings.

D. Async persistence commands

• Convert sync fn Tauri handlers to async fn.
• Use tokio::fs or spawn_blocking for file stores.

E. Backpressure on events

• Replace unbounded_channel with bounded_channel(N).
• Alternatively, batch events before emit to reduce IPC churn.

F. Make shared budgets configurable

• Add per-provider overrides for upstream rate limits.
• Add workload-specific tool budgets if large repositories starve the blocking pool.

Mental model

flowchart TB
    subgraph WhatYouHave["What you have today"]
        direction TB
        W1["1 process"]
        W2["injected Tauri runtime handle"]
        W3["addressable top-level runs<br/>one active run per workflow"]
        W4["1 execution task per run"]
        W5["ready layer work concurrent"]
        W6["shared tools can run in parallel"]
        W7["blocking I/O in spawn_blocking"]
        W8["sync file commands"]
    end

    subgraph WhatYouDontHave["What you don't have"]
        direction TB
        N1["thread pools per subsystem"]
        N2["cross-layer node overlap"]
        N3["async persistence commands"]
        N4["per-provider budget tuning"]
        N5["parallel same-cwd mutation without isolated worktrees"]
    end

    WhatYouHave --- WhatYouDontHave

Key file reference

Concern	File
Runtime handle and backend facade	crates/orchestration/src/backend/mod.rs
Execution loop + spawn	crates/orchestration/src/run/execution/mod.rs, crates/orchestration/src/run/execution/drive/mod.rs
Interactive engine	crates/engine/src/execution/interactive_engine/
Tool concurrency and semaphores	crates/orchestration/src/run/execution/tool_port.rs, crates/orchestration/src/tool/registry.rs
Blocking tool I/O	crates/orchestration/src/tool/dispatch.rs, crates/orchestration/src/tool/runner.rs
Sync file persistence	crates/orchestration/src/adapters/storage/
Tauri command + event bridge	crates/desktop/src/lib.rs
UI event listener	crates/ui/src/context/appProvider/useRunSession.ts