Diagram comparison

Internal page — old components live in src/components/legacy/. Delete this page once the new versions are signed off.

Runtime properties (/plow)

OLD
Runtime properties enabled by a compiled execution plan A lightweight Plow runtime at the center connects to six properties: lower host CPU involvement, fewer synchronization boundaries, fast cold starts, multiple resident models, model pipelines, and NVIDIA and AMD support. SERVE-TIME SURFACE plowrt small runtime · compiled plan 01 Host CPU off the hot path less per-step orchestration 02 Fewer sync boundaries less control-plane waiting 03 Fast cold-start path load a prepared artifact 04 Multiple model plans one runtime surface 05 Pipeline stages compose model-to-model work 06 NVIDIA + AMD shared runtime contract
NEW
Cross-section of the Plow serving stack Requests enter a thin plowrt runtime layer, which executes resident compiled model plans — speech recognition, language model, and speech synthesis stages — on NVIDIA and AMD accelerators. The host CPU sits outside the steady-state loop and cold start is an artifact load. INCOMING REQUESTS text image audio plowrt the entire serve-time surface deliberately small — no compile work here CPU host cpu — outside the loop × 1 RESIDENT MODEL PLANS 4 PLAN asr speech → text PLAN llm generate PLAN tts text → speech 5 ARTIFACT 3 cold start = load, not compile 2 no control-plane round-trips per step NVIDIA AMD 6 one compiled-plan contract across vendors
  • 01 Host CPU off the hot path — no per-step orchestration in the steady-state loop
  • 02 Fewer sync boundaries — dependencies are resolved in the plan, not at run time
  • 03 Fast cold-start path — starting a model means loading an artifact, not compiling
  • 04 Multiple resident plans — several models live behind one serving surface
  • 05 Pipeline stages — model-to-model hand-offs stay inside the runtime
  • 06 NVIDIA + AMD — one compiled-plan contract across vendors

Compile / serve boundary (/plow)

OLD
Plow moves optimization work out of request time A model passes through import, optimization, planning, validation, and packaging before deployment. At request time a small runtime loads the prepared artifact and serves it on NVIDIA or AMD accelerators. Measured performance and correctness feed back into the compiler. BEFORE DEPLOYMENT pay the planning cost once 01 Import model + shapes 02 Optimize whole graph 03 Plan target profile 04 Validate correctness gate 05 Package ready artifact COMPILE / SERVE BOUNDARY AT REQUEST TIME keep the serving path small REQUEST input token · image · audio LIGHTWEIGHT RUNTIME plowrt load + execute prepared plan no compile work in the request path ACCELERATOR NVIDIA · AMD same compiled-plan contract
NEW
Plow moves optimization work out of the realtime path A model passes through import, optimization, planning, validation, and packaging before deployment. The prepared artifact crosses the compile/serve boundary into a small runtime that serves live sessions on NVIDIA or AMD accelerators. A timeline shows the realtime contract: fast first token, then a steady per-token cadence inside a p99 latency budget. Measurements feed back into the compiler. BEFORE DEPLOYMENT pay the planning cost once, before any session starts 01 Import model + shapes 02 Optimize whole graph 03 Plan target profile 04 Validate correctness gate 05 Package ready artifact COMPILE / SERVE BOUNDARY prepared artifact AT SESSION TIME — REALTIME LIVE SESSION stream in mic · tokens · frames LIGHTWEIGHT RUNTIME plowrt execute the prepared plan no decisions left on the hot path ACCELERATOR NVIDIA · AMD same compiled-plan contract steady per-token cadence — no scheduler jitter request t₀ first token p99 budget

Compact pipeline (homepage)

OLD
Plow moves model planning before deployment A model checkpoint is analyzed, optimized for a target, validated for correctness, and packaged as a prepared serving artifact. BEFORE DEPLOYMENT REQUEST TIME model checkpoint analyze whole graph optimize target profile validate correctness gate serve prepared artifact deploy
NEW
Plow compiles a transformer model into one prepared serving artifact A transformer block — norm, attention with q k v, residual adds, and mlp, repeated across N layers — is funneled through the plow compiler: analyze, optimize, validate. The result is a single prepared artifact that crosses the deploy boundary onto the small plowrt runtime running on NVIDIA or AMD, which emits a steady stream of tokens. COMPILE — ONCE SERVE — EVERY SESSION MODEL CHECKPOINT norm attention q · k · v + mlp + × N layers plow compiler analyze · optimize · validate deploy one artifact plowrt runs the plan NVIDIA · AMD same compiled-plan contract steady tokens · no scheduler jitter

Fleet control plane (/orchestrator)

OLD
Compiled plow execution plans flow into the orchestrator, which places them across a mixed fleet of NVIDIA and AMD nodes under latency budgets, with telemetry feeding back from the fleet. compiled plans (plow) plan · llm-a plan · llm-b plan · llm-c orchestrator SLO-aware placement node 0 · NVIDIA node 1 · AMD telemetry — InferSight
NEW
A purpose-built GPU OS for real-time inference Model specs with latency budgets enter a GPU operating system control plane, which admits, places, and tracks them across NVIDIA and AMD GPU nodes and a CPU node, each running a plowrt agent. Placement happens ahead of time; telemetry feeds back through InferSight. MODELS — THE FIRST-CLASS UNIT MODEL SPEC voice-asr plan · plowrt-ready SLO p99 · 120 ms MODEL SPEC chat-llm plan · plowrt-ready SLO p99 · 250 ms MODEL SPEC avatar-tts plan · plowrt-ready SLO p99 · 90 ms orchestrator a GPU OS for realtime inference admission fit or reject against budgets placement models → GPUs, ahead of time fleet state measured, not guessed FLEET — MIXED VENDORS, ONE PLANE node 0 · NVIDIA plowrt agent asr gpu0 asr gpu1 llm gpu2 llm gpu3 node 1 · AMD plowrt agent llm gpu0 tts gpu1 tts gpu2 headroom gpu3 node 2 · CPU — pre/post plowrt agent tokenize vad stream io telemetry — InferSight at serve time: nothing left to decide