Engineering Blog
Deep dives into GPU observability, inference infrastructure, and accelerator tooling.
plow is open source: a compiler that emits packet streams
Ahead-of-time compilation from checkpoint to packets — and the runtime that executes them
plow is now Apache-2.0. What the compiler emits, what a packet stream is, how one execution model covers NVIDIA and AMD across eight GPUs and several resident models, and the gated agent harness that brings up a new model.
Read article →The cold-start tax
Why vLLM recompiles itself at every boot — and how plow removes it
vLLM and SGLang cold starts are CPU-bound JIT work, not GPU compute. A phase-by-phase breakdown, measured boots and loaded footprints on GH200 and RTX PRO 6000, and how plow’s static compile path takes that tax off the critical path.
Read article →Kimi-K3 day-zero bring-up on 8× MI355X
A 93-layer MLA + KDA hybrid MoE, from “cannot load” to a correct answer
What day-zero support for a 93-layer MLA+KDA hybrid with 896 experts looks like from inside a compiled runtime — the interpreter, counter dataflow, the Lean-gated compiler, the wave/warp switch, tensor parallelism, per-XCD queues and the counter hierarchy they unlock, the correctness bugs a near-right model hides, an end-to-end accuracy check through the served endpoint, and a measured 28.9 ms/token decode taken apart against its bandwidth and protocol-chain floors.
Read article →Long-context decode on 8× AMD MI350X
Per-token latency vs context: plow grows 1.52× where vLLM grows up to 2.9×
From 1k to 128k context, vLLM’s per-token latency grows up to 2.9× — plow’s grows 1.52×. Where the fixed execution plan pays off, crossovers included.
Read article →Gemma-4 decode on RTX PRO 6000 Blackwell
A methodology-first, correctness-gated comparison with vLLM
Batch-1 decode on an RTX PRO 6000 Blackwell: plow fp8 beats vLLM fp8 on Gemma-4-26B MoE from 16k to 128k context — methodology, roofline analysis, and the not-yet-tuned paths included.
Read article →Plow: the compiled inference runtime
Why we moved optimization and orchestration out of the request path
The case for moving orchestration and optimization out of the request path — and a clear separation between measured gains, architectural properties, and open evaluation work.
Read article →Plow SMI: cross-vendor GPU observability and control
Production-grade GPU telemetry, monitoring, and power/clock control for heterogeneous AI infrastructure
A unified, vendor-agnostic platform — built in Rust — that brings first-class GPU telemetry, interactive monitoring, and control into a single, extensible toolkit.
Read article →