LLM 4B effective (8B with per-layer embeddings) 128k native context bf16 Gemma Terms of Use

Gemma 4 E4B

Google's small instruction-tuned Gemma 4, the text model of the voice-agent stack.

google/gemma-4-E4B-it

Specs

Developer
Google
Parameters
4B effective (8B with per-layer embeddings)
Task
LLM
Context
131,072 tokens upstream, 8,192 served by our builds
Precisions
bf16
License
Gemma Terms of Use
Checkpoint
google/gemma-4-E4B-it
Updated

Measured A specific workload and configuration against vLLM.

On one H100 in BF16 against vLLM 0.29, with 1,000 input tokens, 128 output tokens and 32 concurrent requests: 1.11 times total throughput, 0.74 times p99 time to first token and 0.93 times p99 time per output token. Greedy decoding, two repeats.

Run of . Settings and every row. More results on benchmarks.

Where it runs Each build is tuned for one part. plowrt runs it beyond that.

Tuned for
NVIDIA H100 SXM5
Runs as-is on H100 NVL, H200 SXM
Elsewhere plowrt still runs the model on other NVIDIA, AMD and Apple hardware, without this build’s tuning: expect lower throughput.

Run it An OpenAI-compatible server on your own machine.

plowrt 0.1.3 Install, pull, serve, call 
  1. curl -fsSL https://pdist.infervisor.ai/install.sh | sh
    Install 
  2. export HF_TOKEN=<your Hugging Face token>   # gated checkpoint
plowrt load gemma-4-e4b --fetch-weights
    Pull with weights 
  3. plowrt serve --model gemma-4-e4b --port 8080
    Serve 
  4. curl -s http://127.0.0.1:8080/v1/chat/completions \
  -H 'content-type: application/json' \
  -d '{"model":"gemma-4-e4b","messages":[{"role":"user","content":"Hello"}]}'
    Try it

Builds plowrt load picks the closest fit for your machine.

NVIDIA H100 SXM5 80 GiB, also H100 NVL, H200 SXM bf16 1 GPU 8k Published 
export HF_TOKEN=<your Hugging Face token>   # gated checkpoint
plowrt load infervisor/gemma-4-e4b:sm90a-h100-tp1 --fetch-weights
This build only
Target
sm90a-h100-tp1 , tuned on NVIDIA H100 SXM5; runs as-is on H100 NVL, H200 SXM; other hardware runs it untuned
Checkpoint
google/gemma-4-E4B-it
Published
29 Oct 2026
Compiled by
plowc 0.1.3 , from this recipe

About

Gemma 4 E4B-it is the text model plow co-serves with an ASR and a TTS model on one H100 for voice agents: short prompts, many concurrent sessions, logprobs on. The H100 build runs decode projections through cuBLASLt inside captured graphs and caps a single request at 2048 rows per launch so an 8192-token context stays a packed launch of a few requests.

The build is gated against HF transformers in bf16 (top-1 agreement above 0.98, mean KL below 2e-3) before it is published.

Questions

What hardware does Gemma 4 E4B run on?
Tuned builds: NVIDIA H100 SXM5. plowrt also runs them on H100 NVL, H200 SXM.
Does Gemma 4 E4B fit on one GPU?
Yes. The sm90a-h100-tp1 build runs on one NVIDIA H100 SXM5.
Can I run Gemma 4 E4B on AMD?
Not with a tuned build yet. Today's builds are for NVIDIA.
What context length does Gemma 4 E4B support?
The upstream model supports 131,072 tokens. Our builds serve up to 8,192.
What license is Gemma 4 E4B under?
Gemma Terms of Use. Check the checkpoint's license on Hugging Face before commercial use.
How do I run Gemma 4 E4B?
Install plowrt, then run export HF_TOKEN=<your Hugging Face token> # gated checkpoint plowrt load gemma-4-e4b --fetch-weights and plowrt serve --model gemma-4-e4b --port 8080. It serves an OpenAI-compatible API.

Related models