What hardware does Gemma 4 31B run on?

Tuned builds: NVIDIA H100 SXM5. plowrt also runs them on H100 NVL, H200 SXM.

Does Gemma 4 31B fit on one GPU?

Yes. The sm90a-h100-tp1-fp8 build runs on one NVIDIA H100 SXM5.

Can I run Gemma 4 31B on AMD?

Not with a tuned build yet. Today's builds are for NVIDIA.

What context length does Gemma 4 31B support?

The upstream model supports 262,144 tokens. Our builds serve up to 16,384.

What license is Gemma 4 31B under?

Gemma Terms of Use. Check the checkpoint's license on Hugging Face before commercial use.

How do I run Gemma 4 31B?

Install plowrt, then run export HF_TOKEN= # gated checkpoint plowrt load gemma-4-31b --fetch-weights and plowrt serve --model gemma-4-31b --port 8080. It serves an OpenAI-compatible API.

LLM 31B 256k native context fp8 Gemma Terms of Use

Gemma 4 31B

The largest dense Gemma 4, instruction tuned, in bf16 and FP8 on NVIDIA.

google/gemma-4-31B-it fp8 builds serve RedHatAI/gemma-4-31B-it-FP8-dynamic

Specs

Developer: Google
Parameters: 31B
Task: LLM
Context: 262,144 tokens upstream, 16,384 served by our builds
Precisions: fp8
License: Gemma Terms of Use
Checkpoint: google/gemma-4-31B-it
Updated: 29 Oct 2026

Where it runs Each build is tuned for one part. plowrt runs it beyond that.

Tuned for

NVIDIA H100 SXM5

Runs as-is on H100 NVL, H200 SXM

Elsewhere plowrt still runs the model on other NVIDIA, AMD and Apple hardware, without this build’s tuning: expect lower throughput.

Run it An OpenAI-compatible server on your own machine.

plowrt 0.1.3 Install, pull, serve, call

curl -fsSL https://pdist.infervisor.ai/install.sh | sh

Install

export HF_TOKEN=<your Hugging Face token>   # gated checkpoint
plowrt load gemma-4-31b --fetch-weights

Pull with weights

plowrt serve --model gemma-4-31b --port 8080

Serve

curl -s http://127.0.0.1:8080/v1/chat/completions \
  -H 'content-type: application/json' \
  -d '{"model":"gemma-4-31b","messages":[{"role":"user","content":"Hello"}]}'

Try it

Builds `plowrt load` picks the closest fit for your machine.

NVIDIA H100 SXM5 80 GiB, also H100 NVL, H200 SXM fp8 1 GPU 16k Published

export HF_TOKEN=<your Hugging Face token>   # gated checkpoint
plowrt load infervisor/gemma-4-31b:sm90a-h100-tp1-fp8 --fetch-weights

This build only

Target: sm90a-h100-tp1-fp8 , tuned on NVIDIA H100 SXM5; runs as-is on H100 NVL, H200 SXM; other hardware runs it untuned
Checkpoint: RedHatAI/gemma-4-31B-it-FP8-dynamic
Published: 29 Oct 2026
Compiled by: plowc 0.1.3 , from this recipe

About

The bf16 label serves Google’s checkpoint directly; the FP8 label serves RedHatAI’s FP8-dynamic checkpoint, which keeps the same tokenizer and chat template. Both run on a single H100 SXM5 with the prefix cache on.

Questions

What hardware does Gemma 4 31B run on?: Tuned builds: NVIDIA H100 SXM5. plowrt also runs them on H100 NVL, H200 SXM.
Does Gemma 4 31B fit on one GPU?: Yes. The sm90a-h100-tp1-fp8 build runs on one NVIDIA H100 SXM5.
Can I run Gemma 4 31B on AMD?: Not with a tuned build yet. Today's builds are for NVIDIA.
What context length does Gemma 4 31B support?: The upstream model supports 262,144 tokens. Our builds serve up to 16,384.
What license is Gemma 4 31B under?: Gemma Terms of Use. Check the checkpoint's license on Hugging Face before commercial use.
How do I run Gemma 4 31B?: Install plowrt, then run export HF_TOKEN=<your Hugging Face token> # gated checkpoint plowrt load gemma-4-31b --fetch-weights and plowrt serve --model gemma-4-31b --port 8080. It serves an OpenAI-compatible API.