halogen™ is the fastest way to run Qwen3.8-Flash-Next on AMD Strix Halo, and it is built for agents: deep into a long session, each turn starts answering in about 1.6 seconds.
Every kernel is written for this one GPU and this one model family. No general-purpose runtime, no portability layer, no fallback path. That is why it can do things a general engine cannot, and why it runs on exactly one piece of silicon.
| 0.18.0, the default configuration | halogen-flash |
|---|---|
| decode | up to 94 tok/s on repetitive output · 73 writing code · 54 prose |
| prefill: 8,192 / 32,768 / 131,072-token prompt | 1,821 / 1,740 / 1,675 tok/s |
| an agent turn, 30,000 to 96,000 tokens of context | 4.2 s (first token in 1.6 s) |
| the first turn of a 34,000-token agent prompt, cold | 21 s |
| decode: editing code / an agent's tool call / six other languages | 81 / 71 / 52 tok/s |
What each row is. The agent turns are three sessions of a coding agent
fixing a bug in a Python repository: each turn sends the whole conversation
back with the model's last tool call and a tool result of 800 to 3,000
tokens appended, and the time runs from the request to the last byte of the
answer, on the client's clock (the median of 115 turns, about 200 tokens of
answer each). Decode is measured on the kind of text each row names. The
best case is five short prompts whose answer repeats itself ("print the word
red ten times" and four like it), shown so it can be compared like for like
with other figures of that kind. Prefill is the image's own sweep, on a
synthetic prompt; real text through the chat template runs about 8 to 12%
slower, which is why the cold first turn takes 21 s for 34,000 tokens.
All of it is the published 0.18.0 image at its defaults (the v2 checkpoint, the prompt cache on, greedy, thinking off) on an AMD Ryzen AI Max+ 395 with 128 GB, at about 85 W of sustained package power. Measured has every number for both checkpoints. The prefill rows, and a decode mean over ten prompt shapes, reproduce with one command from the image (how); the agent turns and the decode by kind of text are measured from the client side, over the same OpenAI-compatible API you use. Prefill barely decays with depth: 131,072 tokens run at 96% of the 32,768 rate.
Precision: 4.16 bpw across the trunk, the experts and the draft head of
the default checkpoint (v2), or 5.25 bpw across all 179.55B parameters with
the FP8 n-gram lookup table counted in. Both are measured from the files
rather than quoted from a format name. docs/QUANT.md gives
the breakdown by tensor family, so it can be checked with arithmetic rather
than taken on trust.
At temperature 0, output is byte-identical to serial greedy decode. Speculation here is a pure speed optimization, verified on every release, not a quality trade. Since 0.6.0 there are two draft sources, the model's own draft head and the request's own text (prompt lookup), and the guarantee covers both.
Since 0.7.0 the engine also opens a llama.cpp GGUF of this model directly:
point it at the file you already have (unsloth's UD-IQ4_XS, say) and it
runs on these kernels, with the same speculation and the same identity
guarantee, and no conversion step. See Bring your own
GGUF for which files, and for the numbers.
There is a Discord for questions, for numbers from your own box, and for comparing setups. Bugs and regressions go to issues, where the changelog can credit them. See Community.
The full documentation is the docs/ tree, one page per task. It is
also hosted with search at
peonist-ai.github.io/halogen-flash-server.
For an AI agent: llms.txt maps every page in one fetch, and
AGENTS.md is the short playbook for deploying, driving or
debugging the server.
| Quickstart | first run on a Strix Halo box, weights included |
| Give it a machine of its own | what the server holds; recipes if you must share it |
| API reference | every endpoint, conventions, status codes |
| Clients | Codex (Responses API) · Claude Code (Messages API) · agent harnesses · sessions: keep, fork, join |
| Requests | sampling · images · token budgets · as a classifier |
| Configuration | cache modes, the KV pool, disk cache, 1M context, attention budget, API key, request records |
| Checkpoints | precision · bring your own GGUF · measuring a checkpoint · QUANT.md |
| Numbers | measured · quality: what is measured, and what is not |
| NPU | small models on the NPU · host details |
| Troubleshooting | will not start, hangs at pool reservation, crawls on long prompts, Fedora/SELinux, the host settings measured on |
| Reference | every flag · classify endpoint · what this release is not |
mkdir -p ~/halogen-models
podman run --rm -p 8731:8731 \
--device /dev/kfd --device /dev/dri --group-add keep-groups \
--ipc=host --ulimit memlock=-1:-1 \
-e HALOGEN_DOWNLOAD=peonist-ai/halogen-qwen3.8-flash-next \
-v ~/halogen-models:/models \
ghcr.io/peonist-ai/halogen-flash-server:0.18.4One condition first: set the BIOS carve-out (UMA frame buffer) to its minimum, not Auto: this engine wants host RAM, not dedicated VRAM. The Quickstart covers that, Docker, the compose file, and upgrading from an older checkpoint.
- Discord for questions, setup help, numbers from your own machine, and comparisons with other runtimes. If you have run a benchmark against this server, post the command and the curve. A result from a box we do not own is worth more to us than one from ours.
- Issues for bugs and regressions. CONTRIBUTING.md says what to include. The changelog credits the report that drove each fix.
- Security issues go to the maintainers directly, not to Discord or a public issue.
The engine is distributed under the terms in LICENSE.md. Third-party components and their licenses are listed in THIRD-PARTY-NOTICES.md. Model weights are licensed separately by their original authors.
"halogen" and "Peonist" are trademarks of Peonist, LLC (U.S. application pending). TRADEMARKS.md says how the names may be used; referring to the project, running it, and publishing numbers about it need no permission.