Issues

• Proposal: load an FTW with fp16 NVFP4 global scales (upcast with a warning) instead of refusing it after #739

  FlashML-org/FreeToken#769 · Filter by author gdevenyi opened on Oct 20, 2026

• Proposal: vLLM scoring fields on /v1/completions (allowed_token_ids, return_tokens_as_token_ids, token-id prompts) for classifier-style clients

  FlashML-org/FreeToken#768 · Filter by author gdevenyi opened on Oct 20, 2026

• /v1/completions: prompt: [] returns 200 with no choices, and both completions paths drop the error code (context_length_exceeded)

  FlashML-org/FreeToken#767 · Filter by author gdevenyi opened on Oct 20, 2026

• An FTW checkpoint can never serve the W4A4 NVFP4 MoE prefill, and the refusal tells you to reconvert with the option that produced it

  FlashML-org/FreeToken#766 · Filter by author gdevenyi opened on Oct 20, 2026

• Two tests fail on a CUDA host since #736 / #697: test_nvfp4_experts_restricted_to_triton_for_swigluoai (sm_120) and test_graph_capture_reuses_warm_offload_cache_before_capture (any Linux host with libcuda)

  FlashML-org/FreeToken#765 · Filter by author gdevenyi opened on Oct 20, 2026

• [Feature] Optional host-resident token-embedding table (pinned, mapped, UVA lookups): ~1 GB of VRAM back for offloaded expert slots

  FlashML-org/FreeToken#754 · Filter by author SUKUNA-AI opened on Oct 19, 2026

• [Perf] Offloaded decode: overlap the expert-miss gather with the same layer's shared expert, event-ordered inside the CUDA graph (bit-exact implementation)

  FlashML-org/FreeToken#753 · Filter by author SUKUNA-AI opened on Oct 19, 2026

• [Perf] Offloaded prefill: keep buffer invalidation off the prefill copy stream and give that stream top priority (short-prompt H2D 21 -> 42 GB/s under Nsight)

  FlashML-org/FreeToken#752 · Filter by author SUKUNA-AI opened on Oct 19, 2026

• [Perf] Offloaded decode: gather missed experts with a few TMA bulk-copy CTAs (cp.async.bulk, .shared::cta) instead of 48 x 1024 LDG threads

  FlashML-org/FreeToken#751 · Filter by author SUKUNA-AI opened on Oct 19, 2026

• [Perf] Layer-major prefill for offloaded MoE: stream each expert layer once per prompt, not once per chunk (qwen3_5_moe)

  FlashML-org/FreeToken#750 · Filter by author SUKUNA-AI opened on Oct 19, 2026

• [Bug] Context overflow is checked only after tokenization in the worker: a streamed request gets 200 plus an in-stream error, and an oversized max_tokens is silently clamped

  FlashML-org/FreeToken#749 · Filter by author SUKUNA-AI opened on Oct 19, 2026

• [Feature] Hybrid radix cache: greedy outputs depend on request history even when cached_tokens is 0 (request: cache flush endpoint / deterministic mode)

  FlashML-org/FreeToken#748 · Filter by author SUKUNA-AI opened on Oct 19, 2026

• [Desktop] weights failed to load: DeepSeek-V4.1-Flash-1017 FP8

  new-model
  FlashML-org/FreeToken#745 · Filter by author pacnet opened on Oct 19, 2026
  1

• hybrid decode halves (23 → 11 tok/s) whenever a foreign thread lands on a pinned spinning CPU-MoE pool CPU; please detect/warn, consider SCHED_FIFO or a yielding spin

  FlashML-org/FreeToken#744 · Filter by author philporter1 opened on Oct 19, 2026

• Regression: first prefill OOMs on RTX 5090 with Qwen3.8-Flash-Next NVFP4 (TMA prefill kernel's first launch allocates ~3.7 GiB)

  buglinux
  FlashML-org/FreeToken#743 · Filter by author void-mckenzie opened on Oct 19, 2026
  4

• perf(moe): fused triton router for sigmoid + selection-bias routing

  perf
  FlashML-org/FreeToken#730 · Filter by author jason-fxz opened on Oct 17, 2026

• Design: how chat logprobs should behave when a reasoning or tool parser is active

  FlashML-org/FreeToken#719 · Filter by author gdevenyi opened on Oct 16, 2026

• Streaming /v1/messages: message_start reports input_tokens 0 and no cache fields

  FlashML-org/FreeToken#718 · Filter by author gdevenyi opened on Oct 16, 2026

• /v1/responses reports usage.output_tokens_details.reasoning_tokens = 0 even when the reply has reasoning

  FlashML-org/FreeToken#717 · Filter by author gdevenyi opened on Oct 16, 2026

• qwen3_coder streaming: a literal <tool_call> in prose hides the rest of the answer

  FlashML-org/FreeToken#716 · Filter by author gdevenyi opened on Oct 16, 2026

• qwen3_coder: non-streaming cuts a tool-call argument at a literal </tool_call>, </function> or <parameter= inside the value

  FlashML-org/FreeToken#715 · Filter by author gdevenyi opened on Oct 16, 2026

• qwen3_coder: a bare <function=NAME> block without <tool_call> is a call plus raw markup in non-streaming, and only text in streaming

  FlashML-org/FreeToken#714 · Filter by author gdevenyi opened on Oct 16, 2026

• qwen3_coder: non-streaming drops the text between two tool calls (streaming keeps it)

  FlashML-org/FreeToken#713 · Filter by author gdevenyi opened on Oct 16, 2026

• qwen3 reasoning parser: streamed content starts with "\n\n" after </think> and streamed reasoning keeps its trailing newline; non-streaming strips both

  FlashML-org/FreeToken#712 · Filter by author gdevenyi opened on Oct 16, 2026

• Reasoning parsers re-enter reasoning on a <think> after </think> when streaming, and delete it when not (qwen3, glm, minimax, deepseekv32, gemma4, minimax_m3)

  FlashML-org/FreeToken#711 · Filter by author gdevenyi opened on Oct 16, 2026