tool-call-parsing.mdx

title	Tool Call Parsing (Dynamo)
subtitle	Connect Dynamo to external tools and services using Dynamo's built-in tool call parsers

Tool calling lets Dynamo connect models to external tools and services by returning function arguments for your application to execute. Requests use the tool_choice and tools parameters. This page covers Dynamo-native parsing; if your model is not listed, use engine fallback, which also explains valid combinations of --dyn-tool-call-parser, --dyn-chat-processor, and --dyn-reasoning-parser.

Configure Tool Calling

Select a `--dyn-tool-call-parser` that matches your model based on the list below. The backend can be SGLang, TensorRT-LLM, vLLM, or another installed backend.

To inspect the available worker flags, run:

```bash
python -m dynamo.<backend> --help
```

The following example starts an SGLang worker for Qwen3.5:

```bash
python -m dynamo.sglang \
  --model Qwen/Qwen3.5-4B \
  --dyn-tool-call-parser qwen3_coder \
  --reasoning-parser qwen3 \
  --dyn-reasoning-parser qwen3
```

<Info>
If the model supports tool calling but its default chat template does not, add `--custom-jinja-template /path/to/template.jinja` to the worker command. The template must be readable inside the container.
</Info>

<Tip>
If the model emits reasoning content that should be separated from normal output, see [Reasoning Parsing (Dynamo)](reasoning-parsing.md) for supported `--dyn-reasoning-parser` values.
</Tip>

Start the Dynamo frontend in another terminal:

```bash
python -m dynamo.frontend
```

Wait for the frontend health endpoint to return a successful response:

```bash
curl --fail http://localhost:8000/health
```

Send an OpenAI-compatible chat completion request with the available tools:

```bash
curl -s http://localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Qwen/Qwen3.5-4B",
    "messages": [
      {"role": "user", "content": "What is the weather in San Francisco and New York?"}
    ],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get the current weather for a location.",
        "parameters": {
          "type": "object",
          "properties": {"location": {"type": "string"}},
          "required": ["location"]
        }
      }
    }],
    "tool_choice": "auto"
  }'
```

Dynamo parses the model output and returns OpenAI-compatible `tool_calls`.

<Accordion title="View an example response">
  ```json
  {
    "id": "chatcmpl-b415caad-9be0-4d9e-ac6d-9d23bfe57703",
    "choices": [
      {
        "index": 0,
        "message": {
          "role": "assistant",
          "content": null,
          "reasoning_content": "The user is asking about the weather in two cities: San Francisco and New York. I need to call the get_weather function for each city. I'll make two separate function calls to get the weather information for both locations.\n",
          "tool_calls": [
            {
              "id": "call-56223a95-3d14-4433-a94e-011f106c0e40",
              "type": "function",
              "function": {
                "name": "get_weather",
                "arguments": "{\"location\":\"San Francisco\"}"
              }
            },
            {
              "id": "call-d5b5772b-6b0c-4120-ad01-623278a937fe",
              "type": "function",
              "function": {
                "name": "get_weather",
                "arguments": "{\"location\":\"New York\"}"
              }
            }
          ]
        },
        "finish_reason": "tool_calls",
        "logprobs": null
      }
    ],
    "created": 1778653281,
    "model": "Qwen/Qwen3.5-4B",
    "object": "chat.completion"
  }
  ```
</Accordion>

<Info>
If a tool call is parsed incorrectly, add `"logprobs": true` to one reproduction request and share the response. See [Troubleshooting Tool Calls](troubleshooting-tool-calls.md) for what to capture when reporting an issue.
</Info>

Custom Freeform Tools

Chat Completions accepts custom tools whose input is freeform text, such as code or a patch. Use the existing tool parser for your model and declare a custom tool:

{
  "tools": [{
    "type": "custom",
    "custom": {
      "name": "run_python",
      "description": "Execute Python source code.",
      "format": {"type": "text"}
    }
  }],
  "tool_choice": {"type": "custom", "custom": {"name": "run_python"}}
}

Returned message.tool_calls entries use type: "custom" with custom.name and custom.input. Your application executes the tool and replays the assistant call followed by a role: "tool" result with its matching tool_call_id. Ordinary function tools can appear in the same request.

Responses supports the same capability using its own custom-tool API format. Custom grammar formats are rejected before generation. Streaming custom input is buffered until the call completes, then emitted as text in custom.input. This streamed custom delta is a Dynamo extension to the published OpenAI Chat streaming schema. The optional tool_call_dispatch events currently cover function tools only; custom calls arrive through ordinary data chunks.

Function-oriented engines generate a JSON wrapper with a string input field. If a completed call does not match that wrapper, Dynamo returns the raw argument text as custom.input, preserving other calls, content, and usage. Applications must validate tool input before executing it. If the token limit interrupts the wrapper, Dynamo preserves finish_reason: "length" without emitting the truncated input.

Supported Tool Call Parsers

Choose a model family, then expand the matching model option to see its parser name and configuration details.

The Upstream name column shows where the vLLM or SGLang parser name differs from Dynamo’s — relevant when using --dyn-chat-processor vllm or sglang (see Parser Engine Fallback). A blank upstream column means the same name works everywhere. Dynamo-only means no upstream parser exists for this format.

**Parser:** `kimi_k2`

    **Upstream name:** Same name across Dynamo, vLLM, and SGLang

    **Notes:** Pair with `--dyn-reasoning-parser kimi` or `kimi_k25`. For Kimi K2.5 thinking models, use `kimi_k25` so `<think>` blocks and tool calls are parsed from the same response. See [Reasoning Parsing (Dynamo)](reasoning-parsing.md).
  </Accordion>
</AccordionGroup>

**Parser:** `minimax_m2`

    **Upstream name:** vLLM: `minimax_m2`

    **Notes:** XML `<minimax:tool_call>`
  </Accordion>

  <Accordion title="MiniMax M3">
    **Parser:** `minimax_m3`

    **Upstream name:** vLLM: `minimax_m3`

    **Notes:** MiniMax namespace-token XML
  </Accordion>
</AccordionGroup>

**Parser:** `deepseek_v4`

    **Upstream name:** vLLM: `deepseek_v4`; SGLang: `deepseekv4`

    **Notes:** DSML tags (`<｜DSML｜tool_calls>...`). Aliases: `deepseek-v4`, `deepseekv4`
  </Accordion>

  <Accordion title="DeepSeek V3 and DeepSeek R1-0528+">
    **Parser:** `deepseek_v3`

    **Upstream name:** SGLang: `deepseekv3`

    **Notes:** Special Unicode markers
  </Accordion>

  <Accordion title="DeepSeek V3.1">
    **Parser:** `deepseek_v3_1`

    **Upstream name:** <Badge intent="note" minimal>Dynamo-only</Badge>

    **Notes:** JSON separators
  </Accordion>

  <Accordion title="DeepSeek V3.2+">
    **Parser:** `deepseek_v3_2`

    **Upstream name:** <Badge intent="note" minimal>Dynamo-only</Badge>

    **Notes:** DSML tags (`<｜DSML｜function_calls>...`)
  </Accordion>
</AccordionGroup>

**Parser:** `qwen3_coder`

    **Upstream name:** Same name across Dynamo, vLLM, and SGLang

    **Notes:** XML `<tool_call><function=...>`
  </Accordion>

  <Accordion title="Qwen2.5-*, QwQ-32B, Qwen3-Instruct, Qwen3-Think, and NousHermes-2/3">
    **Parser:** `hermes`

    **Upstream name:** vLLM: `qwen2_5`; SGLang: `qwen25` (for Qwen models)

    **Notes:** `<tool_call>` JSON
  </Accordion>
</AccordionGroup>

**Parser:** `glm47`

    **Upstream name:** <Badge intent="note" minimal>Dynamo-only</Badge>

    **Notes:** XML `<arg_key>/<arg_value>`
  </Accordion>
</AccordionGroup>

**Parser:** `nemotron_deci`

    **Upstream name:** <Badge intent="note" minimal>Dynamo-only</Badge>

    **Notes:** `<TOOLCALL>` JSON
  </Accordion>

  <Accordion title="Nemotron-Nano">
    **Parser:** `nemotron_nano`

    **Upstream name:** <Badge intent="note" minimal>Dynamo-only</Badge>

    **Notes:** Alias for `qwen3_coder`
  </Accordion>
</AccordionGroup>

**Parser:** `gemma4`

    **Upstream name:** vLLM: `gemma4`

    **Notes:** Custom non-JSON grammar with `<\|"\|>` string delimiters and `<\|tool_call>...<tool_call\|>` markers. Aliases: `gemma-4`. Pair with `--dyn-reasoning-parser gemma4` and `--custom-jinja-template examples/chat_templates/gemma4_tool.jinja`.
  </Accordion>
</AccordionGroup>

**Parser:** `beam`

    **Upstream name:** none (Dynamo-native, served by the `dynamo-parsers-v2` unified family)

    **Notes:** Channel grammar: reasoning in `<\|channel\|>analysis<\|message\|>...<\|end\|>`, the answer in `<\|channel\|>final<\|message\|>...<\|return\|>`, and one `<\|start\|>assistant to=NAME<\|channel\|>analysis <\|constrain\|>xml<\|message\|>...<\|call\|>` message per tool call with `<rfl:parameter name="KEY">VALUE</rfl:parameter>` arguments typed from the tool schema. Pair with `--dyn-reasoning-parser beam`; both flags are required and the pair routes to the unified parser without `DYN_PARSER_VERSION=2`. The route accepts the request field `preserve_thinking` (supplied prior assistant reasoning always stays in the rendered history; `true` and omitted are the same, `false` is accepted and has no effect — omit `reasoning_content` to drop it) and defaults `allow_missing_thinking=true` so history turns without reasoning render an empty analysis block instead of failing and `chat_template_kwargs.builtin_tools` (names resolved against `tools` for rendering only).

    **Per-turn grammar:** on the Beam route the frontend holds every chat turn to this channel grammar with a structural tag (a non-empty analysis channel, then either tool calls whose `rfl:parameter` values are held to their JSON-schema types or the final answer), mirroring the model builder's SGLang serving layer; `required` and named tool choices reuse the same tag, so they are enforced on the Harmony wire. A `response_format` (`json_object` or `json_schema`) is held inside the final channel behind the analysis, so the response keeps its reasoning and the channel structure the parser expects; a forced tool choice ignores `response_format`, and the `nvext` guided-decoding fields (`guided_json`, `guided_regex`, ...) stay bare constraints from the first token. Set `DYN_BEAM_TURN_GRAMMAR=0` on the frontend to disable it for automatic tool choice (forced choices keep theirs; `response_format` then falls back to the bare JSON constraint). A tool declared `strict: true` whose schema is outside OpenAI's strict subset is served as non-strict on this route (logged as a warning) instead of the HTTP 400 other routes return; the grammar holds every typeable tool either way.
  </Accordion>
</AccordionGroup>

**Parser:** `harmony`

    **Upstream name:** <Badge intent="note" minimal>Dynamo-only</Badge>

    **Notes:** Harmony channel format
  </Accordion>
</AccordionGroup>

**Parser:** `phi4`

    **Upstream name:** vLLM: `phi4_mini_json`

    **Notes:** `functools[...]` JSON
  </Accordion>
</AccordionGroup>

**Parser:** `pythonic`

    **Upstream name:** Same name across Dynamo, vLLM, and SGLang

    **Notes:** Python-list tool syntax
  </Accordion>

  <Accordion title="Llama 3 / 3.1 / 3.2 / 3.3 Instruct">
    **Parser:** `llama3_json`

    **Upstream name:** Same name across Dynamo, vLLM, and SGLang

    **Notes:** `<\|python_tag\|>` tool syntax
  </Accordion>
</AccordionGroup>

**Parser:** `mistral`

    **Upstream name:** Same name across Dynamo, vLLM, and SGLang

    **Notes:** `[TOOL_CALLS]...[/TOOL_CALLS]`
  </Accordion>
</AccordionGroup>

**Parser:** `jamba`

    **Upstream name:** <Badge intent="note" minimal>Dynamo-only</Badge>

    **Notes:** `<tool_calls>` JSON
  </Accordion>
</AccordionGroup>

**Parser:** `default`

    **Upstream name:** <Badge intent="note" minimal>Dynamo-only</Badge>

    **Notes:** Empty JSON config (no start/end tokens). Prefer a model-specific parser for production use.
  </Accordion>
</AccordionGroup>

For parser and backend combinations that support them, Dynamo enables **xgrammar structural tags** by default so guided decoding matches the parser's tool-call format at token granularity. See [Structural Tag (Guided Decoding for Tool Calls)](structural-tag.md) for the activation policy, compatibility fallbacks, and opt-out settings.