Tabby turned file contents into numbers

· 10 min read
Contents

Assumes you've pointed a coding agent at a local model server with OpenAI-style tool calls.

TL;DR — TabbyAPI's tool-call parser for the qwen3_coder format runs every argument through strip() and json.loads without looking at the tool's schema. A string argument whose text is valid JSON comes back as a number, boolean, null, list or object, and every argument loses its leading and trailing whitespace. For a coding agent that means write and edit calls with corrupted content. Upstream main still has the same code as of 2026-10-16. A three-file local patch that hands the request's tool schemas to the parser fixes it.

The docstring on TabbyAPI's coerce_param_value promises something modest: strip the newlines the chat template puts around a tool-call argument, then try to parse it. The first line of the body is raw.strip(), which takes every leading and trailing space along with those newlines. The next is json.loads. Between them, those two lines decide what type your argument is. The tool's schema gets no vote.

The lucky part is when this turned up: before the engine had run a single agent task. ExLlamaV3 behind TabbyAPI had just measured 16 to 23 percent faster decode than llama.cpp on the same 27B model from about 25k tokens of context up. Prefill was 12 to 28 percent slower, with one run per depth and different quantizations on each side. Faster decode was worth a benchmark. First, a small probe checked that tool calls came back intact.

The probe: five strings that look like JSON#

The probe, s1s2.py, declares a tool write_file(path: string, content: string) and asks the model to call it with content set to exactly one of five values: 123, true, null, [1, 2], {"a": 1}. Each value runs once with streaming off and once with it on. All five are legal strings. All five are also legal JSON.

On stock Tabby, all ten calls failed. "123" arrived as the integer 123, "true" as true, "null" as null, the list as a list, the object as an object. Streaming made no difference. The schema said string. The response disagreed every time.

coerce_param_value: strip, then json.loads#

The function lives in endpoints/OAI/utils/toolcall_formats/common.py. This is the copy from the pinned container image:

def coerce_param_value(raw: str) -> any:
    """Coerce a raw parameter value string to the appropriate Python type.

    Strategy (safe, no eval()):
      1. Strip leading/trailing newlines (official template emits \\n
         after opening tag and before closing tag).
      2. Try json.loads — handles objects, arrays, numbers, bools, null.
      3. Fall back to plain string.
    """
    # Strip template-inserted newlines around values
    stripped = raw.strip()

    # Empty string
    if not stripped:
        return ""

    # Try JSON parse (handles objects, arrays, numbers, booleans, null)
    try:
        return json.loads(stripped)
    except (json.JSONDecodeError, ValueError):
        pass

    # Fall back to string — never eval()
    return stripped

It keeps the promise about eval(). The newline promise is looser. Called with no argument, str.strip() removes all leading and trailing whitespace, so indentation goes out with the template's newline.

Loaded out of the image's file and fed seven values directly, the function does this:

'123'                            -> 123 (int)
'true'                           -> True (bool)
'null'                           -> None (NoneType)
'[1, 2]'                         -> [1, 2] (list)
'{"a": 1}'                       -> {'a': 1} (dict)
'x.txt'                          -> 'x.txt' (str)
'\n    indented\n  [1, 2]\n\n'   -> 'indented\n  [1, 2]' (str)

The last row stands in for a file body. The four spaces of indentation and both trailing newlines are gone before any JSON is attempted. One caveat on evidence: the S2 probe saw the type coercion live through the server. The whitespace loss above is the function run on its own, not observed end to end.

The qwen3_coder parser never sees the schema#

Coercion without a schema is guessing, and the qwen3_coder parser has no schema. Its signature is parse_toolcalls(text: str), and the dispatcher in tools.py calls it with the generated text and nothing else. The request's tools array, the JSON Schema that says content is a string, sits in params.tools in chat_completion.py and never makes it down the call chain. Inside the parser:

for pm in _PARAM.finditer(func_body):
    key = pm.group(1).strip()
    val = pm.group(2).strip()
    val = coerce_param_value(val)
    args[key] = val

The value is stripped twice, once here and once inside the coercer, in case the first one missed a spot.

qwen3_coder is not the only caller. Upstream, glm4_5.py, hy3.py and minimax_m2.py call the same function, and no format parser's parse_toolcalls accepts tools at all. Only qwen3_coder was tested here; the other three are a reading of the source.

Upstream has not changed it. On 2026-10-16, main at commit 63860e6 carries a common.py and a qwen3_coder.py byte-identical to the image's, and common.py last changed on 2026-09-21. An open pull request, #462, streams partial tool-call arguments for this format and guarantees the result is byte-identical to the end-of-stream parse. It streams string values live only once they "provably cannot parse as a JSON literal, so coerce_param_value() cannot reinterpret them later." Someone is carefully engineering around the coercion and keeping it as the contract. A search of the tracker for coerce_param_value turns up no report of it, and none has been filed from this work.

What a coding agent receives#

Two hand-written tool calls in the template's layout show the damage: a newline after each <parameter=…> tag, the value, one newline before the closing tag. The first writes a one-line config.json:

<tool_call>
<function=write_file>
<parameter=path>
config.json
</parameter>
<parameter=content>
{"debug": true}

</parameter>
</function>
</tool_call>

The second writes return x (four spaces, the code, a trailing newline) to util.py. Both go through the image's parser, then through the patched one described below:

== src-orig call1
{"path": "config.json", "content": {"debug": true}}
== src-orig call2
{"path": "util.py", "content": "return x"}
== patched call1
{"path": "config.json", "content": "{\"debug\": true}\n"}
== patched call2
{"path": "util.py", "content": "    return x\n"}

Stock Tabby hands the client an object where the schema promised a string. Whatever the client does next (reject the call, or serialize the object back into text its own way), the text the model generated never reaches the file. The second call loses its indentation and its trailing newline. strip() works on the ends of the whole value, so only the first line's indentation goes, and the first line is exactly where an edit snippet cut from the middle of a function carries it. An exact-match edit tool's search string goes through the same function.

JSON-shaped files change type: configs, fixtures, a package.json. Everything else loses its edges.

The patch: hand the schema to the parser#

The fix moves the schema to where the parsing happens. It touches three files. chat_completion.py passes params.tools into _parse_tool_calls. tools.py forwards it to any parser whose parse_toolcalls accepts a tools argument, checked with inspect.signature, so every other format parser runs as before. qwen3_coder.py uses it:

+def _trim_template_newlines(raw: str) -> str:
+    """The template puts one newline after <parameter=...> and one before </parameter>; drop only those."""
+    if raw.startswith("\n"):
+        raw = raw[1:]
+    if raw.endswith("\n"):
+        raw = raw[:-1]
+    return raw
+
+
+def parse_toolcalls(text: str, tools=None) -> list[ToolCall]:
+    string_params = _string_params(tools) if tools else None
     # If there are outer <tool_call> wrappers, unwrap them; otherwise use the whole text
     segments: list[tuple[str, str]] = []  # (raw, inner)
     outer_matches = list(_OUTER.finditer(text))
@@ -61,6 +94,10 @@
             args: dict[str, any] = {}
             for pm in _PARAM.finditer(func_body):
                 key = pm.group(1).strip()
+                if string_params is not None and key in string_params.get(func_name, ()):
+                    # Schema says string: keep the text as written (no JSON coercion, no strip)
+                    args[key] = _trim_template_newlines(pm.group(2))
+                    continue
                 val = pm.group(2).strip()
                 val = coerce_param_value(val)
                 args[key] = val

Above that hunk, _string_params builds the lookup. A parameter counts as a string only if its schema allows string and none of integer, number, boolean, array or object. anyOf: [string, null] qualifies. A string-or-integer union keeps the old guessing, because without more information there is no right answer for "42". _trim_template_newlines removes exactly one newline at each end, the two the template adds, and leaves the rest alone. Requests without tools, and every non-string parameter, still go through coerce_param_value unchanged.

Verifying it, and the hole in the probe#

The probe has a blind spot of its own. Its pass condition in s1s2.py:

good = isinstance(c, str) and c.strip() == val

The probe strips too. It catches a type change and cannot see whitespace loss at all, so a passing S2 says nothing about indentation. That needed a separate check.

test-parser.py is that check, run inside the pinned image against the patched files. It parses one write call with five parameters: path holding 123, an indented two-line content with a trailing newline, an integer n, a boolean flag, and an opt typed string-or-null holding the text null. With the schema passed in, path stays "123", content keeps " indented\n [1, 2]\n", n becomes 42, flag becomes true, and opt stays the string "null". Tabby's own unittest run shows the same six errors before and after the patch; the image ships without pytest.

Then the live probe against the patched server, same model:

S1 {"stream": false, "finish": "tool_calls", "content": null, "reasoning_len": 44, "tool_calls": [{"name": "get_time", "arguments": "{}"}]}
S1 {"stream": true, "finish": "tool_calls", "content": "", "reasoning_len": 44, "tool_calls": [{"name": "get_time", "arguments": "{}"}], "first_tool_delta_s": 0.46, "total_s": 0.46}
S2 val='123'      stream=False finish=tool_calls content='123' type=str OK
S2 val='123'      stream=True  finish=tool_calls content='123' type=str OK
S2 val='true'     stream=False finish=tool_calls content='true' type=str OK
S2 val='true'     stream=True  finish=tool_calls content='true' type=str OK
S2 val='null'     stream=False finish=tool_calls content='null' type=str OK
S2 val='null'     stream=True  finish=tool_calls content='null' type=str OK
S2 val='[1, 2]'   stream=False finish=tool_calls content='[1, 2]' type=str OK
S2 val='[1, 2]'   stream=True  finish=tool_calls content='[1, 2]' type=str OK
S2 val='{"a": 1}' stream=False finish=tool_calls content='{"a": 1}' type=str OK
S2 val='{"a": 1}' stream=True  finish=tool_calls content='{"a": 1}' type=str OK
S1S2 PASS

Ten for ten, as strings, streaming off and on.

Then the benchmark shrugged#

With arguments intact, the engine got its benchmark: the same model through OpenCode, three runs per cell, against llama.cpp with MTP on the IQ4_XS quantization of the same model.

The sampler needed one more override first. Tabby's sample config loads the safe_defaults preset (temperature 0.8, top_k 40, top_p 0.95, min_p 0.05), and OpenCode sends no sampler fields for the swift15-27b-128k model id these runs used, so that preset would have been the sampler. Every run here forced the model card's values through a sampler_overrides file instead.

taskpatched Tabby, EXL3 4.00 bpw + MTPllama.cpp IQ4_XS + MTP
t10–t12, 9 cells9/9 PASS, 800 s total9/9 PASS, 814 s total
Zig t15, spec prompt, 128k, 3 runsPARTIAL, FAIL, FAIL (hidden core 15, 12, 13 of 15), 2,749 sPARTIAL, PASS, FAIL (15, 15, 13 of 15), 2,130 s

No corrupted write or edit arguments surfaced. The Zig misses fall in concurrency, idempotency and atomicity on a storage failure, kinds that llama.cpp runs of this model have missed elsewhere; the llama.cpp runs in the table missed atomicity and idempotency. The Zig context post covers that task, and its clean sweep at 128k comes from a different set of runs.

Speed matched on the short tasks and fell behind on the long ones. That matches the speed profile: agent turns are prefill-heavy, and prefill was where Tabby was slower. (Faster prefill is no promise either: ninfer prefilled 2.2x faster and finished the agent loop 1.04x faster.) Three runs per cell rank nothing, as nine cells run five times each showed. The engine is parked. The patch is kept.

Probe the arguments before the agent#

A coding-agent benchmark on stock Tabby would have produced a table. Every cell in it would have measured the parser and the model together, with no way to tell which one failed a task.

The check that prevents that costs a dozen requests and a stdlib script. Use values that are valid strings and valid JSON, plus one with leading indentation and a trailing newline, and compare without stripping: 123, true, null, [1, 2], {"a": 1}, and four spaces in front of return x.

Filed under: local-llm, llm, ai-agents, debugging

Sources#