How we taught Swift to stop overthinking

The loops

Quantized Qwen 3.8 27B sometimes falls into a reasoning loop: it repeats the same few lines of reasoning until it runs out of tokens or happens to break out. We saw it at medium and low reasoning effort too, where people expect short answers. The literature calls this overthinking. Every looped token costs compute and latency.

Starting point: a token penalty

A Meta paper targets this failure in post-training quantized models by penalizing a fixed set of tokens during decoding. Applied as published, it gave mixed results on our setup. Some loops went away, others didn't change.

The method only works if it penalizes the right tokens, and a list chosen for a different model and setup had no reason to fit ours.

Finding the tokens

We generated a large set of reasoning traces on an 8xH100 machine, covering coding, language, vision and agentic tasks, all out of distribution for the benchmarks we report. We separated the traces that looped, looked for tokens they had in common, and kept the most frequent ones as the penalty set.

Penalizing at inference time

The first plan was to decode with the penalty, collect the shorter traces and fine-tune on them with plain cross-entropy. The fine-tuning didn't work. The penalty on its own did: with no weight updates, it removed far more loops than the paper's token list, on quantized weights and on the BF16 model.

Training it in

We then built a loss around the same tokens and trained a LoRA with it on the traces we had generated. Reasoning got shorter, and the effect carried over to tasks outside the training data. Accuracy dropped with it.

Recovering accuracy

We worked on this from the day Qwen 3.8 27B came out. To recover accuracy we tried RL with GSPO, on-policy distillation, and merging in parts of another reasoning adapter, until the loss was under 1% on almost all of our in-house out-of-distribution tests.

Evaluation

A single run is too noisy to compare models that differ by a point, so we ran each benchmark ten times, five on the base model and five on Swift, following the protocol on Qwen's model card for Terminal-Bench. We repeated this across reasoning efforts and precisions.

Swift uses 24 to 51% fewer thinking tokens on average, depending on the benchmark. On five of the nine benchmarks the score changed by about a point or less, and on LiveCodeBench v6 it rose from 76.8% to 81.6%.

Known regression

Math scores dropped: AIME 2026 from 98.7% to 94.0%, and HMMT from 99.3% to 96.0%. We traced the AIME drop to a training bug that put a token important for mathematical reasoning on the penalty list. An updated release will fix it.