The Curious Case of Neural Text Degeneration: Why Better Likelihood Means Worse Text
Source: The Curious Case of Neural Text Degeneration - Holtzman, Buys, Du, Forbes, Choi - University of Washington / Allen Institute for AI, ICLR 2020
The paradox this paper is built on
The intuitive assumption behind decoding is: a good language model assigns high probability to good text, so decoding should search for the most probable continuation. Greedy decoding takes the argmax at each step; beam search approximates the argmax over the whole sequence by tracking the \(k\) most likely partial sequences. Both are, in spirit, trying to maximize likelihood as hard as the search budget allows.
The paper’s central, counterintuitive finding: doing this better makes the text worse. Text generated by beam search is bland, repetitive, and generic - and it gets more repetitive, not less, as you widen the beam and search harder for the maximum. The problem isn’t a weak model or a bad search algorithm doing its job poorly; it’s that the argmax of a good language model’s distribution is not what human-like text looks like in the first place. Degeneration is a property of the decoding objective, not primarily of the model.
The beam search repetition trap
Beam search’s failure mode is specific and mechanical, not vague “repetitiveness.” Once a phrase has been generated, the model assigns it a higher probability the next time the same context recurs - repeated n-grams are, statistically, a safe bet, because plenty of ordinary text (lists, dialogue tags, formulaic phrases) really does repeat, and the model has learned that regularity. Beam search, hunting for high joint probability, walks straight into this: a partial sequence that repeats a phrase looks more attractive to the search than one that moves on, so once a loop starts, every likely continuation of it reinforces staying in the loop.
step: 1 2 3 4 5 6
sequence: "I love my dog. I love my dog. I love my dog. ..."
P(next token
given loop): 0.31 → 0.44 → 0.58 → 0.67 → 0.74 → 0.81 (rising, self-reinforcing)Two consequences the paper highlights, both counterintuitive if you think of beam width as a “quality” knob:
- Wider beams make it worse. A larger beam searches harder for the joint-probability maximum, and the maximum is more degenerate than a narrower, less exhaustive search finds - the opposite of what widening the beam does for translation, where wider beams reliably help (up to the point of the well-documented “beam search curse”, where translation quality also eventually degrades with excessive beam width, for related reasons).
- The true argmax is often nearly empty. The paper connects this to Stahlberg & Byrne’s (2019) finding that for neural MT, the exact highest-probability sequence - found by exhaustive rather than beam search - is frequently the empty string or near-empty, because per-token stopping probability compounds favorably for short sequences. Maximization, taken to its logical conclusion, doesn’t even want to produce long text; every heuristic used to make beam search behave (length penalties, coverage penalties) is effectively patching around this.
Human text doesn’t maximize probability either
The diagnostic that motivates the fix: plot the per-token probability the model assigns to the token that actually comes next, at every position, for two kinds of continuations of the same prompt - one generated by beam search, one written by a human.
Beam search’s per-token probability is high and gets higher over the sequence (it’s climbing toward the repetition trap above). Human-written continuations look completely different: probability fluctuates, frequently dropping well below the model’s top choice, because human writers routinely pick a word that is coherent but not the single most predictable one - that’s most of what makes text read as informative rather than formulaic. A language model that is well-calibrated will therefore put real, human-like text at a lower per-token probability than its own greedy or beam-searched output, on average - which is exactly why chasing maximum likelihood at decode time optimizes away from human-like text rather than toward it.
This reframes the whole decoding problem: the goal isn’t to find the most probable sequence, it’s to sample from the model’s distribution in a way that avoids two different failure modes - the repetitive, low-entropy degeneration of pure maximization, and the incoherent garbage that comes from sampling too freely.
The unreliable tail, and why temperature doesn’t fix it
Sampling directly from the full softmax (ancestral / pure sampling) sounds like the natural alternative to maximization, and it does avoid repetition - but it introduces the opposite failure. A vocabulary has tens of thousands of tokens; even when the model is highly confident about the next word, the combined probability mass sitting in the long tail of implausible tokens can be substantial simply because there are so many of them. Sampling from the full distribution regularly draws from that tail, and a single wildly improbable token derails the entire continuation that follows it - the “unreliable tail” problem.
Temperature sampling (dividing logits by \(T\) before the softmax) is often reached for here, but it doesn’t actually solve this: it reshapes the distribution - \(T < 1\) sharpens it toward the mode, \(T > 1\) flattens it - but it never assigns any token exactly zero probability. The tail shrinks in relative terms but is never removed, so the failure mode is only made less frequent, not eliminated.
Top-k sampling (Fan et al., 2018) truncates to a fixed number of highest-probability tokens and renormalizes over just those. This does remove the tail - but with a fixed \(k\), it can’t adapt to how the distribution’s shape changes from one decoding step to the next:
Peaked distribution (model is confident - e.g. finishing "New York ___")
top-k=40: ████████████████████████████████████████│ … 39 more, mostly noise
└ k is too large here: most of the
included tokens are implausible
Flat distribution (model is genuinely uncertain - e.g. starting a new sentence)
top-k=40: ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓│ (cut off)
└ k is too small here: several
genuinely plausible words excludedA fixed \(k\) is simultaneously too large when the model is confident and too small when it’s uncertain, because “how many tokens are actually plausible” is a property of the distribution’s shape at that step, not a constant.
Nucleus (top-p) sampling: truncate by probability mass, not by count
The paper’s fix is to make the cutoff adaptive: instead of a fixed token count, take the smallest set of tokens whose cumulative probability exceeds a threshold \(p\).
Formally, given the model’s distribution \(P(x \mid x_{1:i-1})\) over the vocabulary at step \(i\), define the nucleus \(V^{(p)} \subset V\) as the smallest set of tokens such that
\[ \sum_{x \in V^{(p)}} P(x \mid x_{1:i-1}) \ge p \]
Then renormalize the probability mass over just that set - \(P'(x) = P(x) / p'\) for \(x \in V^{(p)}\), where \(p' = \sum_{x \in V^{(p)}} P(x \mid x_{1:i-1})\), and zero elsewhere - and sample from \(P'\).
This is exactly the fix top-k is missing: when the distribution is peaked, the nucleus is small almost automatically (a handful of tokens already cross \(p\)), pruning the same implausible tail top-k over-includes; when the distribution is flat, the nucleus grows to include as many tokens as it genuinely takes to reach \(p\), keeping options top-k would have cut off. The cutoff tracks the shape of the distribution at every single step instead of using one number for the whole sequence.
| Strategy | Cutoff rule | Adapts to distribution shape? | Typical failure mode |
|---|---|---|---|
| Greedy | argmax | - | Extreme repetition |
| Beam search | joint-probability argmax over \(k\) sequences | - | Repetition, worse with wider beams |
| Pure sampling | none | N/A | Incoherent tail draws |
| Temperature | rescale logits by \(T\) | Reshapes, doesn’t truncate | Tail still reachable |
| Top-k | fixed token count \(k\) | No | Wrong-sized nucleus on peaked/flat steps |
| Nucleus (top-p) | smallest set with cumulative mass \(\ge p\) | Yes | Degrades gracefully as \(p \to 1\) |
Evaluating “human-like,” not just “high scoring”
Because the whole point is that likelihood is a misleading proxy, the paper doesn’t lean on perplexity alone to compare strategies - it checks whether generated text matches the statistical fingerprint of human text on several axes at once:
- Self-BLEU - n-gram overlap of a generation with itself, as a repetition/diversity signal. Beam search scores very high (repetitive); nucleus sampling with a well-chosen \(p\) tracks human self-BLEU closely.
- Zipf coefficient - human language follows a roughly Zipfian word-frequency distribution (a small number of very common words, a long tail of rare ones used sparingly). Beam search text under-uses the tail - it leans on safe, frequent words - skewing its Zipf slope away from human text; nucleus sampling’s word-frequency profile matches human text much more closely.
- Repetition % - direct measurement of how often generations fall into repeated n-gram loops.
- HUSE (Hashimoto, Zhang & Liang, 2019) - combines human quality judgments with a statistical diversity measure into one score, specifically to penalize both failure modes at once: text that’s generic/repetitive (high likelihood, low diversity) and text that’s incoherent (high diversity, low quality). Nucleus sampling with \(p\) around 0.9-0.95 comes out on top of this combined score, beating both beam search and top-k across the settings tested.
Takeaway
The paper’s real contribution isn’t “sample instead of search” - sampling was already known - it’s diagnosing why pure maximization degrades text (the argmax of a good model isn’t human-like text, and beam search’s repetition trap is a predictable consequence of chasing it harder) and then fixing the right part of the sampling pipeline: not the temperature, not a fixed token count, but an adaptive cutoff shaped by the distribution itself at each step. That’s also why nucleus sampling shows up as “the common default” for open-ended generation in every decoding-strategy summary since (including the Attention Is All You Need notes on this site) - it’s less a stylistic choice than a direct fix for a specific, measured failure mode of the alternative. The practical number worth remembering is \(p \approx 0.9\text{-}0.95\) for open-ended generation, tightened toward more deterministic decoding for tasks (translation, factual QA) where there genuinely is closer to one right answer and beam search’s bias toward high likelihood is less of a liability.