Steering LLM Decoding with DescaPE

DescaPE, or DEcoding Signal Control Against Path Error-snowballing, allows developers to intervene in token generation to suppress hallucinations by monitoring internal model signals. It replaces reactive methods that address errors after they have already propagated through the autoregressive chain with an active gate during the forward pass. This method extracts internal attribution signals from a specific factual-salient layer span within an LLM to steer the generation process, requiring only a lightweight probe rather than post-hoc verification or external knowledge retrieval.

Identifying Factual Salience

Fine-tuning is static and cannot correct errors that emerge during inference. Retrieval-augmented generation adds latency by requiring an external index. DescaPE shifts this by utilizing a probe that approximates the model's internal attribution signals regarding factual grounding. Researchers identify the factual-salient layer span by sliding a window across MLP blocks and measuring accuracy degradation when each is disabled; the span where disabling the MLP causes the largest performance drop is labeled as factually salient.

The logic relies on the finding that these MLP layers map specific internal states to factual knowledge. To build the probe, researchers train a lightweight classifier on a dataset of paired factual and hallucinated sequences. By observing how the hidden states of these sequences diverge within the salient layer span, the probe learns to represent the attribution signal that characterizes factual grounding. During inference, the probe monitors these hidden states. When the hidden state trends toward a pattern associated with hallucination, the system calculates a divergence score. This score indicates that the model's internal trajectory is moving toward a state that consistently correlates with factual error in the training data.

The Mechanism of Intervention

The probe translates the divergence score into an intervention by modifying the model's output logits. During each step of the decoding process, the model calculates the standard probability distribution over the vocabulary. The DescaPE controller intercepts this and applies a logit adjustment. Specifically, the controller adds the divergence score, weighted by a hyperparameter, to the logits of tokens that align with the factual ground truth. By increasing the logits for high-probability, factually-grounded tokens and decreasing the logits for high-risk continuations, the system re-weights the distribution. This enforces a constraint on the internal trajectory before the next token is sampled.

Practical Implementation and Scope

For production systems, the 1.10x latency overhead is a notable metric, as reported in the authors' specific efficiency evaluation. Because the probe operates on a single forward pass, you can integrate it into existing inference pipelines without significantly increasing your hardware footprint. Developers must ensure that the training data for the probe covers the specific domain of the intended deployment.

One practical constraint is the scope of the underlying research. The effectiveness of DescaPE is documented specifically on standard Transformer architectures like Llama-2-7B. Performance is currently unverified for architectures such as Mixture-of-Experts or non-Transformer designs, and it is unclear if the specific attribution signals identified translate to those systems. Furthermore, the reliance on domain-specific training data means that performance may degrade when shifting from highly structured factual tasks to creative or open-ended generation where the definition of a hallucination is less rigid. Future work will need to establish if these probes can maintain accuracy when generalized across varied model sizes or when deployed in environments with higher token-level uncertainty.