Detecting Model Backdoors with Speculative Decoding

Speculative decoding is an optimization technique designed to reduce inference latency by offloading the bulk of work to a smaller draft model. A small, fast draft model proposes a sequence of tokens, which a larger target model then validates in parallel. Because this verification process requires the target model to calculate its own probability for each proposed token, it creates a diagnostic trace. SpecGuard uses this telemetry to identify poisoned models. If a target model contains a hidden trigger, its output behavior diverges from that of a baseline model, causing a detectable shift in how often it accepts the draft model's tokens.

The Mechanics of Divergence

In standard speculative decoding, the draft model generates a sequence of tokens. The target model then performs a single forward pass, calculating the probability distribution for the current context. To verify the draft tokens, the target model compares its own output probability for each draft token against a threshold. Specifically, the target model accepts a draft token if a random value drawn between 0 and 1 is less than the ratio of the target model's probability to the draft model's probability. If the ratio is low, the target model rejects the token and samples from its own, corrected distribution.

A backdoor maps rare, specific input patterns to an attacker-chosen output. By design, a trigger forces the model to ignore standard linguistic patterns and shift its internal state toward a malicious target. This shift forces the model to assign very high probability to malicious tokens that are statistically improbable compared to the normal language patterns predicted by the draft model. Because the draft model is unaware of the backdoor and only understands standard language structure, it continues to suggest tokens that are statistically probable in a normal context. The target model, now forced into its malicious state by the trigger, assigns a very low probability to these benign tokens. This causes the rejection sampling ratio to plummet. The high rejection rate serves as the signal for the backdoor, as the target model repeatedly rejects the draft model's natural language output in favor of its own, trigger-induced path.

Implementation and Trade-offs

Infrastructure already using speculative decoding can implement SpecGuard by monitoring the acceptance rate statistics during inference. You observe the ratio of accepted draft tokens relative to the total tokens proposed. A sudden spike in the rejection rate when specific inputs are processed indicates an attempt to activate a backdoor. This monitoring relies on the fact that an attacker cannot shift the target model's internal weights toward a malicious output without simultaneously reducing the probability mass allocated to the tokens a benign model would predict. If the target model behaves as expected on normal inputs but fails to accept draft tokens on specific, rare sequences, the discrepancy serves as an indicator of a triggered backdoor. While this is a reliable signal, it is important to note that attackers attempting to evade it are theoretically forced to weaken the backdoor's efficacy, as trying to make the backdoor probability match the draft model's benign predictions directly conflicts with the goal of producing a malicious output.

The effectiveness of SpecGuard is limited by the nature of the model's training. It is ineffective against triggers that do not significantly alter the token probability distribution, such as subtle triggers that mimic the statistical signature of clean, benign text. If an attacker designs a trigger that causes the model to output a benign-looking but incorrect result rather than an overt, malicious one, the target model may still assign high probability to the draft model's suggestions, allowing the backdoor to evade detection. It is currently unknown how effectively this method scales to very large models where the internal weight shifts might be more subtle, or whether sophisticated attackers can train backdoors that mimic the statistical properties of a target draft model while still achieving their malicious intent.