Fixing Visual Autoregressive Image Quality
The Logit Refiner fixes structural incoherence in visual autoregressive models by replacing parallel, mean-field token prediction with sequential, autoregressive refinement.
Visual Autoregressive (VAR) models treat image generation as a hierarchical sequence task, moving from low-resolution feature maps to full-scale pixel blocks. Standard VAR implementations generate tokens within a single scale in parallel, which forces a mean-field approximation. This parallel decoding mechanism inherently approximates dependencies by treating every token as an independent variable, leading to the neglect of intra-scale spatial relationships. The Logit Refiner fixes this by introducing a post-hoc autoregressive module that constrains the output distribution to respect spatial structure without modifying the original, pre-trained backbone weights.
The Mechanism of Hierarchical Decoding
In a standard VAR model, the backbone predicts all tokens in a feature map simultaneously to maximize throughput. The Logit Refiner acts as a secondary, lightweight pass over these static outputs. It takes the frozen logits from the backbone and processes them through a small, autoregressive MLP that operates sequentially. During this pass, the Refiner does not change the model's fundamental parameters. Instead, it computes an additive bias to the logit scores for each position based on the backbone's initial prediction and the history of previously sampled tokens in that scale.
This is a transition from an independent probability calculation to a conditional one. In the standard parallel approach, the model calculates the probability of token B without regard for the specific value chosen for neighbor A. Because they are decoded simultaneously, the model lacks the causal link required to ensure spatial alignment. The Logit Refiner enforces this link by forcing the sampling process to acknowledge the state of previously sampled tokens when predicting the current one. By sampling tokens sequentially conditioned on these frozen backbone features, the Refiner mathematically shifts the probability of a token, P(Ti), to account for the spatial context of previously sampled neighbors: P(Ti | T0, ..., Ti-1). The Refiner re-calculates the final output logit for position Ti by adding the bias derived from the already-sampled tokens to the backbone's frozen logit, effectively chaining the probability distribution to the local spatial state.
| Feature | Standard VAR Decoding | Logit Refiner Decoding |
|---|---|---|
| Intra-scale dependency | None (Mean-field) | Sequential (Autoregressive) |
| Training requirement | Full training | None (Pluggable) |
| Parameter overhead | N/A | ~10% |
| Inference cost | Constant | Variable per-token |
Implications for Builders
The Logit Refiner allows a 1.1B parameter variant to outperform models twice its size by capturing local dependencies at inference time. This approach requires less than 5% of the base model's total training compute to produce. This efficiency makes it a viable path for environments where training a massive new foundation model is impossible but visual coherence remains a priority. The primary trade-off remains the latency penalty, which is currently bound by the sequential nature of the sampling process.
Mathematical coherence is the primary gain here. Where the standard model relies on independent probabilities for neighboring tokens—essentially guessing the value of a pixel based on global structure alone—the Refiner forces a chain rule of probability. The model predicts the probability of a patch segment only after considering the specific, sampled values of the adjacent segments in the same sequence. This ensures that features like continuous edges or gradients are not broken by independent-sampling errors. The Refiner ensures that the joint probability of the entire image is better approximated by this sequential chain than by the crude parallel estimate.