Linear Differential Vision Transformer: Learning Visual Contrasts via Pairwise Differentials
Published in Neural Information Processing Systems (NeurIPS), 2025
This paper proposes a Linear Differential Vision Transformer that achieves linear complexity through visual contrast attention, improving the model’s ability to capture local differences while introducing minimal additional parameters and no extra compute.
My contribution included co-designing the linear attention mechanism, optimizing the model architecture, and independently conducting multiple experiments to verify effectiveness and generalization.
Recommended citation: Y. Pu*, Jixuan Ying*, T. Ye, D. Han, Z. Wang, Q. Li, X. Shao, X. Wang, G. Huang, X. Li. "Linear Differential Vision Transformer: Learning Visual Contrasts via Pairwise Differentials." NeurIPS 2025.