"A Smaller Transformer in Your Transformer" Vision Transformers sometimes contain groups of neighboring layers that has similar outputs, suggesting they are making small updates within the same computational phase. This paper finds these groups and distills each one into a single normal Transformer layer that directly maps from the start of the phase to its end. On DINOv2, it cuts compute roughly in half with only a tiny accuracy drop. https://alphaxiv.org/abs/2609.20100