Transformers in vision

From [1]

Survey

Transformers in Vision: A Survey

More papers in 2020 and 2021.

Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet

Transformer in Transformer

A nice video about the above paper can be found here.

End-to-End Object Detection with Transformers

AN IMAGE IS WORTH 16X16 WORDS: TRANSFORMERS FOR IMAGE RECOGNITION AT SCALE

Some theories

On the Relationship between Self-Attention and Convolutional Layers ICLR 2020

Reference

[1]https://chowdera.com/2020/12/20201229100154115d.html

[2]https://zhuanlan.zhihu.com/p/341790423