Transformer, another city! The top of many low-level tasks was occupied, and Peking University Huawei and others jointly proposed the pre training model IPT

https://blog.csdn.net/Extremevision/article/details/110522513
The improvement of random hardware level , A deep learning model pre trained on large data sets ( such as BERT,GPT-3) It shows that it is more effective than traditional methods .transformer The great progress of this project is mainly due to its powerful feature expression ability and various architectures .

In this paper , The author of low-level Computer vision tasks （ For example, noise reduction 、 Over score 、 Go to the rain ） A new pre training model is proposed ：IPT(image processing transformer). For the biggest digging transformer The ability of , The author uses the famous ImageNet A large number of degraded image data are produced , And then we use these training data to compare the results of the training IPT( It has many heads 、 Multi tailed to fit a variety of degradation and degradation models ) Model training . Besides , The author also introduces contrast learning to better adapt to different image processing tasks . After fine tuning , The pre training model can be effectively applied to tasks that are not available . Just need a pre training model ,IPT That can be done in multiple low-level Get better than on a benchmark SOTA The performance of the scheme .