当前位置：网站首页>Read the paper with me - multi model text recognition network

Read the paper with me - multi model text recognition network

2022-07-22 07:58:00 【InfoQ】

This article is shared from Huawei cloud community 《

Multi-Model Text Recognition Network

》, author ： Grain rain moistens a wheat .

Language model is often used in the post-processing stage of character recognition , Used to optimize identification results . However, the a priori information acts independently on the output of the recognizer , So the previous method did not make full use of this information . In this paper, MATRN, Cross modal feature enhancement between semantic features and visual features , So as to improve the recognition performance .

Method

The flow chart of the method is shown in the figure above , Firstly, the preliminary character recognition results are obtained through the visual extractor and the position attention module . Then the recognition result is passed through a pre trained language model , Get the semantic features of the text .

Then through the Module shown on the left of the figure above , utilize transformer Feature enhancement of visual features and semantic features . Finally, the following formula is used , Feature fusion of visual features and semantic features , And make the final classification .

It is worth noting that , Feel the method of self-monitoring , This paper also proposes a method to mask the visual feature map . say concretely , Use the attention map in the position attention module , Randomly select the attention weight at a certain time as the mask ,mask Lose some visual features .

experimental result

It can be seen from the experimental results , This method has a great improvement in difficult irregular images . This shows that when the model is difficult to recognize visually , The semantic features between words help to identify .

Click to follow , The first time to learn about Huawei's new cloud technology ~

原网站

版权声明
本文为[InfoQ]所创，转载请带上原文链接，感谢
https://yzsam.com/2022/203/202207211617161694.html