Google's AI model, which can translate 101 languages, is only one more than Facebook

Big data digest

source ：VB

10 End of month ,Facebook Released a translatable 100 Machine learning models for languages , Microsoft has released a version that can translate 94 Models of languages , Google, of course, is not to be outdone .

Following Facebook After Microsoft , Google has open source a MT5 Model of , The model has achieved the most advanced results in a series of English natural language processing tasks .

 

MT5 It's Google's T5 Multilingual variants of the model , Is already containing 101 Pre training was conducted in data sets of six languages , Just like Facebook There's one more .

 

Github Address ：

https://github.com/google-research/multilingual-t5

 

MT5 contain 3 Million to 130 One hundred million parameters , It can be directly applied to multiple language environments  

MT5 contain 3 Million to 130 One hundred million parameters , It is reported that , It can learn 100 Multiple languages without interference .

 

MT5 Is in MC4 Trained on ,MC4 yes C4 A subset of ,MC4 Contains about 750GB The English text of , These texts come from Common Crawl The repository (Common Crawl Contains billions of web pages crawled from the Internet ). although C4 The dataset is explicitly designed to use only English , but MC4 covers 107 Languages , contain 10,000 Web pages or more .

 

however , There are still some deviations in the data set , Google researchers are trying to remove MC4 Duplicate lines in the document and filter pages with incorrect words to alleviate MT5 The deviation of . They also used tools to detect the main language of each page , And deleted credibility lower than 70% The page of .

 

Google said , maximal MT5 The model has 130 One hundred million parameters , More than the 2020 year 10 All benchmarks for monthly testing . Of course , Whether the benchmark fully reflects the real performance of the model , This is a topic worthy of debate .

 

Some research shows that , Open Domain Question Answering Model （Open-Domain Question-Answering, A model that can theoretically answer novel questions with novel answers ） It's usually just a matter of simply remembering the answers found in the training data based on the data set . But Google researchers assert that MT5 It's a step towards a powerful model , These functions do not require challenging modeling techniques .

 

Google researchers describe MT5 In his paper, he wrote ,“ in general , Our findings highlight the importance of model competence in cross language representation learning , And show , By relying on filtering 、 Parallel data or intermediate tasks , Expanding the simple pre training formula is a viable alternative .”“ We demonstrated T5 Recipes are directly applicable to Multilingual Settings , And it achieves powerful performance on different benchmark sets .”

 

comparison Facebook And Microsoft , Google's MT5 It seems to be a little better

 

Facebook The new model of is called M2M-100,Facebook Claim to be the First Multilingual Machine Translation Model , Can be directly in 100 Translate back and forth between any pair of languages .Facebook AI Build a total of 100 Language 75 A huge data set of hundreds of millions of sentences . Using this dataset , The research team trained a man with more than 150 A universal translation model with hundreds of millions of parameters , According to the Facebook A blog description of , The model can “ Get information about the relevant language , And reflect a more diverse language text and language form ”.

 

And Microsoft's machine learning translation model is called T-ULRv2, It can be translated. 94 Languages . Microsoft claims ,T-ULRv2 stay XTREME( A natural language processing benchmark created by Google ) Got the best search results in , And will use it to improve Word Semantic search in 、Outlook and team Reply suggestions and other functions in .

 

T-ULRv2 stay XTREME At the top of the list

 

T-ULRv2 It is a joint research product of Microsoft Research Institute and Turing team , contain 5.5 One hundred million parameters , The model uses these parameters to predict . Microsoft researchers trained on a multilingual data corpus T-ULRv2, The data corpus comes freely 94 Web pages made up of languages . In the process of training ,T-ULRv2 Translation by predicting the hidden words in sentences of different languages , Occasionally, contextual cues are obtained from paired translations of English and French .

 

All in all , In terms of the number of languages translated , Google's MT5 It seems to be a little better , But large numbers don't mean high accuracy , Just Google and Facebook For two translation models , There is still room for improvement in the translation of some low resource languages , Like wolov 、 Malathi . Besides , Each machine learning model will have a certain deviation , Just like Allen AI What the researchers at the Institute said ,“ The existing machine learning technology can not avoid this defect , People are in urgent need of better training mode and model construction ”.

 

Relevant reports ：

https://venturebeat.com/2020/10/26/google-open-sources-mt5-a-multilingual-model-trained-on-over-101-languages/

https://venturebeat.com/2020/10/20/microsoft-details-t-urlv2-model-that-can-translate-between-94-languages/

The main work of the future intelligent laboratory includes ： establish AI Intelligence system intelligence evaluation system , Carry out the world artificial intelligence IQ evaluation ; Launch the Internet （ City ） Cloud brain research project , Building the Internet （ City ） Cloud brain technology and enterprise map , For the promotion of enterprises , Intelligent level service of industry and city .

   If you are interested in laboratory research , Welcome to the future intelligent laboratory online platform . Scan the QR code below or click on the bottom left corner of this article “ Read the original ”