Source: South China Morning Post
Baidus model, called ERNIE (Enhanced Representation through kNowledge IntEgration), recorded the highest score of 90.1 just ahead of Microsoft and Google on the General Language Understanding Evaluation (GLUE) benchmark and analysis platform for natural language understanding. Baidus model was first developed to understand Chinese language but researchers soon found it was able to understand English better too.
Baidus ERNIE was inspired by Googles BERT, a masked language model used by the US tech giant to train AI to understand human language. Googles model hides 15 per cent of the words in each sequence and then tries to predict them based on the context.
However, given that many Chinese characters do not have an inherent meaning until they are strung together with other characters a key linguistic difference from English the team at Baidu needed to train its AI model to understand how to hide a string of meaningful characters and predict the masked ones.
The team at Baidu illustrated the technique on its Github page, taking Harry Potter is a series of fantasy novels written by J. K. Rowling, as an example.
BERT was able to identify the K through the local co-occurring words J, K, and Rowling, but was not able to learn any knowledge related to the word J. K. Rowling. However, ERNIE was able to extrapolate the relationship between Harry Potter and J. K. Rowling by analysing implicit knowledge of words and entities, to infer that Harry Potter was a novel written by J. K. Rowling.
As the Baidu algorithm started to understand meaningful words instead of individual characters, it performed better in both English and Chinese. The company has since adopted ERNIE for real-world applications by using the AI model to deliver better search results. A paper describing the model has been accepted by the Association for the Advancement of Artificial Intelligence (AAAI) for its annual conference in February.
Comments
Post a Comment