Speller100: Bing Rolls Out Large Scale Spelling Correction Globally

Microsoft Bing is rolling out its large-scale multilingual spelling correction models, jointly known as Speller100, globally with high accuracy and memory in 100+ languages.

According to Bing, around 15% of all search queries performed by users have spelling errors, which can trigger wrong answers and substandard search results.

In order to tackle this particular problem, Bing has built Speller100, which as per the company itself, is “the most comprehensive spelling correction system ever made.”

After performing A/B testing, the company saw these results:

• The number of SERPs with no results decreased by about 30%.
• The number of times people had to reframe their search queries manually reduced by 5%.
• The number of times people clicked on Bing’s spelling suggestion rose from single digits to 67%.
• The number of times people clicked on anything on the page increased from single digits to 70%.

Wondering how Bing achieved this? Continue reading to find out more about Speller100.

Refining Spelling Correction in Bing Search Results

Spelling correction has been a priority for Bing for a long time now. With the addition of more languages from all around, the search engine is now taking things a step further.

In its blog, the company said that it intends to expand its existing spelling correction service to 100+ languages to make Bing more all-embracing while setting the same high standard for quality that the company had set for the initial few dozen languages.

The rollout of Speller100 indicates major progress for the search engine, which has been made possible because of the latest developments in AI technology.

Bing has clearly explained the technology behind Speller100 in its recent blog. However, here are a few primary details of the search engine’s new spelling correction system.

Bing’s New Speller100 Technology

The company gives credit to zero-shot learning for making the Speller100 possible.

Zero-shot learning enables AI models to precisely learn and correct misspellings even for a low resource or no resource languages. On the other hand, conventional spelling correction systems used to rely on training data solely to learn and correct the spelling of a language.

However, when an insufficient amount of data was present, depending on training data to correct the spellings of languages made things harder. And that is the type of problem zero-shot learning is built to resolve.

According to the company, zero-shot learning is a crucial element in Bing’s Speller100 that lets them expand even to languages that have little to no data available.

Spelling Correction Is Different From Natural Language Processing

Stating the difference, Bing said that even though we have seen significant advancements being made in natural language processing, spelling correction is entirely a different task.

We can broadly classify spelling mistakes into two types:

•  Real-word error: This happens when the word is correct but does not fit in the larger context.
• Non-word error: This happens when the word itself is not in the vocabulary for a particular language.

The company has developed a deep learning approach, inspired by Facebook’s BART model, to correct such spelling mistakes. Nevertheless, it is different from BART. In their Speller100 model, Bing frames spelling correction as a character-level problem. Bing’s model is trained using character-level mutations that imitate spelling mistakes to address this character-level problem.

Bing has designed what it calls ‘noise functions’ to create common errors of insertion, rotation, replacement, and deletion.

Noise functions enable Bing to train its new spelling correction model to correct the spellings of languages without requiring a vast collection of misspelled queries.

Instead, the search engine manages with the regular text derived from web pages. According to Bing, this text can be easily extracted via web crawling, and there’s an adequate amount of text present on the web to train hundreds of languages.

Wrapping It Up

The launch of Speller100 in Microsoft Bing is the first step in a broader effort to employ the technology in other Microsoft products as well, according to the company.

As the A/B testing results clearly indicate, the rollout of this new spelling correction system will undoubtedly improve the user experience across the platform.

With this technology, the search engine has expanded its accurate spelling correction to more than 100 languages to present better results.