AI in search: Insights from Bing’s Fabrice Canel

I recently connected with Fabrice Canel, a 26-year search veteran at Microsoft and principal product manager leading the team crawling, processing and indexing at Bing.

Canel offers insights into the state of artificial intelligence in search and the integration of chat experiences in the new Bing. 

Can you discuss your most recent announcements with Bing Chat?

Fabrice Canel (FC): In February 2023, we introduced the world to the all-new AI-powered Bing and Microsoft Edge, your copilot for the web. Since then, more users have been using Bing than before, and we recently hit 100 million daily users and 100 million chats. 

We then enabled the new AI-powered Bing in Windows, Skype, Edge mobile and Bing mobile, including voice access.

In recent weeks, we’ve added various ways to interact with new AI-powered experiences that extend these capabilities to millions of additional people across devices and around the globe.

• Bing Image Creator offers the ability to create images with your own words, enabling new AI-powered visual Stories allowing to explore topics that interest Bing customers with facts, image-based timelines, polls, actions, and related topics, 
• AI-generated stories provide Bing customers with multiple ways to consume bite-sized information – through text, images, video, and audio,
• AI-generated Bing Knowledge Cards help Bing customers explore topics that interest Bing customers with facts, image-based timelines, polls, actions, and related topics.
• Another way to access and interact with the new Bing is by enabling it in the SwiftKey keyboard app, which puts the Bing AI experience one touch away from any iOS or Android mobile experience that supports a third-party keyboard. You can see the full announcement here. (Author’s note: Bing acquired SwiftKey in 2016.)

Can you discuss the thoughts behind integrating Chat experiences into Search (philosophy, expectations)?

FC: Let’s go behind the scenes. I want to share a little bit more about the new Bing and how it came to be. 

Next-generation model. The new Bing is running on a new, next-generation large language model customized specifically for search. It takes key learnings and advancements of past models and is even faster, more accurate and more capable. In short, it is much more capable of synthesizing, summarizing, chatting and creating.

Microsoft Prometheus model. We have developed a proprietary way of working with our next-generation model to best leverage its power.

We call this collection of capabilities and techniques the Prometheus model. This combination gives Bing customers more relevant, timely and targeted results, with improved safety.

When we had a breakthrough in large language models (LLMs), much like other LLMs, it was trained with data through a given point in time, so we thought that we could make the user experience richer, more relevant, and more accurate by combining it with the power of Bing’s back-end.

To be more concrete, we developed a proprietary technology called Prometheus, a first-of-its-kind AI model that combines the fresh and comprehensive Bing index, ranking, and answers results with the creative reasoning capabilities of most-advanced models. 

Prometheus leverages the power of Bing and language models to generate a set of internal queries iteratively through a component called Bing Orchestrator. It aims to provide an accurate and rich answer for the user query within the given conversation context. All within a matter of milliseconds. We refer to this Prometheus-generated answer as the Chat answer.

Selecting the relevant internal queries and leveraging the respective Bing search results is a critical component of Prometheus since it provides relevant and fresh information to the model, enabling it to answer recent questions and reduce inaccuracies – this method is called grounding.

Put another way, the model reasons over the data provided by Bing; hence, Bing data ground it via the Bing Orchestrator. 

The diagram below illustrates how Prometheus works at a high level.

The last step is when Prometheus attaches relevant Bing search answers, such as weather, stock, sports, news, and so on, to the Chat answer to provide a richer and more engaging user experience that further leverages the power of Bing. 

Thanks to the Bing grounding technique, Prometheus is also able to integrate citations into sentences in the Chat answer so that users can easily click to access those sources and verify the information. 

Sending traffic to these sources is important for a healthy web ecosystem and remains one of our top Bing goals. This matters to us because there won’t be a web ecosystem if we don’t send traffic to the people creating the content. People are investing in content for a reason, and if they don’t get rewarded, they’ll stop doing so.

Applying AI to the core search algorithm. We’ve also applied an AI model to our core Bing search ranking engine, leading to the largest relevance jump in two decades.

This AI model makes even basic search queries more accurate and relevant. The new models improved our understanding of documents on the internet and better understanding of how to match those documents up with user queries.

New user experience – Merging Search and Chat. We reimagined how Bing customers interact with search, browser and chat by pulling them into a unified experience. This will unlock a completely new way to interact with the web.

While Prometheus is a remarkable, pioneering AI-based innovation from a user experience perspective, it was unclear how we should integrate its capabilities into Bing. There were two main points of view:

• Some in our team felt that search is such an ingrained habit that we needed to keep the UX like today’s web search and add the Prometheus-powered Chat answer on the main UX. Just like any other answer, the Chat answer would earn its place based on its relevance to the query.
• Others in Bing felt that this was an opportunity to change the search paradigm from the classic web and answers results to a new interactive, chat-based way of searching.

Both schools of thought were partially correct, yet neither had the complete picture.

The team advocating for classic search generally used navigational queries to defend their point of view.

The team pushing for the conversational approach would present research-like search sessions such as shopping or travel. I plotted their queries in a graph below to illustrate this observation.

It then became clear that some queries would perform better in the classical or Search mode while others would do better in a conversational or Chat mode.

Also, some users may prefer one over the other, depending on the query. Therefore, the ideal Bing search product must smoothly transition between Search and Chat modes based on the user’s intent and preference. 

Our design team took on the challenge and, after multiple iterations, developed a new UX that unified Search and Chat in a single interface. Users could easily switch back and forth by clicking on UX elements on the page or simply scrolling or swiping up and down.

This product or UX innovation is arguably just as important as the Prometheus technology to ensure people can intuitively use the product to its full potential.

I feel that UX and SEO nearly always have the same goals these days. What are your thoughts on that?

FC: I agree. It’s important to understand what you want a user to do when they come to one of your webpages.

You should design your webpages to do that well. Destination UX also matters for search engines. Relevance is only a part of the equation.

If they come to your site and are really interested in converting, but they don’t because of a problem with your site, that’s a bad experience for the search engine, too. 

Google has its E-E-A-T standards, and the Bing equivalent of E-E-A-T is very similar. It’s not just about the content itself. Users want to get actions done, and we want to rank pages higher that help them do that.

Given the potential for errors, how would Bing advise people to use the output from Bing Chat?

FC: It’s hard to believe It’s been just over two months since we released the new AI-powered Bing and Edge to the world. We have learned a lot and shipped several improvements in that time.

We are delighted by the virtuous feedback cycle and iteration driving strong Bing improvements of our models and the experience. 

For instance, we have introduced the ability to toggle the tone of chat between:

• Precise, which focuses on shorter, more search-focused answers.
• Creative, which gives responses that are longer and more descriptive.
• The middle setting, Balance, is somewhere in between. 

Bing customers will notice a color change in the UX between purple, blue and green, depending on which is selected. Our goal is to let Bing customers decide on the type of chat behavior that best meets their needs. We’ll continue to tune this experience based on feedback.

At Bing, we’re committed to delivering the best possible result. If you ask Bing Chat if the Earth is flat it will tell you that Earth is not flat, even if there is a lot of misleading content online on this topic. 

We have made huge investments in AI because we’re optimistic about what it can do to help people, industry and society.

We’re committed to bringing technology and people together to realize the promises of AI responsibly. Feedback and data will help to improve the quality of the answers. Keep your feedback coming.

Should people use Bing Chat to create and publish content without editing it?

FC: We do our best to find the best content. 

Are we there yet? No, we are not perfect yet. No one is. 

Will we be someday? Maybe. Remember that it’s a tool, and this problem is harder than you might think. 

For example, sometimes there are multiple conflicting answers from highly reputable websites. It can be hard to figure out which one is wrong.

I went through this exercise recently trying to answer a question on a complex math geometry problem for my daughter, and I found two different answers from respected experts. One was right, but it took me 1 to 2 hours to figure it out. Even experts make mistakes.

How does Bing Search’s usage compare with Bing Chat’s usage for those that have access to it?

FC: It’s still incredibly early, we released only two months ago, and most Bing customers are experiencing this ability to chat with a search engine for the first time in their life. They are discovering and learning. They are finding value in having search, chat, answers, and creation capabilities all in one experience. 

There are now more than 100 million daily active users of Bing. New scenarios like chat drive engagement, including over 100 million chats.

Of the millions of users in Preview, one-third are new to Bing, creating net-new opportunities for publishers and webmasters. This is still early data, but the signs are encouraging.

For SEO, as it is still early, my recommendation is to stick with the same SEO playbooks as before.

But please keep track of our communication and monitor data to make necessary adjustments as the usage of the new Bing continues to increase, transforming how people search.

In the coming weeks, we will start reporting Bing Chat clicks and impressions on the Bing Webmaster Tools website and Bing Webmaster Tools API. 

Can you discuss the importance of data freshness? 

FC: The recent breakthrough in LLMs is great, but these models are trained with data through a given point in time. What’s special at Bing is that you benefit from the LLMs and the power of Bing Search’s back end.

This means that Bing Chat contains content published seconds ago online. You can Bing Chat what’s happening today – that’s happening right now.

Freshness matters a lot these days, especially for the younger generation. I found it fascinating to see my kids using Bing Chat to learn more about recent events. Bing can handle these types of queries well, even in Bing Chat, because it’s built on top of the Bing index.

One area where people spend a lot of time is on news-type events, including things like celebrity news. People spend a lot of time on things that are happening right now.

Can you provide an overview of what IndexNow is?

FC: IndexNow is an easy way for website owners to instantly inform all search engines about the latest content changes on their websites.

In its simplest form, IndexNow is a simple ping so that search engines know that a URL and its content have been added, updated, or deleted, allowing search engines to reflect this change in their search results quickly.

I find it interesting that you ask this question and speak about IndexNow in the context of Bing Chat which leverages the best-in-class AI models. 

While AI is great in helping retrieve the best content online and to see what’s changed on a website, we simply don’t know with 100% precision if you have added, modified or deleted content on your site. But you know that with 100% precision, or at least your CMS knows that. 

Instead of repeatedly crawling to discover if your content has changed, we provide webmasters the ability to ask search engines crawler to visit to collect the latest content changes.

Since we enabled the IndexNow protocol in the Fall of 2021, the adoption of IndexNow has continued to grow.

Today, more than 20 million websites publish over 2.5 billion URLs to the IndexNow API daily. This month, IndexNow attributed to 10% of all new URLs clicked in web search results.

It’s also great to see more search engines, content management systems (i.e., Wix), and top websites adopting IndexNow. We target that more the 50% of the web will adopt IndexNow within two years. 

Why is crawling so complex? Just how large is the web?

The true size of the internet is infinite. Every day we discover more than 70 billion distinct URLs. There is so much spam and duplicate content.

Spammers grab domains and generate hundreds of subdomains. No search engine communicates on index size anymore because it’s not a good measure of quality. 

The big issue is how you navigate through all the low-quality content to index what matters. To make this easier, we need webmasters to help guide search engines to the real stuff. The best way to do this is with a combination of IndexNow plus XML sitemaps. This is a good setup for all search engines.

How much traffic does Bing anticipate will go to websites in the new Bing Chat model?

FC: In the next few weeks, we will start reporting Bing Chat clicks and impressions in Bing Webmaster Tools so that each website owner can compare them to Bing Search pages.

Search engines have played a role in helping people find the content they want. The new Bing is helping to better address people’s search needs with new capabilities like chat, answers, and content creation. 

Our goal is to satisfy our Bing users and drive more traffic to publishers in this new world of search. It is a top goal for us, and we measure success partly by how much traffic we send from the new Bing/Edge. 

The early progress is encouraging. Based on our data from the preview, we are driving more traffic from all types of users. We have brought more people to Bing/Edge for new scenarios like chat and are seeing increased usage.

Then, we have uniquely implemented ways to drive traffic to publishers, including citations within the body of the chat answers linked to sources and citations below the chat results to “learn more” with links to additional sources. 

We enable an expanded hover experience where hovering over a link from a publisher will display more links from that publisher, giving the user more ways to engage and driving more traffic to the publisher’s website.

We are also listening, and we’re hearing positive feedback as we look for opportunities to maximize these new experiences for the entire ecosystem. 

How does one create content likely to be cited by Bing Chat? 

FC: With the release of the all-new AI-powered Bing and Microsoft Edge, we have enhanced our Bing Webmaster Tools Guidelines to support the experience.

We listed that ranking within Bing Chat generally relies on the same parameters as the main web search results page.

So, since this is still early at this point, I recommend sticking with the same SEO playbooks as before, and most of the recommended playbooks are listed in our Bing Webmaster Tools Guidelines.