Dynamic Re-Ranking

The relevance of your search results often changes with time, and it can change fast. Take a generic drug-store as an example. Users that searched for “mask” before February 2020 (the outbreak of the COVID-19 pandemic) most probably were looking to buy facial masks to take care of their skin. During the pandemic, it’s more likely the users are looking for surgical masks.

To address users’ changing expectations, you might have to continually tweak your results using tools like Rules or optionalFilters. This can become time-consuming because trends come and go fast. A better, more scalable approach is to use Dynamic Re-Ranking.

What is Dynamic Re-Ranking?

Dynamic Re-Ranking is an Algolia feature that leverages AI to find trends in your users’ behavior. Based on the query and the position of the result they click or convert, it can make improvements to your relevance by boosting results that are rising in popularity. Dynamic Re-Ranking can also demote outlier results that sometimes show up because of textual relevance edge cases, but aren’t actually relevant to your users in the context of a particular query.

Getting back to the generic drug-store example, the Dynamic Re-Ranking feature detects that people searching for “mask” are often looking for surgical masks instead of face masks and automatically boosts surgical masks higher in the results. This, in turn, leads to a more relevant search and can result in a higher conversion rate, without you having to create Rules or optionalFilters to tweak your relevance.

Only Premium plans can access this feature.

Effect on relevance

While Dynamic Re-Ranking can be a powerful tool in tweaking your relevance, you don’t want it to interfere with the relevance you’ve already set up. For this reason, Dynamic Re-Ranking kicks in after the engine has computed the textual relevance of your results and has applied your custom ranking. If you use Rules to pin or hide specific records, the engine applies them after Re-Ranking, so that they keep working as expected. The engine applies all other Rules before Re-Ranking. This means Re-Ranking can override Rules that boost and bury categories.

Re-Ranking doesn’t take effect if you enable Personalization and Algolia has enough data to personalize the search. If there’s not enough data to personalize a search, for first-time users, for example, Re-Ranking kicks back in. The engine doesn’t apply both Personalization and Dynamic Re-Ranking concurrently.

In summary, the engine applies the relevance strategies in this order:

• Textual relevance (through the textual ranking criteria)
• Business relevance (through custom ranking)
• Personalization (if you’ve enabled it and the engine has enough information to personalize results for a particular user)
• Boosting and burying categories (through Rules)
• Dynamic Re-Ranking
• Promoting and hiding specific items (through Rules)

By default, Algolia doesn’t apply Re-Ranking on empty searches. You can enable Re-Ranking for empty searches that are filtered on a certain facet value.

Set up Dynamic Re-Ranking

There are a few prerequisites to set up before you can leverage Dynamic Re-Ranking to improve your relevance automatically.

Click and conversion events

To make use of Dynamic Re-Ranking, you need to send Algolia click or conversion events (or both if they’re different). Algolia AI uses this data to detect trends in your users’ behavior and re-rank results based on these trends.

Dynamic Re-Ranking refreshes each query’s ranking every 24 hours. Re-Ranking always takes the last 30 days into account, meaning it’s based on a sliding window of the most recent analytics data. For a record to be promoted for a given query, Re-Ranking requires it to gather at least 20 clicks or two conversions during the 30 days time window. Re-Ranking promotes up to 20 items with the highest scores matching those criteria per query.

Security best practices

To prevent malicious users from manipulating the ranking of your records, Dynamic Re-Ranking uses a distinct userToken per event policy. That means when a userToken sends duplicated events, only one counts for Re-Ranking purposes. For instance, if the same person clicks on the same record 100 times after the same query, it counts as one click, as long as they’re using the same userToken. If the same userToken clicks on the same record after different searches, they count as separate events.

To prevent users from manually crafting several userTokens, you should use Secured API keys. With Secured API keys, you can include the userToken as an embedded parameter when you generate the key on your back end. This lets you provide each end user with a unique key in which they can’t change the userToken.

Dynamic Re-Ranking options

You can configure the following options from the AI Re-Ranking page in the dashboard.

Disable Re-Ranking for specific records

If you set a Re-Ranking Filter, the Dynamic Re-Ranking only promotes items matching the configured filter.

For instance, if you select instock = true, it prevents Dynamic Re-Ranking from promoting popular items that are currently out-of-stock.

Use a different events source index

By default, Dynamic Re-Ranking uses the events sent to the re-ranked index. You can use another index as the source for the events by changing the Events source index option.

Enable Re-Ranking on browsing pages

By default, Algolia doesn’t dynamically re-rank empty queries. Setting a Browsing facet re-ranks empty queries that include a facetFilters for that attribute.

For instance, if you use Algolia to populate the category pages of your website by performing an empty query with a category facet, you can set category as a Browsing facet. Re-ranking reorders records in each category based on their popularity.

Give more weight to recent events

To give more weight to recent events in the ranking computation, you can enable the Event freshness option. If you don’t, an event that happened four weeks ago impacts re-ranking the same way as an event that happened yesterday. Enabling event freshness better handles seasonality and shorter search trends.

Once you toggle Event freshness on, the engine recomputes re-ranking so that newer events have more weight than older ones. You don’t have to make any other changes—coding or otherwise.

Simulate Dynamic Re-Ranking

You can simulate Re-Ranking for any of your indices from the AI Re-Ranking page in the dashboard. On this screen, you can try out any query you want, and see how Dynamic Re-Ranking affects the results, without activating Re-Ranking on your live index.

In the simulator, each record has an attractiveness score per query. This score is computed with the number of events the record received during the last month compared to the others records. It explains why Re-ranking boosts one record over another.

Promotion Rules take precedence over Dynamic Re-Ranking. If you use Rules to pin top items for the popular queries, Dynamic Re-Ranking has a limited impact. You can turn off the Rules in the simulator using the Rules toggle to assess the benefits of Dynamic Re-Ranking actions. You can then consider removing these Rules if you are satisfied with the records the Dynamic Re-Ranking promotes.

Enable Dynamic Re-Ranking for production

Before you enable Dynamic Re-Ranking for an index, you should always A/B test your Re-Ranking first.

You can enable Re-Ranking on the AI Re-Ranking page in the dashboard. Select the index you want to activate Re-Ranking for, and press the Activate Re-Ranking button.

A/B test Dynamic Re-Ranking

You should always use A/B tests to test Re-Ranking on an index. This way, you can accurately measure the impact Re-Ranking has on your search. You can create an A/B test for Re-Ranking directly from the AI Re-Ranking page in the dashboard, by clicking the Launch an A/B test button at the top right of the page. Clicking this button shows a form that allows you to create an A/B test with the desired traffic split and duration. You can also set filters so that Dynamic Re-Ranking only re-ranks records that match this specific filter. This way, you can prevent, for example, Dynamic Re-Ranking from promoting items that aren’t in stock.