Indexing Queue

If you are having any issues with your data, indexes, or queue, please check our troubleshooting guide.

Before we can search your data, we have to upload it to Algolia. This process is called indexing. It is handled automatically by the indexing queue.

If you want to know more about indexing in general, please check out our indexing overview.

The extension uses the indexing queue to keep your data - like products, categories, pages, etc. - synchronized with Algolia in real time. Every update to your Magento data will trigger an event. This event tells the extension to update your data into Algolia. This provides the best and most up-to-date search experience for your customers.

The queue and its operations run asynchronously, meaning that the administrator can continue any task in the Magento administration while your data is updated and sent to Algolia in the background.

The Indexing Queue

To enable the indexing queue, navigate to System > Configuration > Algolia Search > Indexing Queue/Cron tab in your Magento administration. On this page, you can set the number of jobs you want the queue to process simultaneously. The default is 10, but you can increase or decrease the value based on your catalog and on the server running your Magento store.

All the queues operations appear in the database table algoliasearch_queue.

Enable the queue process

After you enable the queue, you have to set up the process that will run the queue. There are multiple ways to do this.

With cron

We recommend that you setup the queue process by configuring a crontab. This will run the indexing queue at regular time intervals. The following cron will run the indexer every 5 minutes.

1
*/5 * * * * php -f /absolute/path/to/magento/shell/indexer.php -- -reindex algolia_queue_runner

Without cron

You can also process the queue manually from the command line:

1
php -f /absolute/path/to/magento/shell/indexer.php -- -reindex algolia_queue_runner

The command above will run the indexing queue once. This single run will not necessarily empty the queue because the number of pending jobs might exceed the number of jobs processed per run.

Processing the full queue

If you want to force the queue to process all the jobs in one run, pass the EMPTY_QUEUE=1 parameter in your command:

1
EMPTY_QUEUE=1 php -f /absolute/path/to/magento/shell/indexer.php -- -reindex algolia_queue_runner

Please note that forcing the queue to process all jobs might not always be successful. Errors like network timeouts can occur. Further, if the data you’re trying to process is too large for one job, the job will fail: the queue will not be empty and your data will not be in sync. Read our guide on common indexing errors to any problem you may encounter.

Clearing the queue

To clear your queue you can go to System > Algolia Search > Indexing Queue page in your Magento backoffice and click the “Clear Queue” button to remove all jobs from the queue.

Additionally, you can also truncate your algoliasearch_queue table in your database.

We recommend that if you clear your indexing queue, you should run a full reindex of your data to ensure that they are up to date in Algolia.

Full Section Reindexing

With the indexing queue enabled

With the indexing queue enabled, a full reindex is done by creating temporary indices. Instead of sending your data directly to the production index, we upload it to a new, temporary index. This prevents the production index from being inconsistent and from having temporary duplicate data. After all the data has been added to the temporary index, the temporary and production indices switch places. This process is called atomic reindexing, and it has a couple of advantages:

1. The (re)indexing speed is higher
2. It avoids potential inaccuracies in your indices
3. A lower number of operations is needed

Once the switch between the temporary and production index takes place, the changes made by the reindex will be visible.

With the indexing queue disabled

With the indexing queue disabled, a full reindex is done by processing your complete catalog synchronously. Nonetheless, we strongly encourage that you allow the indexing queue to process your catalog because this synchronous approach has downsides.

1. The (re)indexing takes a lot of time and resources
2. If your catalog is big, PHP may trigger timeouts (you can read more about large indices)
3. You can’t use the Magento administration while the process is running

Because of all these downsides, running a full reindex without the indexing queue is unreliable. This is why we encourage you to use the indexing queue.

Automatic Indexing

By default, each change or deletion of any product or category is propagated to Algolia instantly. This ensures that your data in Magento and in Algolia remain in sync. You can change this default behavior if needed, by changing the indexer’s mode to Manual Update. This will prevent the indexer from updating the information in Algolia on every change you make.

Please note that if you choose this approach, you have to run a full product or category reindex on a regular basis - every night for example - to keep your data synchronized.

Manual Reindex

There are two ways to trigger a complete reindex of your catalog.

Through the Magento administration

In your Magento administration, navigate to System > Index Management and click the Reindex button next to the indexers you want to reindex:

Through the command line

You can also trigger a complete reindex of your catalog through the command line. For example, run the following command to reindex all your products:

1
php -f /absolute/path/to/magento/shell/indexer.php -- -reindex algolia_search_indexer

It’s possible to use more than one indexer name in your command to trigger a reindex for multiple entities. These are the names of all the indexers you can use in your command:

• algolia_search_indexer - reindexes all products
• algolia_search_indexer_cat - reindexes all categories
• algolia_search_indexer_pages - reindexes all CMS pages
• search_indexer_suggest - reindexes all search query suggestions
• search_indexer_addsections - reindexes all data from additional sections
• algolia_queue_runner - processes jobs in the indexing queue