Send and Update Your Data

Pushing your data to an Algolia index

Algolia uses its own servers to search your data. Once you’ve structured your data into records, you need to send them to Algolia.

There are several ways to push your data to an Algolia index:

• Using one of Algolia’s API clients. You don’t have to explicitly create an index. If you try to add records to a nonexistent index, Algolia creates it for you.
• Using the Algolia dashboard by sending a JSON file.
• Using the Algolia crawler. It automatically crawls your content directly from your site and indexes it into Algolia. The Algolia crawler is available as an add-on.

Naming your index

Choose a unique, readable name and make sure you don’t use any sensitive or personally identifiable information (PII) as your index name. This includes customer names, user IDs, or email addresses. Index names appear in network requests, so you can consider them as publicly available.

Passing Algolia credentials

You need an application ID and a valid API key to push your data. These credentials protect your data, and you can set different access levels for your team.

Every indexing operation requires these credentials. The application ID gives you access to your indices, and the appropriate API key gives you rights to perform indexing operations.

Using unique object identifiers

The engine identifies each object with a unique objectID, ideally one that you set yourself. If you don’t, Algolia generates them for you. You can retrieve the objectID by browsing the index. Later on, when you need to update or delete records, you must provide the objectID.

Sending records in batches

Algolia lets you send one or more records in a single API call, or batch records. You can batch your indexing operations via the API or the dashboard.

For optimal indexing performance, you should send your records in batches. An ideal batch size is ~10 MB, which represents between 1,000 or 10,000 records depending on the average record size.

Batching has several benefits: it reduces network calls and speeds up indexing. Users with lots of records see the most significant impact on performance, but every Algolia user should batch indexing operations whenever possible.

Batching doesn’t change how Algolia counts indexing operations. Every indexing operation adds to your count. If you send several records, each of them still counts as one operation.

Updating your data

After the initial data import, you need to keep your index up-to-date with the latest changes on your application or website. The appropriate update frequency depends on how often your content changes, and how fast you need it to be searchable.

Take the example of an e-commerce store:

• You want to update price changes or product availability in real time.
• You don’t need to update the number of sales (used for custom ranking) as often, so you can periodically send them in batches.

You need to find a balance between having fresh information in the search as fast as possible, and reducing the number of operations. This impacts your pricing and performance.

Handling concurrency

When working in a concurrent environment, you might need more control to guarantee data consistency. The engine supports built-in operations that allow for conditional updates based on versioning and optimistic locking.

Integrating with third-parties

You may use a framework or platform to power your application. Most of these solutions have their own databases and front ends. The goal is to send your data from these platforms to Algolia, and manage data updates and search configurations.

Algolia offers several official integrations to simplify the indexing process for popular frameworks and platforms. If you’re using one of the solutions listed below, you can install the corresponding extension to index your content.