Tools / Crawler / Plugin Configuration

Plugin Configuration

A

The plugin is configurable via the netlify.toml file and the environment variables of your Netlify Site.

Usage# A

Copy 
1
2
3
4
5
6
7
8

[[plugins]]
package = "@algolia/netlify-plugin-crawler"
  [plugins.inputs]
  branches = ['main', 'develop', 'feat/add-algolia']
  disabled = true
  pathPrefix = "/blog"
  customDomain = "example.com"
  renderJavaScript = true

Available parameters# A

netlify.toml#

Plugin inputs can be set in netlify.toml. They’re all optional.

Options #

branches #
type: string[]
default: ['master', 'main']
Optional

List of branches the crawler should build. By default, the plugin only builds your main branch, but you can use this to build multiple branches. Each branch has a dedicated Algolia index, named netlify_<site-id>_<branch-name>_all. You need to target the right branch in your front-end code. You can use wildcards in the branch names too.

Copy 
1
2
3

branches = ['develop']
branches = ["*"]
branches = ["feat/*"]
mainBranch #
type: string
Optional

The main project’s branch. Used to propagate your Algolia index settings modifications to other branches if set. Settings of the Algolia index of your main branch will be used to create the new index when the plugin runs for the first time on a new branch.
pathPrefix #
type: string
Optional

The prefix of your website if it’s not at the root level.

Copy 
1

  pathPrefix = "/blog" # will alias `/blog` to `/`
customDomain #
type: string
Optional

The custom domain that you use, if it’s not possible to define it through Netlify settings.

Copy 
1

  customDomain = "example.com" # will alias `example.com` to `<your-site-url>.netlify.app`
renderJavaScript #
type: boolean
default: false
Optional

If true, the crawler uses JavaScript to render your website. Useful for single page applications. See the renderJavaScript documentation to understand the implication of this option.
template #
type: string
Optional

Used to modify the way we extract records and their schema. Supported templates: 'hierarchical'. More information on how the extraction works in the dedicated extraction documentation.

Copy 
1

template = "hierarchical"
disabled #
type: boolean
default: false
Optional

Set to true to turn off the plugin without removing it.

Environment variables#

The environment variables that needs to be set in your Netlify Site settings. They will be automatically added by the plugin.

Options #

ALGOLIA_BASE_URL #
type: string
Required

URL to target, should be “https://crawler.algolia.com/”.
ALGOLIA_API_KEY #
type: string
Required

API Key to authenticate the call to the crawler.
ALGOLIA_DISABLED #
type: boolean
default: false
Optional

Set to true to turn off the plugin without removing it.
Previous
Front-end configuration
Next
Extraction strategy

Did you find this page helpful?