Prevent Back-End Rendering

General

In order to make the Magento search more performant and faster, it’s possible to prevent the rendering in the back end of the search result pages. With back-end rendering turned off, the search result pages will be powered by the Instant Search feature. When the page is sent to the users’ browser, it will immediately re-render using JavaScript.

This feature is experimental. We can’t guarantee it will work in all use cases, so please test thoroughly before using this in production.

Advantages

The main advantage to using this approach is that the search result page will not be rendered on the server before sending it to the user. This means the server will respond faster and with a smaller footprint, which has a positive effect on loading times and user experience. Because of this, the server will also consume less resources to handle the response.

Disadvantages

Most website crawlers - including some search crawlers like Google, Bing, etc. - rely on the website code being generate on the server. These bots can have problems with pages that are being generated by JavaScript since they are not (always) able to execute this. Because not all crawlers are able to execute JavaScript, some pages might not be indexed or links might not be followed.

This can be prevented.

Crawlers can be identified on the server by looking at the User-Agent header. Almost all crawlers have a unique value, that normal users would not have.

Most of the website crawlers (including search crawlers like Google, Bing, …) rely on the website code generated on server and cannot properly crawl a page which is generated by only JavaScript on front end. In the extensions’ configuration, it’s possible to specify a list of User Agents for which the back-end rendering still needs to happen, thus preventing the issue of pages not being followed. The extension will check the User Agent of the request to see if it is in the list provided. Click here for a list of User Agents in use by crawlers.

The extension will, by default, turn on back-end rendering for Googlebot and Bingbot.

Google Search Console and Bing Webmaster Tools are tools to keep an eye on the amount of indexed pages if it’s not clear whether the crawlers index the pages.

Enabling

To prevent back-end rendering, navigate to Stores > Configuration > Algolia Search > Advanced in the Magento admin section. This tab contains a setting called Prevent back-end rendering?, which can be switched to Yes to activate this feature.

When this setting is set to yes, a list of User-Agents can be filled in for which the back-end rendering should still be enabled. By default, the list contains “Googlebot” and “Bingbot”.

Behind the scenes

With this feature enabled, the extension will remove the template blocks from rendering through the extensions’ layout XML file.

The following blocks will be removed from the content container:

• category.products
• search.result

The following containers will be removed from the template:

• sidebar.main
• sidebar.additional