Fixing Solr Azure App Service Core Locking Issues

solr-core-locking-error

We have had intermittent issues with our Sitecore Solr instance in Production which runs in an Azure App Service. Every so often it crashes and we see Core Initialisation errors like so:

sitecore_web_index: org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: Index dir 'D:\home\site\wwwroot\server\solr\sitecore_web_index\data\index/' of core 'sitecore_web_index' is already locked. 
The most likely cause is another Solr server (or another solr core in this server) also configured to use this directory; 
other possible causes may be specific to lockType: native

Doing some research into this it seems that the consensus is that Azure App Services shouldn’t really be used for Solr:  https://sitecore.stackexchange.com/questions/23466/solr-cores-locking-v-9-0-2

However I couldn’t actually find any official Sitecore documentation that says that Azure App Services are not supported / recommended for Solr and there are plenty of blog posts out there about setting up Solr for Sitecore in Azure App Services.

We could have switched it to an VM but we’d rather not do that as it’s easier to manage and maintain an Azure App Service than an VM and Microsoft will also handle all the security patches too etc.

So I opened a support ticket with Sitecore to confirm if App Services can be used for Solr and they suggested that I try the following:

1. Make sure the Azure Web App is not scaled out (instance count should be set to 1).
2. Make sure the environment variable WEBSITE_DISABLE_OVERLAPPED_RECYCLING is set to 1

We already have our App Service scaled to just one instance but when checking the configuration settings we didn’t have the WEBSITE_DISABLE_OVERLAPPED_RECYCLING setting set.
So we added this config and set the value to 1:

solr-recycling-config
Setting this to 1 stops new instances of the app being started before the current instance is shut down – which can cause locking.

Since making this change we’ve not had any issues with it intermittently locking.

Update: I still had issues with cores locking occasionally and started on a solution for it using PowerShell in a Runbook a while ago and I have now completed this and shared it here: https://www.flux-digital.com/blog/auto-re-starting-solr-indexes-get-locked-azure-app-service/

Hope this helps someone else who’s having this issue too.