We have had intermittent issues with our Sitecore Solr instance in Production which runs in an Azure App Service. Every so often it crashes and we see Core Initialisation errors like so:
sitecore_web_index: org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: Index dir 'D:\home\site\wwwroot\server\solr\sitecore_web_index\data\index/' of core 'sitecore_web_index' is already locked. The most likely cause is another Solr server (or another solr core in this server) also configured to use this directory; other possible causes may be specific to lockType: native
Doing some research into this it seems that the consensus is that Azure App Services shouldn’t really be used for Solr: https://sitecore.stackexchange.com/questions/23466/solr-cores-locking-v-9-0-2
However I couldn’t actually find any official Sitecore documentation that says that Azure App Services are not supported / recommended for Solr and there are plenty of blog posts out there about setting up Solr for Sitecore in Azure App Services.
We could have switched it to an VM but we’d rather not do that as it’s easier to manage and maintain an Azure App Service than an VM and Microsoft will also handle all the security patches too etc.
So I opened a support ticket with Sitecore to confirm if App Services can be used for Solr and they suggested that I try the following:
1. Make sure the Azure Web App is not scaled out (instance count should be set to 1).
2. Make sure the environment variable WEBSITE_DISABLE_OVERLAPPED_RECYCLING is set to 1
We already have our App Service scaled to just one instance but when checking the configuration settings we didn’t have the WEBSITE_DISABLE_OVERLAPPED_RECYCLING setting set.
So we added this config and set the value to 1:
Setting this to 1 stops new instances of the app being started before the current instance is shut down – which can cause locking.
Since making this change we’ve not had any issues with it intermittently locking.
Update: I still had issues with cores locking occasionally and started on a solution for it using PowerShell in a Runbook a while ago and I have now completed this and shared it here: https://www.flux-digital.com/blog/auto-re-starting-solr-indexes-get-locked-azure-app-service/
Hope this helps someone else who’s having this issue too.
We also had the same issue with our instance in Azure PaaS for more than 2 years now. I think every month this issue occurs once or twice at least. All we can do is to restart the app service. I will be implementing your suggested fix, hopefully this will finally resolve it. Thanks mate!
Hopefully this will work for you. We did continue to see issues sometimes but this certainly helped and we saw less issues.
I actually prototyped an Azure Automation to check the Solr app service regularly and re-start the app service when it errored and have been meaning to update this blog post to add that as an alternative approach but haven’t got round to it yet. I will do so though soon.