Azure Functions Access to Restricted Storage Accounts
With requirements for additional security, many customers have started enforcing network restrictions on their Azure storage accounts. This in turn can cause major issues with Azure Functions without the proper configuration, as Azure functions rely heavily on Azure storage for functionality.
Background
Azure storage
There are two core concepts for Azure Functions related to Azure storage.
- The first is the AzureWebJobStorage app setting. The setting points to a general-purpose storage account and is used by the Webjob SDK’s host runtime for indexing, state store, and other functionality. This setting utilizes Azure blob. All hosting options require this app setting.
- The second is WEBSITE_CONTENTAZUREFILECONNECTIONSTRING. This app setting is required for the consumption and Function premium tiers, but is optional for the Dedicated SKU. The setting specifies where the function code will be hosted within a Azure Files storage account. In the dedicated SKU only, if not specified, the code will be stored in the App service infrastructure’s file system. In general, the recommendation is to not use this setting for the dedicated SKU.
Hosting Options
Consumption Plan
– True serverless that runs on shared compute.
– Does not support VNET integration.
– Code is hosted in Azure files
https://docs.microsoft.com/en-us/azure/azure-functions/consumption-plan
Functions Premium (Elastic premium)
– Hybrid SKU: Supports dedicated VMs with more compute power and dynamic scaling.
– Supports VNET integration
– Code is hosted in Azure Files
https://docs.microsoft.com/en-us/azure/azure-functions/functions-premium-plan?tabs=portal
Dedicated (Standard, premium, premiumV2/V3)
– Similar to your standard App Service SKU.
– Supports VNET integration.
– Code is by default hosted in App Service managed storage accounts. You can also configure the function to use Azure Files as well but it is not recommended.
https://docs.microsoft.com/en-us/azure/azure-functions/dedicated-plan
Why is Virtual Network Integration required for a Function App to communicate with storage that has “Select networks” enabled?
Services deployed in the same region as the storage account use private Azure IP addresses for communication to Azure storage. Thus, you can’t restrict access to specific Azure services based on their public outbound IP address range.
https://docs.microsoft.com/en-us/azure/storage/common/storage-network-security?tabs=azure-portal#grant-access-from-an-internet-ip-range
The next question I usually get is “Why can’t I just check ‘ Allow Trusted Azure Services’”. The answer is trusted Azure Services are limited to services that do not execute custom code or scripts.
So in order to make this communication between the function app and Azure storage behave in a way that one can restrict access to a known endpoint, the function has to be integrated with a VNET/subnet to then allow the subnet on the Azure storage’s firewall. After the VNET integration is completed, the subnet can be allowed on the Azure storage account’s “Select Networks” configuration allowing the communication to flow without issue.
Now that a baseline has been set explaining how functions use storage, the next section will discuss how one can properly integrate the function app with a VNET.
How it works with the two supported SKUs?
I reference service endpoints setup below but the same should work with private endpoints on storage as well.
Dedicated Function
Remember by default, with the recommended configuration, the Azure function app is only using Azure storage (specifically Azure blob) for indexing, state management, and other functionality. The function code is stored in the infrastructure’s storage. Since the code is not in the Azure storage account the function app may start, if the VNET integration has not been enabled, but the webjob/function host will not properly start as it cannot pull metadata from the storage account. You may see errors such as 401 unauthorized if you try and invoke your function inside of the function app, prior to the VNET integration being enabled.
- Create a function app in the Azure Portal. A storage account can be created or you can select a storage account with or without “Select Networks” enabled.
- Enable the Regional VNET integration on the newly created Azure function.
- On the subnet that the function app is integrated with, enable storage Service Endpoints.
- Under the storage account firewall, if you have not already, choose “Select Networks” and allow add the subnet you picked in step 2.
Premium Functions/Elastic Premium
The premium function SKU stores both the metadata and code in Azure storage (blob and files respectively). Since the function app requires access to Azure File storage to start the actual function app, if the storage account is locked down to only allow “Selected Networks” and if no integration is enabled, the function app will fail to startup and possibly even deploy, returning a 403 response code.
- Create a function app in the Azure Portal selecting a storage account that does not have “Select Networks” enabled. If you need to deploy the function app to a storage account with “Select Networks” enabled you may have to use an automated method such as an ARM template.
- Enable the Regional VNET integration on the newly created Azure function.
- Add the app setting WEBSITE_CONTENTOVERVNET and set the value to 1. This allows the connection to the storage account to be made through the VNET integration.
- On the subnet that the function app is integrated with, enable storage Service Endpoints.
- Under the storage account firewall, choose “Select Networks” and allow add the subnet you picked in step 2.
Happy integrating…
Hi,
thanks for your article – I was struggling to get my function app to work until I found your blog with the magic “WEBSITE_CONTENTOVERVNET” setting ! The only issue I now have is that I have deployment slots set (production and staging) with the staging slot CI integrated to GitHub repo. When I try and do a swap, it fails with a horrible error in a red block on the actual Swap tab. It appears to be related to the storage firewall as when I change it back to Allow Access from ‘All networks’ , the swap works fine. The error text is shown below (repeated several times in the red block) – any ideas of what this is would be gratefully received!
Failed to complete swap between slot ‘staging’ and slot ‘production’. Error: {“_body”:”{\”Code\”:\”InternalServerError\”,\”Message\”:\”There was an unexpected error swapping slots ‘staging’ and ‘production’ for site ‘func-sbcmonitoring(staging)’. Please try to cancel your swap operation.\”,\”Target\”:null
Hi Anthony,
I believe there’s a bug at the moment that blocks swap operations with this configuration that’s planned in the next release. I’d recommend creating a support case or create an question below to track if this is infact the issue. https://docs.microsoft.com/en-us/answers/topics/25345/azure-functions.html
I am facing exactly same issue when I try to swap the function slots.
I’d recommend opening a support case to track if the fix was released to support this functionality.
To anyone else who ended up here with the same error as Antony.
The solution for us was to add a setting of WEBSITE_OVERRIDE_STICKY_DIAGNOSTICS_SETTINGS=0 to both the production and staging slots.
Thanks Dom for providing the solution!
Hi Jeremy,
Thank-you for this excellent article.
Glad to see it helped!
Thanks a lot for this. I was also wondering if allow trusted services will help or not.
I have a function created in function App and I want to read that function code to create same function with same code.
is It possible how can I deploy azure function using any other source code which is publically accessible
If you are looking to use the same code across multiple functions without having to deploy multiple times I’d recommend using run from package from a blob storage account.
https://docs.microsoft.com/en-us/azure/azure-functions/run-functions-from-deployment-package#using-website_run_from_package–url
Fantastic blog as usual Jeremy !!
Great article. I’m not sure this statement is correct “The answer is trusted Azure Services are limited to services that do not execute custom code or scripts.” I can Allow Trusted Azure Services on an Azure SQL Server and use Logic Apps to communicate with the SQL Server. Logic Apps can definetly execute code and scripts and are considered Trusted Azure Service
I’m glad you found this helpful 🙂
This comment is specific to Azure Storage which I believe has a different “Trusted Services List” than SQL.
https://docs.microsoft.com/en-us/azure/storage/common/storage-network-security?tabs=azure-portal#trusted-access-for-resources-registered-in-your-subscription
In looking at this document it explicitly says the following, so I believe you are correct about the logic apps and SQL:
“When set to ON, your server allows communications from all resources inside the Azure boundary, that may or may not be part of your subscription.”
https://docs.microsoft.com/en-us/azure/azure-sql/database/network-access-controls-overview?view=azuresql#allow-azure-services
Really useful article, the documentation around the WEBSITE_CONTENT… settings seems strangely vague! I’m wondering if the following is something you’ve come across.
I’ve got a function app, it’s storage and a key vault all inside a vnet (with public access disabled). The function appsettings has a Key Vault Reference (I’ve tried both the SecretUri and the VaultName/SecretName approach).
If I have WEBSITE_CONTENTOVERVNET set to 1, I get this error at the top of the app settings page “Could not access key vault reference metadata”, if I set it to 0, it appears to work as expected and I see the reference statuses.
When I have WEBSITE_CONTENTOVERVNET set to 1, the actual function app is able to access secrets (even with the warning message on the appsettings page) so it seems I can ignore the error however it makes me a bit uncomfortable as it could mask actual issues.
Any ideas? Thank you!
Hi Chris, this isn’t something I’ve come across in the path. From what you’ve described it sounds like a portal UX issue but like you said, to sleep better at night I’d recommend a support case. Generally looking at a HAR trace (browser F12 developer tools) can give you insight into what API call is causing the specific response in the UX.
Hi Jeremy, I have a requirement that a function app needs to be created via ARM template, and while creating it the function code and associated files needed to be stored in the storage account file share, and in the storage account networking is “Enabled from selected virtual networks and IP addresses”. When I try to store the file while creating the app and I am getting an error as forbidden error:403 and the deployment fails. Now how can I store the files in files in the file share, Your suggestions, please. Thank you.
Hello. I haven’t worked a lot with CI/CD and automation of the is type of deployment yet. If its an ARM template I’d suspect though you’d want to use a zipdeploy or msdeploy task after the app has been integrated with the VNET to make sure the integration is completed prior to writing code. Have you tried using the DependsOn parameter to make the code deployment the last step after the infrastructure is deployed?