Azure @ Enterprise - Serverless scale problem - Functions in App Service Environment connecting to Azure SQL

This post is about working with Azure Functions in App Service Environment hereafter ASE. Not sure it is qualified to call a classic problem but we all get into trouble when we use the serverless scale model. That is mainly due to the inability to scale by the dependencies of the Serverless compute tier.

To be more clear the Azure Functions will scale out but underlying dependencies such as Azure SQL will not scale with Functions and eventually lead to system failure.

Problem

Let us take a scenario. There is Azure Function that is invoked by the Azure storage queue. There will be one instance of Function running all the time. When the queue has more than 100 messages in the queued state, the Function will scale out to 4 instances to complete processing. Then scale down to 1. 

Each Azure Function instance can dequeue the maximum of 32 messages in a batch.32 is a hard limit.

When the scale-out happens the Azure SQL fails as it cannot accept more connections.

Observations

The SQL server sp_who2 command shows there are a lot of connections in the sleeping state from the Function instances.

The Azure SQL is in General Purpose Gen 5, 4 vCores tier. It can accept only 400 connections.

Root cause

.Net has a default connection pool size of 100. There are other applications also connecting to the same database apart from the Function. 4 Function instances wanted to establish 4x100=400 connections which are not possible with the Azure SQL tier as the other applications also making a similar number of connections. Hence it says cannot accept more connections.

Solution(s)

There can be multiple solutions

• Scale up the Azure SQL tier to General Purpose Gen 5, 8 vCores that can accept 800 concurrent connections.
• Throttle the .Net connection pool to 32 or 64. This number is based on the maximum number of messages dequeued by one Function instance. So max connections from all 4 pools will be maxed 128 or 256. There are some left for the other applications as well.
• The Serverless Azure SQL tier is an option. But please understand that the Serverless compute has a warmup delay after idle usage periods hence best suited for intermittent and unpredictable workloads that can afford this delay.

Happy Serverless

Azure @ Enterprise - Functions v/s AWS Lambda

Serverless @ enterprise

Serverless is a buzzword in the software industry right now after Microservices. As with any other technology revolution, startups will jump into it and Enterprise will only be entering to Serverless slowly. Earlier the technologies would be Enterprise ready from the starting itself. But now a days it seems Enterprise support is little behind at the technology edge. 

This post is an attempt to examine the Serverless offering is ready for enterprise. Since the series is Azure @ Enterprise, lets see from the Azure side.

Cost

If we consider cost, that was not a big factor for Enterprise. They usually buy big boxes to meet their peak load and enjoy during the normal load and struggle during the heavy load. Whatever capacity planning we do there are chances that the real load differ because of  unknowns.

But with the Cloud adoption, enterprise started thinking about cost. If they had embraced 2 pizza team strategy or Microservice architecture style or devOps teams there are high chances that small teams started maintaining services / applications end to end. In order to prove their success they are forced to reduce cost in the operations. Ultimately cost becomes a factor in Enterprise.

This think enterprise to adopt Serverless or true pay per use cloud offerings. At present in Azure, Functions are the real pay per use service. Hence the title limited to Functions. Cost wise, Functions has got good score.

Versioning

The versioning might be a concern if the enterprise is using blue green deployment where the applications move to production and there might be some rollbacks required.

In Azure Functions there is no versioning concept except pulling from the source control repo. But in Amazon lambda which is the Function equivalent in Amazon cloud, there is versioning. We can have smooth version control of Functions in even in the production environment.

https://docs.microsoft.com/en-us/azure/azure-functions/durable-functions-versioning

https://docs.aws.amazon.com/lambda/latest/dg/versioning-intro.html

Security

Next factor is securing the Enterprise. There will be less chance for the Enterprise to compromise on security because something is cheap. If we take that thought to Functions, there are 2 challenges with Functions right now.

Securing external internet facing Functions

This can be done via API Gateways. There is not much challenge in that once the suitable gateway is selected. It ensure there is no DDoS, brute force, injections etc...

Securing internal Functions

There could be lot of internal services which are supposed to be exposed inside Enterprise. Normally on premise network would be protected using appropriate measures. The end points might not be visible to outside. This is the area Functions lag.
Suppose we need to have an internal Function there are 2 options. One is to setup the firewall rules. Second to host inside vNet or virtual network. Then the Functions end point will not be accessible outside.
The best way is to use vNet but the problem here is pay per use Functions don't support hosting inside vNet unless they are under AppServiceEnvironment (ASE). AppServiceEnvironment provides an isolated environment for the Enterprise to host things. But the problem with ASE is that it is highly costly. around $1200/month. The real problem is that the cost is fixed regardless of the usage. We lost everything we talked about Serverless if it is fixed billing.

Amazon Lambda

Amazon have VPC instead of vNet. It doesn't seems there is a fixed high cost if we wanted to host Lambda inside the VPC. Please note the information is obtained from google and feel free to correct. 

Some links below on Lambda and VPC

https://aws.amazon.com/premiumsupport/knowledge-center/internet-access-lambda-function/
https://aws.amazon.com/blogs/apn/why-use-aws-lambda-in-a-custom-vpc/

More differences

There are more links out there which compare Azure Function with Lambda and Cloud Functions from Google cloud.

https://cloudacademy.com/blog/microsoft-azure-functions-vs-google-cloud-functions-fight-for-serverless-cloud-domination-continues/

Azure @ Enterprise - Is Azure Functions Enterprise ready?

"It Doesn't Work That Way in Enterprise". This is a famous quote when we work in enterprise. Many things which does in a tick at startups never work that way in enterprise. Coming to Azure development, if we want to create a new Azure Function it might be just an entry in ARM template in startup and next deployment will have the Azure Function, but it might be 2 weeks process in Enterprise. Sometimes the Azure Function might have to be created manually based on tickets in Enterprise after multi level approvals and refer its name in ARM.

The focus of this post is 'Is Azure Function Enterprise ready?' Azure Functions is the flagship product from Microsoft to meet the demands of Serverless world.

Signing the assemblies

The basic thing in .Net is to sign our assemblies before shipping them. If we use third party libraries and those contain unsigned assemblies, we cannot sign our assemblies. 

Why we are discussing this basics with Functions? The current Azure Functions SDK which is coming via nuget package has references / dependencies which are not signed!!!

At the time of writing this post, Visual Studio 2017 (15.4.0) creates Azure Function projects which has references to unsigned dlls.

To be more precise, SDK has dependency to Microsoft.Azure.WebJobs.Extensions.Http which has the assembly with same name. If we open the assembly in JustDecompile, we can see the below.

Microsoft.Azure.WebJobs.Extensions.Http, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null

It is reader choice to decide whether a SDK with unsigned assembly is Enterprise ready.

Beta dependencies of Azure Functions SDK

No need to explain in detail. The Function SDK has dependency Microsoft.Azure.WebJobs.Extensions.Http which is still in beta1 stage. Yes its the same which has unsigned assembly referred above. The latest Functions.SDK version available at the time of writing this post in nuget repo is 1.0.7 and it still has this beta reference. Refer the below link for dependencies. The SDK is in production but not the dependencies.

https://www.nuget.org/packages/Microsoft.NET.Sdk.Functions/1.0.7

Another thing Enterprise loves to put into production.

Pricing & vNet capability

If a service is internal ie called by other front end services or scheduled tasks etc..., enterprise never want to put it publicly. The solution in Azure Function to have internal service is to use the AppServiceEnvironment+AppServicePlan combination. What we loose there is the cost factor. ASE has fixed money in it and never pay per use.

Execution timeout of Functions

At the of writing this post the max timeout of a Function is 10 mins if it is in consumption plan ie the real pay per use model. But Enterprise may need to execute long running operations greater than 10 mins if there is queuing system or scheduled processes and those should be strictly internal. In order to achieve it they can never use pay per use model of Functions, instead use either ASE+Function or WebJobs. It would be considered as road block by enterprise users.

It seems Microsoft wanted to have an answer to Amazon Lambda in the Serverless world and someone hacked Azure WebJobs to become Azure Functions. But its really excellent idea for start ups to reduce time to market and reduce upfront investments.

Comparing Amazon v/s Azure Function for enterprise use is altogether separate topic.

Disclaimer

The Azure evolves at a fast pace and the facts might not be relevant in future.