Ideally this could have been a Twitter tweet.
Never make a intermediate component Serverless that auto scale
Serverless¹ is relatively new buzzword in the industry. We normally jump into serverless offerings as they are really attractive to manage and operate. The main feature that comes with Serverless is the auto scale. The Serverless compute that host out code is expected to scale out as the traffic increase.
Problem
If everything is that easy what is the problem? Let us consider the below architecture.
In case the architecture is not understandable, leave a comment. At first as a developer, we may not see any problem. But let us see this from the operations perspective.
- It runs most of the day time
- Intermittent failure starts in the evening
- In the night it works fine again
Root cause
Debugging this problem would not be a herculean task as far as there is proper logging.
The S3 Azure SQL tier has limits ² on how many connections. During the day there are only users the load they generate can be handled by S3 tier. Evening there is overlap. Demon apps starts and there are still users. Then it fails. Again in the night users are logged out and it serves all daemon apps.
The catch here is the Azure Functions will auto scale but the SQL cannot hence fails.
No comments:
Post a Comment