My experience with cold starts in Azure Functions Serverless is pretty awful. Like most other Azure services, their affordable consumer grade offerings are designed from the ground up not to be good enough for "serious" use.
Cold start times compared to Lambda are worse, and in addition, we would get random 404s which do not appear in any logs; inspecting these 404s indicated they were emitted by nginx, leading me to believe that the ultimate container endpoint was killed for whatever reason but that fact didn't make it back to the router, which attempted and failed to reach the function.
Of course the cold start and 404 are mitigated if you pay for the premium serverless or just host their middleware on their own App Service plans (basically VMs)
Same experience with Firebase. I just joined a team that has been using it. I've never worked with serverless before, and it boggles my mind how anyone thought it would be a good idea.
The cold starts are horrendous. In one case, it's consistently taking about 7 seconds to return ~10K of data. I investigated the actual runtime of the function and it completes in about 20ms, so the only real bottleneck is the fucking cold start.
Why didn't the Firebase minInstances work for you? I found amazing performance benefits, but at a $ cost. I actually forgot[1] a function to set a minInstance and a user complained that this particular functionality was slow (compare to the rest of the site). However, it isn't cheap.
You also want to be sure your code is optimized[2]. For example, don't require module packages unless the function needs it, else you're loading unnecessary packages. I usually set a global variable `let fetch;` and in the function that requires fetch initialize it `fetch = fetch ?? require("node-fetch);`.
I'm assuming by Firebase you mean Firebase Functions? We have a fairly complex infrastructure running off a combination of Firebase's RTDB, Firestore, and Google Cloud Functions and have never seen anywhere near what you're describing. Are you sure you're experiencing a 7 second "cold start", or is the invocation simply taking 7 seconds to run? Because the latter is far more easily explained.
I've confirmed that the actual execution of the function itself takes ~30ms, and that the time to download is standard (~200ms). That only leaves the cold start; nothing else makes sense.
edit: I even set up a node server on AppEngine itself, copied over the exact code to an endpoint, and it was taking 300-500ms tops (after the initial cold start of the AppEngine server).
I've seen 30s on AWS, so it's not that surprising. They have now improved it greatly though.
And yet I still believe it's a great technology, as always it's a matter of putting it on the right use case. Message consumption from a queue or topic, low traffic and low criticality API are two great use cases.
No, you've seen 30s on a random implementation running on AWS.
If you write your lambdas without knowing what you're doing then you can't blame the technology for being grossly misused by you.
Case in point: developing AWS Lambdas for the JDK runtime. The bulk of the startup time is not the lambda at all but the way the lambda code is initialized. This means clients and auth and stuff. I've worked on JDK Lambdas where cold starts were close to 20s due to their choice of dependency injection framework (Guide, the bane of JDK lambdas) which was shaven down to half that number by simply migrating to Dagger. I've worked on other JDK lambdas which saw similar reductions in cold starts just by paying attention to how a Redis client was configured.
Just keep in mind that cold start times represent the time it takes for your own code to initialize. This is not the lambda, but your own code. If you tell the lambda to needlessly run a lot of crap, you can't blame the lambda for actually doing what you set it to do.
That sort of time could be easily reached with lambdas that require VPC access, where a new ENI needs to be provisioned for each lambda container. I don't think alive seen 30s, but could easily see 5-10s for this case. And since this is required to run an isolated DB that is not some other AWS service, it isn't that uncommon. I believe they have since improved start times in this scenario significantly more recently.
And yet it magically came down to 10s when amazon improved their system. Specifically it became much faster to join a VPC.
And don't get me wrong: yes I was running some init code but not that much: load config from ssm, connect to a DB. I did bundle a lot of libs that didn't need to be there. But:
- fact is, it took 30s
- the use case didn't need it to be faster so I didn't care much
30s is probably an edge case. Did this use Java/JVM runtime without AOT/GraalVM? I cannot imagine any other runtime that would cause 30s cold start. Care to share more details on this?
That's what I thought. We've spent weeks investigating this one function and tried everything that Firebase recommends in their docs. Nothing has worked.
I'm extremely surprised to hear that. I know that there can be implementation differences, but on the level of application-code, this stuff is super simple. Create a javascript function then upload it. Not really much else to it, so I can't fathom what the difference is between your project and my own.
We're not doing anything crazy, it's just a basic CRUD application with minimal data (entire DB is less than 100MB at this point). And yet we're seeing constant, constant lags of several seconds for almost every single request. I can't explain it.
Never use Basic SKU, plan your network carefully before you even create the vnets, monitor NAT capacity, beware undocumented ARM rate limits. Good luck
> consistent and pervasive security model with Azure AD
Wait, this is the first time I hear this about Azure. Could you elaborate? It is possible that things have improved significantly since I last worked with Azure but lack of a consistent security model (like IAM on AWS) to control human and service (Azure Functions, App Service apps etc) access to specific resources (Cosmos databases, EventHubs etc) especially painful.
some of it is wonky, such as the login model for postgres on Azure SQL (you create login-capable postgres groups that exactly mirror the name of an Azure AD group, and then the "password" you pass in is actually a JWT proving YOU are in fact a member of that AD group -- so you have to hit a funky endpoint to get a time-limited "password")
I like Azure in general, but Function cold start times are really awful.
I regularly see start up times exceeding 10s for small, dotnet based functions. One is an auth endpoint for a self-hosted Docker registry, and the Docker CLI often times out when logging in if there is a cold start. I'm planning on moving these functions to Docker containers hosted in a VM.
I have other issues with Functions too. If you enable client certificates, the portal UI becomes pretty useless, with lots of stuff inaccessible. I have one such endpoint in production just now, and it's even worse than that, as every now and then it just... stops working until I manually restart it. Nothing useful in the logs either.
Azure Functions cold start times also depend on the underlying tech stack. I was using Python on a Linux host for Slack related Azure Functions and they ran into timeouts sometimes (which for the Slack API is 3s I think). After I switched to Nodejs on Windows I never got a timeout again.
For the Azure Functions consumption plan this can be mitigated to an extent by just having a keep alive function run inside the same function app (set to say a 3-5 minute timer trigger).
Azure Functions, in my opinion, should mostly be used in cases when you want to do some work over some time every now and then. It will also probably be cheaper to use something else in your case. In later versions of AF you can use a real Startup file to mitigate some life cycle related issues.
The way Azure Function scales out is different and is not entirely suited for the same goal as lambdas. Lambdas happily scale from 1 to 1000 instances in seconds* (EDIT: not A second), whereas Azure Functions just wont do that.
Last time I tried this was a few years ago, but seems like its still the case.
For an initial burst of traffic, your functions' cumulative concurrency in a Region can reach an initial level of between 500 and 3000, which varies per Region. After the initial burst, your functions' concurrency can scale by an additional 500 instances each minute. This continues until there are enough instances to serve all requests, or until a concurrency limit is reached. When requests come in faster than your function can scale, or when your function is at maximum concurrency, additional requests fail with a throttling error (429 status code).
That's just burst concurrency. It takes a lot longer that 1 second to actually reach the peak burstable limit. So while the Lambdas are scaling up your clients are either waiting several seconds, or getting 429s.
Cold start times compared to Lambda are worse, and in addition, we would get random 404s which do not appear in any logs; inspecting these 404s indicated they were emitted by nginx, leading me to believe that the ultimate container endpoint was killed for whatever reason but that fact didn't make it back to the router, which attempted and failed to reach the function.
Of course the cold start and 404 are mitigated if you pay for the premium serverless or just host their middleware on their own App Service plans (basically VMs)