Keep warm

[j08lue]

Zappa by default keeps the function warm, it seems: https://github.com/Miserlou/Zappa/blob/3ccf7490a8d8b8fa74a61ee39bf44234f3567739/zappa/cli.py#L1169

But we are still seeing long cold starts. Why could that be?

[dionhaefner]

Because it only keeps one instance warm? Terracotta pretty much always fires at least a couple of requests at once, so during a cold start only the first request hits the warmed-up instance, while the others trigger a cold start of one new instance each.

[dionhaefner]

Since it's one of the fundamental downsides to Lambda there's not much we can do for low-traffic services. Maybe display a "please stand by" banner in the client.

[mrpgraae]

Because it only keeps one instance warm?

If it's low traffic, then it shouldn't be an issue that it only keeps one instance warm.
Relevant issue here: Miserlou/Zappa#851

[mrpgraae]

Because it only keeps one instance warm? Terracotta pretty much always fires at least a couple of requests at once, so during a cold start only the first request hits the warmed-up instance, while the others trigger a cold start of one new instance each.

Wrote my previous comment before your edit @dionhaefner, I see the issue there.

[mrpgraae]

This guy has a custom keep-alive script that he uses to keep 10 containers warm https://www.zrzka.dev/2016/10/30/aws-journey-api-gateway-lambda-vpc-performance.html

[dionhaefner]

If you keep too many containers warm it just becomes a more expensive version of EC2. Don't know how much there's to gain (10 might be fine though).

[mrpgraae]

If you keep too many containers warm it just becomes a more expensive version of EC2. Don't know how much there's to gain (10 might be fine though).

I think it would have to be quite a lot of containers we would be keeping warm, for it to be more expensive than EC2? According to the blog post i just linked, we only have to fire n requests about every 15 minutes, where n is the number of containers to keep alive. Remember that the first million requests per month are free and that we are talking about low-traffic services here.

[mrpgraae]

For 10 containers, we are talking ~30000 requests per month

[mrpgraae]

But I don't think that this should be a Terracotta feature? Seems more like something you would set up on the side. Maybe we can mention it in the docs.

[dionhaefner]

Yep. It would be cool if you could do some testing with that.

[mrpgraae]

Yep. It would be cool if you could do some testing with that.

Will do.

[j08lue]

I thought once there was one warm container, spinning up more was comparatively cheap because they just clone the existing one (rather than cold-starting new instances from scratch). So all you would need is a seed. But maybe cloning is still too expensive (we are talking many concurrent requests for TC) or my understanding was wrong.

[dionhaefner]

If there is such as thing as cloning (and this is the first time I hear about it) it's not effective enough 😕

[j08lue]

A Microsoft guy talked about that the other day. Aparently Azure Functions do that. I took it for general practice.

[mrpgraae]

That was also how I understood it, that once the first container was running, they basically just did a memcpy to spawn the others.

[j08lue]

https://read.acloud.guru/how-to-keep-your-lambda-functions-warm-9d7e1aa6e2f0