Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I think what they're talking about is using one Lambda to persist, handling many crawl requests. That doesn't make a ton of sense, as each Lambda has to make sure to push all its results and shut down before the max timeout is up, but it would technically work and be more efficient than what I described. Much better, though, just to use long-running spot instances, though, or just a regular instance running something like Scrapy.


Consider applying for YC's Fall 2026 batch! Applications are open till July 27.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: