Serverless async back ends for compute-heavy operations

Hacker News

Share card

Serverless async back ends for compute-heavy operations

Hi HN, we’re Jessie and Eric. We’ve been baking away at Cakework ( https://www.cakework.com/ ), which is a way to build async backends without needing to manage cloud infrastructure. Cakework is for operations that take time or more compute, like file processing, report generation, or machine learning. Devs write backends as Python functions and deploy them with our CLI. They use our client SDKs to make requests, get status, and get processing results. Each request runs with its own CPU and memory parameters in its own microVM, with no timeouts. Devs can query for failures and view logs, inputs, and outputs for each request. Under the covers, we package and deploy code as Docker containers. We queue each request on a NATs cluster, and spin up a Fly Machine ( https://fly.io/docs/machines/ ) to process it. Cakework is open source ( https://github.com/usecakework/cakework ) if you want to dig in! We started exploring Cakework because we liked the idea of serverless for compute-heavy operations, but in practice found that an application made up of wiring together queues, Lambdas, storage, and step functions made iteration really slow. We also didn’t like having to switch away from Lambda when operations ran longer than fifteen minutes. If you want to give it a whirl, you can follow the quickstart on our website ( https://www.cakework.com/ ), or check out the docs ( https://docs.cakework.com/gettingstarted ). We’re super excited to share an early build and get everyone’s thoughts, thanks for checking it out!