🐾Serverless Big data processing🐾
✅ Step Functions Distributed Map state allows you to orchestrate large-scale parallel workloads in a Serverless fashion. You can use it in any step of your StepFunction workflow.
Capabilities:
🔹maximum concurrency of up to 10,000 executions in parallel
🔹optimized to work with files stored on S3
🔹ability to process large files with Lambda Power tools streaming feature
🔹separate execution history from that of the parent workflow
Tutorial on how to use the Map state in Distributed mode.
If you like this post, you can share APAWS newsletter with friends: