Lambda functions over S3 objects with concurrency control (forEach, map, reduce, filter)

programming 1 Minute

s3-lambda enables you to run lambda functions over a context of S3 objects. It has a stateless architecture with concurrency control, allowing you to process a large number of files very quickly. This is useful for quickly prototyping complex data jobs without an infrastructure like Hadoop or Spark.

At Littlstar, we use s3-lambda for all sorts of data pipelining and analytics.

https://github.com/littlstar/s3-lambda

Published by irrlab

https://about.me/irocha

Published