We have been using web frameworks to develop web applications since long before serverless came around, and middlewares are stable in these web frameworks. Express.js, for instance, lets you create middlewares at several stages of the request handling pipeline, and even ships with a few common middlewares out of the box.
As our code moves into Lambda functions and we move away from these web frameworks, are middlewares still relevant? If so, how might they look in this new world of serverless?
In this post, we’ll revisit the idea of middlewares, their role in application development with AWS Lambda, and how we can use middlewares to enforce consistent error handling across all of our Lambda functions.
The premise behind autoscaling in AWS is simple: you can maximize your ability to handle load spikes and minimize costs if you automatically scale your application out based on metrics like CPU or memory utilization. If you need 100 Docker containers to support your load during the day but only 10 when load is lower at night, running 100 containers at all times means that you’re using 900% more capacity than you need every night. With a constant container count, you’re either spending more money than you need to most of the time or your service will likely fall over during a load spike.
In the serverless world, we often get the impression that our applications can scale without limits. With the right design (and enough money), this is theoretically possible. But in reality, many components of our serverless applications DO have limits. Whether these are physical limits, like network throughput or CPU capacity, or soft limits, like AWS Account Limits or third-party API quotas, our serverless applications still need to be able to handle periods of high load. And more importantly, our end users should experience minimal, if any, negative effects when we reach these thresholds.
There are many ways to add resiliency to our serverless applications, but this post is going to focus on dealing specifically with quotas in third-party APIs. We’ll look at how we can use a combination of SQS, CloudWatch Events, and Lambda functions to implement a precisely controlled throttling system. We’ll also discuss how you can implement (almost) guaranteed ordering, state management (for multi-tiered quotas), and how to plan for failure. Let’s get started!
In 2018, I wrote a series of blog posts on building a multi-region, active-active, serverless architecture on AWS [1, 2, 3 and 4]. The solution was built using DynamoDB Global Tables, Lambda, the regional API Gateway feature, and Route 53 routing policies. It worked well as a resiliency pattern and as a disaster recovery (DR) strategy . But there was an issue.
In the engineering world a lot of our practices, even at times our best practices, are often just common wisdom passed along from one person to another. With Stack Overflow, Slack, and even Twitter, it’s easier today than it ever was for ideas to propagate. However, a lot of what passes for common wisdom is really just widely held opinions. And nothing says common wisdom has to be right. Where I ran into this distinction recently was with Python’s Boto3 modules (boto3 and botocore) and whether or not I should bundle them with my AWS Lambda deployment artifact.
Recently I found out the common wisdom I’ve adhered to was wrong. (Yes, someone on the internet was wrong.) Like many people, I use the Boto3 modules provided by the AWS Lambda runtime. However after talking with several folks at AWS I discovered, you should not be using the AWS Lambda runtime’s boto3 and botocore module. And you shouldn’t use botocore’s vendored version of the requests module whether no matter what instance of botocore you are using. I’ll explain how I found this out and explore why more than just me have probably gotten this best practice wrong.
The Game Engine Black Book: DOOM features a whole chapter about DOOM console ports and the challenges they encountered. The utter failure of the 3DO, the difficulties of the Saturn due to its affine texture mapping, and the amazing “reverse-engineering-from- scratch” by Randy Linden on Super Nintendo all have rich stories to tell.
Once heading towards disaster, the Playstation 1 (PSX) devteam managed to rectify course to produce a critically and commercially acclaimed conversion. Final DOOM was the most faithful port when compared to the PC version. The alpha blended colored sectors not only improved visual quality, they also made gameplay better by indicating the required key color. Sound was also improved via reverberation effects taking advantage of the PSX’s Audio Processing Unit.
The devteam did such a good job that they found themselves with a few extra CPU cycles they decided to use to generate animated fire both during both the intro and the gameplay. Mesmerized, I tried to find out how it was done. After an initial calling found no answer, I was about to dust off my MIPS book to rip open the PSX executable when Samuel Villarreal replied on Twitter to tell me he had already reverse-engineered the Nintendo 64 version. I only had to clean, simplify, and optimize it a little bit.
It was interesting to re-discover this classic demoscene effect; the underlying idea is similar to the first water ripple many developers implemented as a programming kata in the 90’s. The fire effect is a vibrant testimony to a time when judiciously picked palette colors combined with a simple trick were the only way to get things done.