The premise behind autoscaling in AWS is simple: you can maximize your ability to handle load spikes and minimize costs if you automatically scale your application out based on metrics like CPU or memory utilization. If you need 100 Docker containers to support your load during the day but only 10 when load is lower at night, running 100 containers at all times means that you’re using 900% more capacity than you need every night. With a constant container count, you’re either spending more money than you need to most of the time or your service will likely fall over during a load spike.
In the serverless world, we often get the impression that our applications can scale without limits. With the right design (and enough money), this is theoretically possible. But in reality, many components of our serverless applications DO have limits. Whether these are physical limits, like network throughput or CPU capacity, or soft limits, like AWS Account Limits or third-party API quotas, our serverless applications still need to be able to handle periods of high load. And more importantly, our end users should experience minimal, if any, negative effects when we reach these thresholds.
There are many ways to add resiliency to our serverless applications, but this post is going to focus on dealing specifically with quotas in third-party APIs. We’ll look at how we can use a combination of SQS, CloudWatch Events, and Lambda functions to implement a precisely controlled throttling system. We’ll also discuss how you can implement (almost) guaranteed ordering, state management (for multi-tiered quotas), and how to plan for failure. Let’s get started!
In 2018, I wrote a series of blog posts on building a multi-region, active-active, serverless architecture on AWS [1, 2, 3 and 4]. The solution was built using DynamoDB Global Tables, Lambda, the regional API Gateway feature, and Route 53 routing policies. It worked well as a resiliency pattern and as a disaster recovery (DR) strategy . But there was an issue.
In the engineering world a lot of our practices, even at times our best practices, are often just common wisdom passed along from one person to another. With Stack Overflow, Slack, and even Twitter, it’s easier today than it ever was for ideas to propagate. However, a lot of what passes for common wisdom is really just widely held opinions. And nothing says common wisdom has to be right. Where I ran into this distinction recently was with Python’s Boto3 modules (boto3 and botocore) and whether or not I should bundle them with my AWS Lambda deployment artifact.
Recently I found out the common wisdom I’ve adhered to was wrong. (Yes, someone on the internet was wrong.) Like many people, I use the Boto3 modules provided by the AWS Lambda runtime. However after talking with several folks at AWS I discovered, you should not be using the AWS Lambda runtime’s boto3 and botocore module. And you shouldn’t use botocore’s vendored version of the requests module whether no matter what instance of botocore you are using. I’ll explain how I found this out and explore why more than just me have probably gotten this best practice wrong.
The Game Engine Black Book: DOOM features a whole chapter about DOOM console ports and the challenges they encountered. The utter failure of the 3DO, the difficulties of the Saturn due to its affine texture mapping, and the amazing “reverse-engineering-from- scratch” by Randy Linden on Super Nintendo all have rich stories to tell.
Once heading towards disaster, the Playstation 1 (PSX) devteam managed to rectify course to produce a critically and commercially acclaimed conversion. Final DOOM was the most faithful port when compared to the PC version. The alpha blended colored sectors not only improved visual quality, they also made gameplay better by indicating the required key color. Sound was also improved via reverberation effects taking advantage of the PSX’s Audio Processing Unit.
The devteam did such a good job that they found themselves with a few extra CPU cycles they decided to use to generate animated fire both during both the intro and the gameplay. Mesmerized, I tried to find out how it was done. After an initial calling found no answer, I was about to dust off my MIPS book to rip open the PSX executable when Samuel Villarreal replied on Twitter to tell me he had already reverse-engineered the Nintendo 64 version. I only had to clean, simplify, and optimize it a little bit.
It was interesting to re-discover this classic demoscene effect; the underlying idea is similar to the first water ripple many developers implemented as a programming kata in the 90’s. The fire effect is a vibrant testimony to a time when judiciously picked palette colors combined with a simple trick were the only way to get things done.
Serverless architecture is the new kid on the block, and according to a recent surveyby Serverless, Inc., a vast majority of developers will start using it by the end of the year. The serverless paradigm involves running code in the cloud without managing any servers, allowing you to build business logic and create value without ever thinking about the infrastructure or underlying software. Essentially, it lets you focus on your code.
Serverless does not only cover AWS Lambda and other FaaS providers, but basically everything you can use to run code, host files, and store images and data. This means that you, as an engineer, don’t need to manage, scale, or operate any servers whatsoever. And here’s the icing on the cake: you only pay for the time your code is running!
Although serverless offers many benefits, there are still some pitfalls, such as latency. In this article, we’ll discuss how to minimize latency in AWS Lambda. This dreaded phenomenon is caused by cold starts, which are, by definition, slower initial responses from your serverless APIs.
Before we begin, let’s dig deeper into what FaaS is and how it works.
Recommender systems (or recommendation engines) are useful and interesting pieces of software. I wanted to compare recommender systems to each other but could not find a decent list, so here is the one I created. Please help me keep this post up-to-date by submitting corrections and additions via pull-request, or tweet me @grahamjenson.