Using API Gateway WebSockets with the Serverless Framework

As we approach the end of 2018, I’m incredibly excited to announce that we at Serverless have a small gift for you: You can work with Amazon API Gateway WebSockets in your Serverless Framework applications starting right now.

But before we dive into the how-to, there are some interesting caveats that I want you to be aware of.

First, this is not supported in AWS CloudFormation just yet, though AWS has publicly stated it will be early next year! As such, we decided to implement our initial support as a plugin and keep it out of core until the official AWS CloudFormation support is added.

Second, the configuration syntax should be pretty close, but we make no promises that anything implemented with this will carry forward after core support. And once core support is added with AWS CloudFormation, you will need to recreate your API Gateway resources managed by CloudFormation. This means that any clients using your WebSocket application would need to be repointed, or other DNS would have needed to be in place, to facilitate the cutover.

I recommend you check out my original post for a basic understanding of how WebSockets works at a technical level via connections and callbacks to the Amazon API Gateway connections management API.

With all that out of the way, play with our new presents!


AWS Lambda in a VPC Will Soon be Much Faster

One of the most common pains for users of AWS Lambda is cold starts. Cold starts add unwanted delays to Lambda invocations, and in cases where a Lambda is used inside of a Virtual Private Cloud (VPC), the latency can be as high as several seconds. This practically negates the speed benefits of Lambda functions.

Fortunately, the Lambda team announced at AWS re:Invent 2018 that they are changing the architecture of Lambdas running in a VPC in order to reduce this latency and make Lambdas start much faster.

Making an Unlimited Number of Requests with Python aiohttp + pypeln

This post is a continuation on the works of Paweł Miech’s Making 1 million requests with python-aiohttp and Andy Balaam’s Making 100 million requests with Python aiohttp. I will be trying to reproduce the setup on Andy’s blog with some minor modifications due to API changes in the aiohttp library, you should definitely read his blog, but I’ll give a recap.

UPDATE: Since Andy’s original post, aiohttp introduced another API change which limited the total number of simultaneous requests to 100 by default. I’ve updated the code shown here to remove this limit and increased the number of total requests to compensate. Apart from that, the analysis remains the same.

EC2 Network Performance Cheat Sheet

EC2 Network Performance Cheat Sheet

What is the maximum network throughput of your EC2 instance? The answer to this question is key to choosing the type of an instance or defining monitoring alerts on network throughput. Unfortunately, you will only find very vague information about the networking capabilities of EC2 instances within AWS’s service description and documentation. That is why I run a network performance benchmark for almost all EC2 instance types within the last few days. The results are compiled into the following cheat sheet.

On Incomplete HTTP Reads and the Requests Library In Python

The requests library is arguably the mostly widely used HTTP library for Python. However, what I believe most of its users are not aware of is that its current stable version happily accepts responses whose length is less than what is given in the Content-Length header. If you are not careful enough to check this by yourself, you may end up using corrupted data without even noticing. I have witnessed this first-hand, which is the reason for the present blog post. Lets see why the current requests version does not do this checking (spoiler: it is a feature, not a bug) and how to check this manually in your scripts.

Integration layer between Requests and Selenium for automation of web actions

Requestium is a python library that merges the power of Requests, Selenium, and Parsel into a single integrated tool for automatizing web actions.

The library was created for writing web automation scripts that are written using mostly Requests but that are able to seamlessly switch to Selenium for the JavaScript heavy parts of the website, while maintaining the session.

Requestium adds independent improvements to both Requests and Selenium, and every new feature is lazily evaluated, so its useful even if writing scripts that use only Requests or Selenium.


  • Enables switching between a Requests’ Session and a Selenium webdriver while maintaining the current web session.
  • Integrates Parsel’s parser into the library, making xpath, css, and regex much cleaner to write.
  • Improves Selenium’s handling of dynamically loading elements.
  • Makes cookie handling more flexible in Selenium.
  • Makes clicking elements in Selenium more reliable.
  • Supports Chrome and PhantomJS.