How to build a serverless clone of Imgur using Amazon Rekognition and DynamoDB

In a previous article, we managed to build a very simple and somewhat primitive Imgur clone — using Amazon Cognito for registration and login before uploading images to the site for all to see.

Now, it had a few issues and these must be addressed before we go on to any funding rounds. We don’t want to scare away any potential investors with a few teething issues.

The issues preventing funding

Let’s go through the issues that need to be resolved prior to a round of Series A funding from any potential investors.

  1. In order to render the home page, it would hit the s3 bucket storing all of these images and then return them as a big JSON list. No pagination, no smaller images. If this thing is going to scale in any real sense then this will have to be addressed. We will have to introduce a database and proper pagination of results.
  2. It doesn’t really do anything “cool”. In order to address this, I thought I’d play around with AWS Rekognition and see if we could add some machine learning image recognition to the site. We can then browse images based on type should we so wish!
  3. There were a couple of frontend things that could have been improved upon, like for instance, you can’t click on an image to view just that one image by itself. We need to add a single page that will fetch the image location and its tags from a database. I won’t cover how I fixed this, but feel free to browse the code which I link to at the bottom of the article!

Once we have addressed these we should hopefully be in a far better place to attract big-money investors. Our finished product after we’re finished with our updates should look something like this:

Notice the tags — these were generated using Amazon Rekognition

Writing a systemd Service in Python

Many Linux distributions use systemd to manage the system’s services (or daemons), for example to automatically start certain services in the correct order when the system boots.

Writing a systemd service in Python turns out to be easy, but the complexity of systemd can be daunting at first. This tutorial is intended to get you started.

When you feel lost or need the gritty details, head over to the systemd documentation, which is pretty extensive. However, the docs are distributed over several pages, and finding what you’re looking for isn’t always easy. A good place to look up a particular systemd detail is systemd.directives, which lists all the configuration options, command line parameters, etc., and links to their documentation.

Aside from this file, this repository contains a basic implementation of a Python service consisting of a Python script ( and a systemd unit file (python_demo_service.service).

The systemd version we’re going to work with is 229, so if you’re using a different version (see systemctl --version) then check the systemd documentation for things that may differ.

HFT-like Trading Algorithm in 300 Lines of Code You Can Run Now

Commission Free API Trading Can Open Up Many Possibilities

Alpaca provides commission-free stock trading API for individual algo traders and developers, and now almost 1,000 people hang around in our community Slack talking about many different use cases. Among other things, like automated long-term value investing and Google Spreadsheet trading, high-frequency trading (“HFT”) often came up as a discussion topic among our users.

Is High-Frequency Trading (“HFT”) That Special?

Maybe because I don’t come from a finance background, I’ve wondered what’s so special about hedge funds and HFTs that those “Wallstreet” guys talk about. Since I am a developer who always looks for ways to make things work, I decided to do research and to figure out myself on how I could build similar things to what HFTs do.

I am fortunate to work with colleagues who used to build strategies and trade at HFTs, so I learned some basic know-how from them and went ahead to code a working example that trades somewhat like an HFT style (please note that my example does not act like the ultra-high speed professional trading algorithms that collocate with exchanges and fight for nanoseconds latency). Also, because this working example uses real-time data streaming, it can act as a good starting point for users who want to understand how to use real-time data streaming.

The code of this HFT-ish example algorithm is here, and you can immediately run it with your favorite stock symbol. Just clone the repository from GitHub, set the API key, and go!

AWS Lambda And Python Boto3: To Bundle Or Not Bundle With Your Function

In the engineering world a lot of our practices, even at times our best practices, are often just common wisdom passed along from one person to another. With Stack Overflow, Slack, and even Twitter, it’s easier today than it ever was for ideas to propagate. However, a lot of what passes for common wisdom is really just widely held opinions. And nothing says common wisdom has to be right. Where I ran into this distinction recently was with Python’s Boto3 modules (boto3 and botocore) and whether or not I should bundle them with my AWS Lambda deployment artifact.

Recently I found out the common wisdom I’ve adhered to was wrong. (Yes, someone on the internet was wrong.) Like many people, I use the Boto3 modules provided by the AWS Lambda runtime. However after talking with several folks at AWS I discovered, you should not be using the AWS Lambda runtime’s boto3 and botocore module. And you shouldn’t use botocore’s vendored version of the requests module whether no matter what instance of botocore you are using. I’ll explain how I found this out and explore why more than just me have probably gotten this best practice wrong.

All the Serverless announcements at re:Invent 2018

re:Invent 2018 announcements


Most Exciting: