Node.js Express API Development Security Checklist

The folks at RisingStack have published a really good article on security in Node.js applications and this checklist is meant to complement it with specifics for API development using the express framework.

  • [ ] Secure headers: use helmet, especially to set the Strict Transport Security header which will keep all your connections on HTTPS. Also see here on how to setup https using a free certificate from letsencrypt.
  • [ ] Log all errors but don’t expose stacktraces to the client.
  • [ ] Rate limit api calls to protect against DoS attacks. Can use expres-rate-limit.
  • Sanitize all user input
    • [ ] Sql injection: use prepared statements in favor of concatenating user input. For e.g.
      app.get('/', function(req, res) {
        Promise.using(getSqlConnection(), function(connection) {
          var sql = 'SELECT * from users where id = "' + req.query.username + '"';
          return connection.queryAsync(sql, [id])
            .then(function(rows, cols) {
              return rows;
            });
        });
      });

      can be hijacked to /?username=anything%22%20OR%20%22x%22%3D%22x which results in the following sql query being executed: select * from users where id = "anything" OR "x"="x". This will always result in true and return data for all the users in the system. This can be further extended to cause a lot more damage.

    • [ ] XSS: prevent the ability of an attacker to inject arbitary code into your application by sanitizing user input. For e.g. the following end point which accepts user input
      app.get('/', function(req, res) {
        var html = 'Hello ' + req.query.username;
        res.send(html);
      });

      can then be hijacked to create a url as follows /?username=%3Cbody%20onload%3Dalert(%27test1%27)%3E. This link can then be sent to unsuspecting users of your website and have arbitary code being executed on their machine. See here for more types of XSS attacks and examples.

    • [ ] Command injection: for example, a url like https://example.com/downloads?file=user1.txt could be turned into https://example.com/downloads?file=%3Bcat%20/etc/passwd.
    • [ ] MongoDb query injection: similar to sql injection but using MongoDb’s special operators instead. As an example consider the following end point
      app.post('/', function (req, res) {
        db.users.find({username: req.body.username, password: req.body.password}, function (err, users) {
            // TODO: handle the rest
        });
      });

      where sending in

      POST http://target/ HTTP/1.1
      Content-Type: application/json
      
      {
          "username": "vic@smalldata.tech",
          "password": {"$gt": ""}
      }
      

      will result in a successful match. Use mongo-express-sanitize to sanitize all user input.

    • [ ] Regex Denial of Service: a situation where user inputted regex can lead to blocking the event loop and a hanging application. See here for examples.
  • [ ] Use TLS for all connections. Also see here on how to setup https using a free certificate from letsencrypt.
  • [ ] Keep dependencies updated to stay ahead of any security issues. Use nsp to check dependencies for security vulnerabilities. Another great platform for open source projects is snyk.io.
  • [ ] Check for permissions at every step of the API chain: for e.g. GET /users/:userId/contacts/:contactId should not assume that the userId authenticated for the request is also authorized to make this call. Check that request.params.userId === request.authenticatedUserId or isAuthorized(authenticatedUserId, {userId: authenticatedUserId, resource: 'CONTACTS'}.
  • [ ] Don’t block the event loop: as an example parsing json is not a free operation and can potentially block the event loop for large json files (> 1Mb). Note that using the bodyparser module globally will give you a default maximum of 100kb for json payloads. It is efficient to only use it for routes which require it.

Please note that this checklist is meant to be used as a reference for further study. It is by no means an exhaustive list of all potential security issues. See also the web developer security checklist. Additions and comments are welcome.

https://smalldata.tech/blog/2017/05/19/nodejs-express-api-development-security-checklist

A Guide to Crypto Currencies

This guide is meant to serve as both an easy-to-understand introduction to the world of cryptocurrencies as well as an insightful view into the different projects competing for your investments and market dominance and a look at the underlying technology, history and trends.

For many years Bitcoin would occasionally appear in the media after it spiked in price. I didn’t think there was anything inherently useful about it. I thought it was a novelty, a ponzi scheme, hysteria. It was only after the most recent price spike in another cryptocurrency, Ethereum, that the crazy returns finally tempted me. What started out as a skeptical look into a get-rich-quick scheme led me down a rabbit hole and my mind was promptly blown at the potential of the technology. The hype surrounding it is nothing short of mania, but it’s not without merit. Cryptocurrencies will almost certainly revolutionize everything from insurance, logistics and the stock market to ownership and even create entire economies which don’t currently exist. You may feel skeptical when hearing something so optimistic but when banks, governments and research institutions start to take notice and want to work with these projects maybe it’s time we paid some attention.

Many of you reading may be likening the current craze with the dotcom bubble and I’m afraid I absolutely agree with you. The speculation surrounding cryptocurrencies and the ease of which the average person can invest has created an environment where an idea can raise hundreds of millions of dollars without even a proof of concept. This is part of the reason this guide was written, to steer you clear of these massively overvalued “pet.com” equivalents and towards the future Amazons and Googles.

https://mycrypto.guide/

Fast Properties in V8

In this blog post we would like to explain how V8 handles JavaScript properties internally. From a JavaScript point of view there are only a few distinctions necessary for properties. JavaScript objects mostly behave like dictionaries, with string keys and arbitrary objects as values. The specification does however treat integer-indexed properties and other properties differently during iteration. Other than that, the different properties behave mostly the same, independent of whether they are integer indexed or not.

However, under the hood V8 does rely on several different representations of properties for performance and memory reasons. In this blog post we are going to explain how V8 can provide fast property access while handling dynamically-added properties. Understanding how properties work is essential for explaining how optimizations such as inline caches work in V8.

This post explains the difference in handling integer-indexed and named properties. After that we show how V8 maintains HiddenClasses when adding named properties in order to provide a fast way to identify the shape of an object. We’ll then continue giving insights into how named properties are optimized for fast accesses or fast modification depending on the usage. In the final section we provide details on how V8 handles integer-indexed properties or array indices.

http://v8project.blogspot.com.br/2017/08/fast-properties.html

Start Flutter

Flutter is an open-source mobile app development SDK by Google and we’ve recently started experimenting with it.

Now why did it interest us?

  1. Consistency of UI elements.
  2. Lot of predefined widgets which serve as building blocks for making complex pages easily.
  3. Performance is faster because of reduced interaction between the Native Layer and Runtime Environment. Instead, Flutter uses a compiled programming language, namely Dart. Dart allows Flutter to communicate with the platform without going through a JavaScript bridge.
  4. Flutter does not use the OEM widgets or DOM WebViews, but provides its own widgets that look and feel good, are fast, and are customizable and extensible.
  5. Flutter ignores the traditional model of layout. In Flutter, each widget specifies its own simple layout model. Since each widget has a much smaller set of layout rules to consider, layout can be heavily optimised. What’s more, almost everything in Flutter is considered as a widget.
  6. Less chances of ending up writing a buggy code as we have a lesser need of implementing external libraries.

So, while we’re working on our first open source Flutter theme, we’ve taken the initiative of Start Flutter, a library of free and open source Flutter templates.

For now, we’ve curated and added three open source themes, i.e….

 
https://blog.geekyants.com/announcing-start-flutter-c81d773a96ab

Integrate SQS and Lambda: serverless architecture for asynchronous workloads

The Gold Standard for modern cloud-native applications is a serverless architecture. AWS Lambda allows you to implement scalable and fault tolerant applications without the need of a single virtual machine.

A serverless infrastructure based on AWS Lambda has two key benefits:

  1. You don’t need to manage a fleet of virtual machines anymore.
  2. Deploying new versions of your code can be entirely controlled by API calls.

This article shows you how to process asynchronous tasks serverless. Possible use cases are: sending out massive amounts of emails, transcoding video files after upload, or analyzing user behavior. An SQS queue will be used to decouple your microservice from other parts of your system. You’ll learn how to implement the microservice with AWS Lambda.

https://cloudonaut.io/integrate-sqs-and-lambda-serverless-architecture-for-asynchronous-workloads/

All the fundamental React.js concepts, jammed into this single Medium article

This article is not going to cover what React is or why you should learn it. Instead, this is a practical introduction to the fundamentals of React.js for those who are already familiar with JavaScript and know the basics of the DOM API.

All code examples below are labeled for reference. They are purely intended to provide examples of concepts. Most of them can be written in a much better way.

https://medium.freecodecamp.org/all-the-fundamental-react-js-concepts-jammed-into-this-single-medium-article-c83f9b53eac2

Simple task runner / Make alternative written in Go

Task is a simple tool that allows you to easily run development and build tasks. Task is written in Golang, but can be used to develop any language. It aims to be simpler and easier to use then GNU Make.

https://github.com/go-task/task

Building Business Systems with Domain-Specific Languages for NGINX & OpenResty

This post is adapted from a presentation at nginx.conf 2016 by Yichun Zhang, Founder and CEO of OpenResty, Inc. This is the first of two parts of the adaptation. In this part, Yichun describes OpenResty’s capabilities and goes over web application use cases built atop OpenResty. In Part 2, Yichun looks at what a domain-specific language is in more detail.

You can view the complete presentation on YouTube.

https://www.nginx.com/blog/building-business-systems-with-domain-specific-languages-for-nginx-openresty-part-1/
https://www.nginx.com/blog/building-business-systems-with-domain-specific-languages-for-nginx-openresty-part-2/

ANALYZING CRYPTOCURRENCY MARKETS USING PYTHON

A DATA-DRIVEN APPROACH TO CRYPTOCURRENCY SPECULATION

How do Bitcoin markets behave? What are the causes of the sudden spikes and dips in cryptocurrency values? Are the markets for different altcoins inseparably linked or largely independent? How can we predict what will happen next?

Articles on cryptocurrencies, such as Bitcoin and Ethereum, are rife with speculation these days, with hundreds of self-proclaimed experts advocating for the trends that they expect to emerge. What is lacking from many of these analyses is a strong foundation of data and statistics to backup the claims.

The goal of this article is to provide an easy introduction to cryptocurrency analysis using Python. We will walk through a simple Python script to retrieve, analyze, and visualize data on different cryptocurrencies. In the process, we will uncover an interesting trend in how these volatile markets behave, and how they are evolving.

Combined Altcoin Prices

This is not a post explaining what cryptocurrencies are (if you want one, I would recommend this great overview), nor is it an opinion piece on which specific currencies will rise and which will fall. Instead, all that we are concerned about in this tutorial is procuring the raw data and uncovering the stories hidden in the numbers.

 https://blog.patricktriest.com/analyzing-cryptocurrencies-python/

How I do Developer UX at Google

When people talk about User Experience (UX), they often talk about their beloved consumer products: a smartphone, a messaging app, or perhaps a pair of headphones.

User Experience also matters when you build something for developers. People tend to forget that developers are users too, and software development is an intrinsically human activity limited by not only how computers work, but also how programmers work. Admittedly, there are fewer developers than consumers in general, but the more usable developer tools are, the more energy developers can spend on delivering value to their users. Therefore, the UX of developer products is just as important as for consumer products. In this post, I am going to introduce the developer experience, explain one of the ways we assess it at Google, and share some lessons we learned from a specific study we conducted on Flutter, a new SDK for building beautiful mobile apps.

The idea of developer experience is not exactly new. Research on developer experience dates back to the early days of computing, since all users at the time were developers to some degree. “The Psychology of Computer Programming”, published in 1971, is a landmark book on the topic. When we talk about developer experience, especially applying the term to an SDK or library, we usually refer to three aspects of the product:

  • API Design, which includes the naming of classes, methods and variables, the abstraction level of the API, the organization of the API, and the way the API is invoked.
  • Documentation, which includes both the API reference and other learning resources such as tutorials, how-tos, and developer guides.
  • Tooling, which involves both the command-line interface (CLI) and GUI tools that help editing, debugging, and testing the code. For example, research has shown that autocomplete in the IDE has a large impact on how APIs are discovered and used in programming.

These three pillars of developer experience complement one another, so they need to be designed and assessed as a package.

 
https://medium.com/google-design/how-i-do-developer-ux-at-google-b21646c2c4df