Month: November 2015

A whirlwind tour of Go’s runtime environment variables

~ irrlab

“The Go runtime, in addition to providing the usual services of garbage collection, goroutine scheduling, timers, network polling and so forth, contains facilities to enable extra debugging output and even alter the behaviour of the runtime itself.

These facilities are controlled by environment variables passed to the Go program. This post describes the function of the major environment variables supported by the runtime…”

http://dave.cheney.net/2015/11/29/a-whirlwind-tour-of-gos-runtime-environment-variables

Capturing 5xx Errors with a Debug Server

~ irrlab

The twist is that we’re going to set up a special application server – we’ll call it the Debug Server – and feed it only requests that have caused errors on a regular application server. We’re taking advantage of how NGINX can detect 5xx errors coming back from an upstream application server and retry the responsible requests with a different upstream group, in our case the one containing the Debug Server. This means that Debug Server will only receive requests that have already produced errors, so its log files contain only erroneous events, ready for investigation. This reduces the scope of your needle search from a haystack to merely a handful of needles.

Unlike the main application servers, the Debug Server does not have to be built for performance. Therefore you can enable all of the available logging and diagnostic tools at your disposal, such as:

  • Running your application in debug mode, with full stack trace enabled
  • Debug logging of application servers
  • Application profiling, so that interprocess timeouts can be identified
  • Logging of server resource usage

Debugging tools like these are usually reserved for a development environment, because production environments are tuned for performance. However, as the Debug Server only ever receives erroneous requests, you can safely enable debug mode on as many components as possible.

Here’s what our application infrastructure looks like.

The Debug Server captures 5xx errors generated on other application servers

https://www.nginx.com/blog/capturing-5xx-errors-debug-server

TCP over IP Anycast – Pipe dream or Reality?

~ irrlab

With anycast, the same IP address can be assigned to n servers, potentially distributed across the world. The internet’s core routing protocol, BGP, would then automatically route packets from the source to the closest (in hops) server. For example, in the following illustration, if a user Bob wants to connect to an anycast IP 1.1.1.1, its packets would be routed to PoP A because A is only 3 hops away from Bob, while all other PoPs are 4 hops or more.

How anycast works

Anycast’s property of automatically routing to the closest PoP gives us an easy solution to our PoP assignment problem. If LinkedIn assigned the same IP to all its PoPs, then:

  • We would not need to rely on DNS-based geographical assignments (DNS would just hand out that one IP)
  • None of the problems associated with DNS-based PoP assignments would arise
  • Our users would get routed to the closest PoP automatically…

https://engineering.linkedin.com/network-performance/tcp-over-ip-anycast-pipe-dream-or-reality

Kotlin, the Swift of Android

~ irrlab

“Introducing Kotlin — JVM-based language made by JetBrains (folks behind IntelliJ IDEA and, by extension, Android Studio) and named after an island near Saint Petersburg, which is where the development office behind the project is located. Introduced in 2011, it’s been around for a few years now and the Android support came in the second milestone release (M2).

Expecting the standard reaction: yes, another JVM alternative, but this one is specifically designed to be light by excluding all the domain-specific heft and only keeping the core features missing from Java. If that sounds good, read on…”

https://blog.gouline.net/2014/08/31/kotlin-the-swift-of-android/

HTTP Live Streaming In Javascript

~ irrlab

“We can finally play HLS using only javascript. It works faster than Flash, unlocking the potential to stream live in 4k and 60fps…

HLS, which stands for HTTP Live Streaming, is a video format invented by Apple for live streaming. It’s file based, meaning you can serve it over an HTTP connection on the web.

In its simplest form, you have a manifest file, a.k.a playlist, that is the ‘main’ file for a stream. This file contains metadata about the stream and a list of URIs for the video segments of the stream. When a player requests this playlist, a server responds with an updated manifest file that specifies segments that have recently been added to the stream.
HLS Manifest - HLS Live Streaming With Peer5 This file format has many advantages over alternatives like RTP or RTMP in terms of device support, ease of deployment, and scalability. As a result, it has become a mainstream technology for live streaming, and is widely used for VoD as well…”

https://blog.peer5.com/http-live-streaming-in-javascript/

Reflection in C++14

~ irrlab

“A few months ago, I was developing an MVC web framework in C++, mostly inspired by the Ruby on Rails framework. Apart the fact that, like most of the side projects, it’s been a while I haven’t coded any line for this project, it leads me to some interesting problematics.

The problematic that is interesting here and that can introduce this article concerns the configuration of the routes. Indeed, one of the most important feature of an MVC web framework is to configure some routes that the client can request. Each route forward the request to a specific action of a controller.

For example, we can have a /articles route which is linked to the index action of the articles controller. When the client requests this path, articles_controller::index is called and returns a list of the articles.

For my MVC project, I wanted it to be easy to configure the routes and to hide this part from the user. I didn’t want the developer to declare his routes by writing C++ code: it would have been less convenient, less readable and maybe less maintainable…”

http://blog.simon-ninon.fr/reflection-in-c-plus-plus-14/