Month: November 2015

A whirlwind tour of Go’s runtime environment variables

~ irrlab

“The Go runtime, in addition to providing the usual services of garbage collection, goroutine scheduling, timers, network polling and so forth, contains facilities to enable extra debugging output and even alter the behaviour of the runtime itself.

These facilities are controlled by environment variables passed to the Go program. This post describes the function of the major environment variables supported by the runtime…”

http://dave.cheney.net/2015/11/29/a-whirlwind-tour-of-gos-runtime-environment-variables

Lua Workshop in Stockholm 2015

~ irrlab

“As in previous workshops (2005, 2006, 2008, 2009, 2011, 2012, 2013, 2014), the main goal of the workshop is to allow the Lua community to get together and meet in person and talk about the Lua language, its uses, and its implementation.

The workshop is open to everyone interested in Lua…”

http://www.lua.org/wshop15.html

https://www.youtube.com/channel/UCW5AXY0vcX2KZY1pB79C3vQ/playlists

 

Capturing 5xx Errors with a Debug Server

~ irrlab

The twist is that we’re going to set up a special application server – we’ll call it the Debug Server – and feed it only requests that have caused errors on a regular application server. We’re taking advantage of how NGINX can detect 5xx errors coming back from an upstream application server and retry the responsible requests with a different upstream group, in our case the one containing the Debug Server. This means that Debug Server will only receive requests that have already produced errors, so its log files contain only erroneous events, ready for investigation. This reduces the scope of your needle search from a haystack to merely a handful of needles.

Unlike the main application servers, the Debug Server does not have to be built for performance. Therefore you can enable all of the available logging and diagnostic tools at your disposal, such as:

  • Running your application in debug mode, with full stack trace enabled
  • Debug logging of application servers
  • Application profiling, so that interprocess timeouts can be identified
  • Logging of server resource usage

Debugging tools like these are usually reserved for a development environment, because production environments are tuned for performance. However, as the Debug Server only ever receives erroneous requests, you can safely enable debug mode on as many components as possible.

Here’s what our application infrastructure looks like.

The Debug Server captures 5xx errors generated on other application servers

https://www.nginx.com/blog/capturing-5xx-errors-debug-server

TCP over IP Anycast – Pipe dream or Reality?

~ irrlab

With anycast, the same IP address can be assigned to n servers, potentially distributed across the world. The internet’s core routing protocol, BGP, would then automatically route packets from the source to the closest (in hops) server. For example, in the following illustration, if a user Bob wants to connect to an anycast IP 1.1.1.1, its packets would be routed to PoP A because A is only 3 hops away from Bob, while all other PoPs are 4 hops or more.

How anycast works

Anycast’s property of automatically routing to the closest PoP gives us an easy solution to our PoP assignment problem. If LinkedIn assigned the same IP to all its PoPs, then:

  • We would not need to rely on DNS-based geographical assignments (DNS would just hand out that one IP)
  • None of the problems associated with DNS-based PoP assignments would arise
  • Our users would get routed to the closest PoP automatically…

https://engineering.linkedin.com/network-performance/tcp-over-ip-anycast-pipe-dream-or-reality

Kotlin, the Swift of Android

~ irrlab

“Introducing Kotlin — JVM-based language made by JetBrains (folks behind IntelliJ IDEA and, by extension, Android Studio) and named after an island near Saint Petersburg, which is where the development office behind the project is located. Introduced in 2011, it’s been around for a few years now and the Android support came in the second milestone release (M2).

Expecting the standard reaction: yes, another JVM alternative, but this one is specifically designed to be light by excluding all the domain-specific heft and only keeping the core features missing from Java. If that sounds good, read on…”

https://blog.gouline.net/2014/08/31/kotlin-the-swift-of-android/

HTTP Live Streaming In Javascript

~ irrlab

“We can finally play HLS using only javascript. It works faster than Flash, unlocking the potential to stream live in 4k and 60fps…

HLS, which stands for HTTP Live Streaming, is a video format invented by Apple for live streaming. It’s file based, meaning you can serve it over an HTTP connection on the web.

In its simplest form, you have a manifest file, a.k.a playlist, that is the ‘main’ file for a stream. This file contains metadata about the stream and a list of URIs for the video segments of the stream. When a player requests this playlist, a server responds with an updated manifest file that specifies segments that have recently been added to the stream.
HLS Manifest - HLS Live Streaming With Peer5 This file format has many advantages over alternatives like RTP or RTMP in terms of device support, ease of deployment, and scalability. As a result, it has become a mainstream technology for live streaming, and is widely used for VoD as well…”

https://blog.peer5.com/http-live-streaming-in-javascript/

Reflection in C++14

~ irrlab

“A few months ago, I was developing an MVC web framework in C++, mostly inspired by the Ruby on Rails framework. Apart the fact that, like most of the side projects, it’s been a while I haven’t coded any line for this project, it leads me to some interesting problematics.

The problematic that is interesting here and that can introduce this article concerns the configuration of the routes. Indeed, one of the most important feature of an MVC web framework is to configure some routes that the client can request. Each route forward the request to a specific action of a controller.

For example, we can have a /articles route which is linked to the index action of the articles controller. When the client requests this path, articles_controller::index is called and returns a list of the articles.

For my MVC project, I wanted it to be easy to configure the routes and to hide this part from the user. I didn’t want the developer to declare his routes by writing C++ code: it would have been less convenient, less readable and maybe less maintainable…”

http://blog.simon-ninon.fr/reflection-in-c-plus-plus-14/

Recent improvements to Redis Lua scripting

~ irrlab

“Lua scripting is probably the most successful Redis feature, among the ones introduced when Redis was already pretty popular: no surprise that a few of the things users really want are about scripting. The following two features were suggested multiple times over the last two years, and many people tried to focus my attention into one or the other during the Redis developers meeting, a few weeks ago.

1. A proper debugger for Redis Lua scripts.
2. Replication, and storage on the AOF, of Lua scripts as a set of write commands materializing the *effects* of the script, instead of replicating the script itself as we normally do.

The second feature is not just a matter of how scripts are replicated, but also touches what you can do with Lua scripting as we will see later. Back from London, I implemented both the features. This blog post describes both, giving a few hints about the design and implementation aspects that may be interesting for the readers…”

http://antirez.com/news/97

Practical Natural Language Processing for Determing Wifi Quality in Hostels

~ irrlab

“I was planning my trip to Amsterdam in January and was looking through hostels in Hostel World filtering for different features and amenities. One amenity that I thought I would definitely need was free wifi if I wanted to do some programming from the hostel and also just because life demands it in general. While there’s a ton of hostels that offer free wifi, I’ve definitely been at the end of the stick where the quality of wifi has been unmentionably bad. This probably goes for hotels as well as hostels, but generally hostels are cheaper and offer less in the way of complementary services.

That got me thinking about creating an interesting application that could judge the quality of wifi in reviews. Randomly I decided to spin up a new idea for a scraping/api for Hostel World where I could actually find the reviews that mention wifi and other amenities that would be useful. Instead of meticulously scanning through hundreds of reviews, I could just scrape the reviews, parse out keywords, and assign sentiment scores to each review…”

http://www.racketracer.com/2015/11/18/practical-natural-language-processing-for-determing-wifi-quality-in-hostels/