Python-like Enumeration in C++11

“Range-based for loops in C++11 are a nice addition to the language. However, there is still no support in the STL for enumerating items in a container without creating an extra counter variable that must be incremented separately from the looping variable:

std::list<int> list = { 1, 2, 3, 4, 5, 6, 7 };

int index = 0;  // this is outside the loop block although only used inside it
for (auto x : list)
    cout << index << " " << x << endl;
    index ++;  // this has to be incremented manually; error-prone in more complex loop logic

Note that the index is not to access the items in the container; it is to keep track of how many items have been processed so far in the loop.

I would like to be able to enumerate items in an STL container as simply as it can be done in Python, i.e. via a generic “enumerate()“:

# Python code: 
list = { "a", "b", "c", "d", "e", "f" }

for index, x in enumerate(list):
    print(index, x)

In C++11, it could look like this:

std::list<int> list = { 1, 2, 3, 4, 5, 6, 7 };

for (auto x : enumerate(list))
    cout << x.index << " " << x.value << endl;

This is concise and clear. The next section provides some background. The main section shows the code that supports the above syntax and more. Finally, some possible extensions…”


Swarm v. Fleet v. Kubernetes v. Mesos

“There are clearly a lot of choices for orchestrating, clustering, and managing containers. That being said, the choices are generally well differentiated. In terms of orchestration, we can say the following:

  • Swarm has the advantage (and disadvantage) of using the standard Docker interface. Whilst this makes it very simple to use Swarm and to integrate it into existing workflows, it may also make it more difficult to support the more complex scheduling that may be defined in custom interfaces.
  • Fleet is a low-level and fairly simple orchestration layer that can be used as a base for running higher level orchestration tools, such as Kubernetes or custom systems.
  • Kubernetes is an opinionated orchestration tool that comes with service discovery and replication baked-in. It may require some re-designing of existing applications, but used correctly will result in a fault-tolerant and scalable system.
  • Mesos is a low-level, battle-hardened scheduler that supports several frameworks for container orchestration including Marathon, Kubernetes, and Swarm. At the time of writing, Kubernetes and Mesos are more developed and stable than Swarm. In terms of scale, only Mesos has been proven to support large-scale systems of hundreds or thousands of nodes. However, when looking at small clusters of, say, less than a dozen nodes, Mesos may be an overly complex solution…”

Nim for scientific computing

“In the last few months, I have been shifting the focus of my work towards scientific computing, be it for cryptographic applications, machine learning or neural networks. I have been hard-pressed to find an environment that satisfies me fully.

I do most of my daily work in Scala, and while I am still a big fan of it, trying to make it into a tool for scientific programming often hits its limits. A few things are desirable:

  • raw speed, in particular when writing inner loops
  • a predictable memory layout is necessary to be cache-friendly and to interface easily with C libraries
  • the possibility to interface easily with dedicated hardware such as GPUs and FPGAs
  • did I mention speed?…”

Getting started with OTP: creating psycho families

“OTP or Open Telecommunications Platform is what makes Erlang — “Erlang” (there are a lot of other thing too but I just wanted to sound dramatic !). It is such an integral part of Erlang that whenever people talk or write about Erlang they usually use Erlang/OTP. So you might be wondering what is this OTP thing. OTP is nothing but a framework for creating servers and process hierarchies. Just like you have web frameworks like django, ruby on rails for creating websites, Erlang has OTP framework for creating servers and process trees. This post will describe what process trees are, why is this OTP thing so cool and also describe why OTP creates psycho families…”

Introducing the Geo API in Redis

“Since a few days, the Geo API has been introduced in Redis. At the time of writing, the work is quite complete but still considered in progress: everything you’ll read here is actually available in the unstable development branch and not yet released for production (plans are to release it with the next 3.2 version).

The Geo API consists of a set of new commands that add support for storing and querying pairs of longitude/latitude coordinates into Redis keys. GeoSet is the name of the data structure holding a set of (x,y) coordinates. Actually, there isn’t any new data structure under the hood: a GeoSet is simply a Redis SortedSet…”

MariaDB 10.1 can do 1 million queries per second

The benchmark is sysbench-mariadb (sysbench trunk with a fix for a more scalable random number generator) OLTP simplified to do 1000 point selects per transaction. The data set is 1 million rows in 20 tables. Fewer tables can be used, but below 4 tables the performance drops somewhat due to a hot spot in the table definition cache.

This is the my.cnf used for this test:

max_connections = 400
table_open_cache = 800
query_cache_type = 0
innodb_buffer_pool_size = 512M
innodb_buffer_pool_instances = 10
innodb_adaptive_hash_index_partitions = 20

And this the sysbench command line:

sysbench-mariadb --test=lua/oltp.lua --oltp-tables-count=20 \
--oltp-table-size=50000 --num-threads=... --oltp-read-only=on \
--oltp-point-selects=1000 --oltp-distinct-ranges=0 \
--oltp-simple-ranges=0 --oltp-sum-ranges=0 --oltp-order-ranges=0 \
--max-time=100 --max-requests=0 run

IncludeOS – Run your C++ code directly on virtual hardware

IncludeOS aims to be the thinnest, lightest possible layer, between your C++ code and virtual hardware. We provde a bootloader, standard libraries, lots (we hope) of modules, and the build- and deployment system. You provide the service.

IncludeOS is designed for KVM/Linux but previous versions have also been tested successfully on VirtualBox (which again runs on OS X Windows and Linux) and Bochs…