libdill: Structured Concurrency for C

Concurrency means that multiple functions can run independently of each another. It can mean that they are running in parallel on multiple CPU cores. It can also mean that they are running on a single CPU core and the system is transparently switching between them.

http://libdill.org/structured-concurrency.html

liblfds, a portable, license-free, lock-free data structure library written in C

Lock-free data structures are process, thread and interrupt safe (i.e. the same data structure instance can be safely used across processes, threads and both inside and outside of interrupt handlers), never sleep (and so are safe for kernel use when sleeping is not permitted), operate without context switches, cannot fail (no need to handle error cases, as there are none), perform and scale literally orders of magnitude better than locking data structures, and liblfds itself (as of release 7.0.0) is implemented such that it performs no allocations and compiles not just on a freestanding C89 implementation, but on a bare C89 implementation.

http://liblfds.org/

You Can’t Always Hash Pointers in C

Occasionally I’ve needed to key a hash table with C pointers. I don’t care about the contents of the object itself — especially if it might change — just its pointer identity. For example, suppose I’m using null-terminated strings as keys and I know these strings will always be interned in a common table. These strings can be compared directly by their pointer values (str_a == str_b) rather than, more slowly, by their contents (strcmp(str_a, str_b) == 0). The intern table ensures that these expressions both have the same result.

As a key in a hash table, or other efficient map/dictionary data structure, I’ll need to turn pointers into numerical values. However, C pointers aren’t integers. Following certain rules it’s permitted to cast pointers to integers and back, but doing so will reduce the program’s portability. The most important consideration is that the integer form isn’t guaranteed to have any meaningful or stable value. In other words, even in a conforming implementation, the same pointer might cast to two different integer values. This would break any algorithm that isn’t keenly aware of the implementation details.

http://nullprogram.com/blog/2016/05/30/

TrailDB – An Efficient Library for Storing and Processing Event Data

tl;dr Today, we are open-sourcing TrailDB, a core library powering AdRoll. TrailDB makes it fast and fun to handle event data. Find it at traildb.io.

http://tech.adroll.com/blog/data/2016/05/24/traildb-open-sourced.html

TrailDB was created at AdRoll to power processing of time-series of events. You can use it, for instance, to

  • Compute metrics, such as the bounce rate
  • Analyze usage patterns
  • Detect anomalies
  • Cluster and predict user behavior

Since 2014, AdRoll has used TrailDB to store and query over 20 trillion events originating from the web.

http://traildb.io/

Balde – A microframework for C based on GLib and bad intentions

It is designed to be fast, simple, and memory efficient. Most of its architecture is based on other microframeworks, like Flask, and it can run on any web server that supports SCGI.

With balde you can serve hundreds of requests per second with a minimal memory footprint and all the performance features provided by SCGI.

FEATURES

  • Simple templating engine, that converts markup to C code that is linked directly to the application binary.
  • Static resources are embedded and served from the application binary.
  • RESTful request dispatching. Supports most HTTP methods.
  • Reverse resolution of URLs (similar to Flask’s “url_for” function).
  • Secure cookies (client side sessions).
  • File uploads.
  • Compatible with SCGI and CGI servers.
  • Runs on any GLib-capable system.

https://balde.rgm.io/

Embedding PyPy in a C application

At the PyGrunn 2016 conference last Friday I gave a talk about the Python cffi package and how to leverage it to embed PyPy into a C application. A video of this talk will be published later when it becomes available.

With the release of cffi 1.5.0 and its subsequent inclusion in PyPy 5 it becomes possible to embed PyPy. This is done by compiling the Python code into a dynamic library which can then be used in any other language. In this article I will show you how to do this.

http://codelle.com/blog/2016/5/embedding-pypy-in-a-c-application/

“Lispsy” Lisp(ish) to C Converter (designed for CLISP)

LISP/c is a powerful macrolanguage for C. It basically turns this:

(header stdio)
(main
(@printf (str “Hello, world!”))
(return 0))
into (after it being cleaned up (more on this later)) this:

#include

int main(int argc,char **argv)
{
printf(“Hello, world!”);
return 0;
}

But why?

First, you might check out this video. Because LISP is expressive and C is fast and I wanted the best of both worlds is the short answer. The longer answer has something to do with macros. But instead of immediately boring you with that, I’ll answer what you really want to know:

Why Should I Care?

First let’s discuss if you can use it. Not to be elitist (I wish everyone would use this tool), but you must know both C and LISP fairly well to be able to use LISP/c.

Suppose, however, that you do already know both LISP and C pretty well. You might want to use LISP/c because it features access to LISP to write C code both implicitly and explicity. You might also want to use it if you like writing CUDA code, because it has built-in support for CUDA as well.

But really, to see why you might like to use LISP/c, check out a few examples, and feel free to skip around a little.

https://github.com/eratosthenesia/lispc

nginx module for Brotli compression

Brotli is a generic-purpose lossless compression algorithm that compresses data using a combination of a modern variant of the LZ77 algorithm, Huffman coding and 2nd order context modeling, with a compression ratio comparable to the best currently available general-purpose compression methods. It is similar in speed with deflate but offers more dense compression.

ngx_brotli is a set of two nginx modules:

  • ngx_brotli filter module – used to compress responses on-the-fly,
  • ngx_brotli static module – used to serve pre-compressed files.

https://github.com/google/ngx_brotli#brotli_comp_level

Debugging of CPython processes with gdb

pdb has been, is and probably always will be the bread and butter of Python programmers, when they need to find the root cause of a problem in their applications, as it’s a built-in and easy to use debugger. But there are cases, when pdb can’t help you, e.g. if your app has got stuck somewhere, and you need to attach to a running process to find out why, without restarting it. This is where gdb shines.

http://podoliaka.org/2016/04/10/debugging-cpython-gdb/