Lock-free programming for the masses

Efficient concurrent programming libraries are essential for taking advantage of fine-grained parallelism on multicore hardware. In this post, I will introduce reagents, a composable, lock-free concurrency library for expressing fine-grained parallel programs on Multicore OCaml. Reagents offer a high-level DSL for experts to specify efficient concurrency libraries, but also allows the consumers of the libraries to extend them further without knowing the details of the underlying implementation.


Designing and implementing scalable concurrency libraries is an enormous undertaking. Decades of research and industrial effort has led to state-of-the-art concurrency libraries such as java.util.concurrent (JUC) for the JVM andSystem.Collections.Concurrent (SCC) for the .NET framework. These libraries are often written by experts and have subtle invariants, which makes them hard to maintain and improve. Moreover, it is hard for the library user to safely combine multiple atomic operations. For example, while JUC and SCC provide atomic operations on stacks and queues, such atomic operations cannot be combined into larger atomic operations.

On the other hand software transactional memory (STM) offers composability, but STM based data structures are generally less efficient than their lock-free counterparts, especially when there is moderate to high levels of contention. Aaron Turonintroduced reagents, an expressive and composable library which retains the performance and scalability of lock-free programming. Reagents allow isolated atomic updates to shared state, as well as message passing communication over channels. Furthermore, reagents provide a set of combinators for sequential composition à la STM, parallel composition à laJoin calculus, and selective communication à la Concurrent ML, while being lock-free. Reagents occupy this sweet-spot between expressivity and performance, and we believe it could serve as a great default1 for writing fine-grained concurrent programs in Multicore OCaml.


Functional Programming in the Real World

“Here is a list of functional programs applied to real-world tasks. The main criterion for being real-world is that the program was written primarily to perform some task, not primarily to experiment with functional programming. Functional is used in the broad sense that includes both `pure’ programs (no side effects) and `impure’ (some use of side effects). Languages covered include CAML, Clean, Erlang, Haskell, Miranda, Scheme, SML, and others…”


OCaml Briefly

“This document gives you a brief tour of OCaml. It covers a rather small selection of features; the selection has been based on what features I personally think represent OCaml the best.

This document does very little to explain use-cases for the selected features but rather focuses on syntax. For a more in-depth coverage of all of these features I recommend reading the OCaml Document and User’s Manual and Real World OCaml.

For each feature there is a small explanation of the syntax, some examples, and links for further reading. As such this document is ideal for someone who wants to get a taste of the features of OCaml or who want to learn more about a specific feature…”


Why We Use OCaml

“Here at Esper, we have chosen to use the OCaml language for the server-side backend that implements most of our application’s functionality. OCaml includes many features that are not available in the more mainstream programming languages listed above, and we believe this gives us a competitive advantage. But even among Silicon Valley startups, OCaml is an unusual choice. The purpose of this post is to explain the benefits of OCaml and compare it to other languages. We hope to convince readers, especially other developers, to consider adopting OCaml for their projects as well…”

Why We Use OCaml

Lua API OCaml library

“Lua is a powerful, light-weight programming language designed for extending applications. It provides a good general purpose programming language to replace DSL that don’t really need to be specific.

This library provides bindings to Lua API which allows the application to exchange data with Lua programs and also to extend Lua with OCaml functions.

This is the OCaml complete binding of the Lua Application Program Interface as described in the official documentation.

In this moment only the version 5.1.x is supported, while 5.2.x is on my TODO list. In general my plan is to support newest versions, but not the oldest ones…”


Thoughts on ocaml

“I don’t write enough to have genuine blog, I’ve decided to write about the experience of using ocaml here. Firstly, I’d like to acknowledge that I’m still in the honeymoon period with OCaml; so keep in mind this isn’t a particularly objective critique. That being said, I will try to talk about concrete issues and items, instead of discussing the usual “functional programming will expand your mind”, “it’ll make you a better programmer even if you don’t use it”, or other arbitrary issues such as code elegance. This also isn’t intended to be an exhaustive list; I’m just trying get you interested enough to start googling around. If I can get at least one child to use OCaml, then it was worth it. 😉

For some reason, I had never gotten around to anything in the ML family even though I’ve programmed in a variety of languages. Conventional wisdom says that you can either have an expressive programming language (like python or ruby) or a fast programming language (like C, C++, or assembly). But these guys and gals in the ML world have been saying that you can have the best of both worlds. You can have an high-level language that performs like a systems-level language. It sounded too good to be true. So I decided I was going to learn Objective Caml, hoping that it would have 75% the expressiveness of of python, and 75% the performance of C. In the end, I’d say it has about 50-60% the expressiveness of python, and 90% the performance of C. Keep in mind that you can do some really honestly truely wacky stuff in python, so 60% is still really good…”