Challenging Clojure in Common Lisp

Clojure is a dialect of the Lisp programming language created by Rich Hickey [Clojure]. According to Wikipedia, Mr. Hickey created Clojure “because he wanted a modern Lisp for functional programming, symbiotic with the established Java platform, and designed for concurrency.[Clojure]. Since then, Clojure has gathered somewhat of a following in the functional programming community, the other chief contender for functional programming on the JVM being the Scala programming language. This blog will attempt to document my quest to achieve two of Mr. Hickey’s goals using straight-up Common Lisp: idiomatic functional programming & concurrency. Why am I doing this? I tried Clojure and was disheartened by two aspects of the language: lackluster performance and the constant resurfacing of the Java and the JVM in error messages and other aspects of working with Clojure. I know of ClojureScript and the ability to run it on Google V8, but I suppose I really wanted a non-hosted language where I can intuitively map abstractions to assembler code by calling something like (disassemble … ) on a simple function. I tried Scala and missed homoiconic macros. Somehow I could not shake the feeling that homoiconic macros were intimately tied to the very essence of functional programming. I was also puzzled by the suggestion that Lisp, in its current form, somehow lacked modernity. Popularity, yes, modernity I wasn’t sure of… We will add one twist to the story. With Lisp being a programmable programming language, I can surely re-engineer any design goals of Clojure using Common Lisp by way of macros — but the challenge here is to do without such an effort. Therefore there are two rules in this game: 1) I am not allowed to write the keyword “defmacro” 2) I must rely on stock libraries / frameworks. “Stock” will be defined as installable via Common Lisp’s package manager Quicklisp [QuickLisp]. The “long and short of it is” that I want the utility of Clojure with the performance of Common Lisp. As Dough Hoyte points out in the book Let Over Lambda [Let-Over-Lambda], Lisp is not precisely a functional programming language, not in the the sense that Haskell and ML are functional programming language. I aim to challenge that. In [Lisp-Converge-C++] we showed how to bring Lisp close to C in performance by matching the generated Assembler code.  In this blog we aim to “bring Lisp in line” with idiomatic functional programming concepts. With our two rules in mind, let us see how far we can travel on this road…

The Idea of Lisp

LISP. It conjures up visions of a bygone age of computers the size of refrigerators, ALL CAPS CODE, and parentheses. Oh! so many parentheses! So why is Object-Oriented Programming’s creator so enamored with the idea of Lisp? And what can he mean by a programming language being an idea anyway? Should I blame my Computer Science education for not teaching it to me?

hitchhiker-tree: Functional, persistent, off-heap, high performance data structure

Hitchhiker trees are a newly invented (by @dgrnbrg) datastructure, synthesizing fractal trees and functional data structures, to create fast, snapshottable, massively scalable databases.

What’s in this Repository?

The hitchhiker namespaces contain a complete implementation of a persistent, serializable, lazily-loaded hitchhiker tree. This is a sorted key-value datastructure, like a scalable sorted-map. It can incrementally persist and automatically lazily load itself from any backing store which implements a simple protocol.

Outboard is a sample application for the hitchhiker tree. It includes an implementation of the IO subsystem backed by Redis, and it manages all of the incremental serialization and flushing.

The hitchhiker tree is designed very similarly to how Datomic’s backing trees must work–I would love to see integration with DataScript for a fully open source Datomic.