“You probably have heard of LevelDB it’s a blazing fast key value store (as a library not a daemon), that uses Snappy compression.
There is plenty of usages for it, the API is very simple at least in Go (I will be using Goleveldb).
The key is a
byte the value is a
byte so you can “get”, “put” & “delete” that’s it.
I needed a low memory low cpu system that could collect millions of geo data and query over them, Geohash has an interesting property you can encode longitude and latitude into a string :
f2m616nn this hash represents the lat & long 46.770, -71.304 f2m616nn, if you shorten the string to
f2m61 it still refers to the same lat & long but with less precisionsf2m61.
A 4 digits hash leads to 19545 meters precision, to perfom a lookup around a position you simply query for the 8 adjacent blocks.A Geohash library for Go.
Here you would store all of your data points matching a geohash to the same set.
Problem there is no such thing as a set in LevelDB.
But there is a cursor so you can seek to a position then iterate over the next or previous one (byte ordered).
So your data could be stored that way: 4 digits geohash + a uniq id.
Then you can perform proximity lookup by searching for the 8 adjacents hashes from the position your are looking with a precision of 20km, good but not very flexible.
We can have a more generic solution, first we need a key a simple int64 uniq id…”
“I recently had to process data about places, or points of interest, around the globe. It was intuitive to me to try organize these records by their location. The standard way to group hadoop records is to make the records in the same group share the key prefix. I needed to somehow convert a latitude, longitude in a string of characters and that is when found Geohash. It is a well known dimensionality reduction technique that transforms the two dimension spatial point (latitude,longitude) into a alphanumerical string, or hash.
I’ll describe the details of the points of interest processing in a future post. In this post, I will describe Geohash visually because I believe it is easier for some people (like myself) to understand and it would had saved me a some time had anyone else done it…”
“This is a project to port Android open source project to x86 platform, formerly known as “patch hosting for android x86 support“. The original plan is to host different patches for android x86 support from open source community. A few months after we created the project, we found out that we could do much more than just hosting patches. So we decide to create our code base to provide support on different x86 platforms, and set up a git server to host it…”
It was mainly designed for fuzzing/evil testing purposes, when toxy becomes particularly useful to cover fault tolerance and resiliency capabilities of a system, especially in service-oriented architectures, where toxy may act as intermediate proxy among services.
toxy allows you to plug in poisons, optionally filtered by rules, which essentially can intercept and alter the HTTP flow as you need, performing multiple evil actions in the middle of that process, such as limiting the bandwidth, delaying TCP packets, injecting network jitter latency or replying with a custom error or status code.
Requires node.js +0.12 or io.js +1.6…”
“Perl Training Australia regularly sends out a useful tips about the Perl programming language to make your coding easier. These tips include Perl’s core features, useful modules, tricks and traps, and recent developments…”