“You probably have heard of LevelDB it’s a blazing fast key value store (as a library not a daemon), that uses Snappy compression.
There is plenty of usages for it, the API is very simple at least in Go (I will be using Goleveldb).
The key is a
byte the value is a
byte so you can “get”, “put” & “delete” that’s it.
I needed a low memory low cpu system that could collect millions of geo data and query over them, Geohash has an interesting property you can encode longitude and latitude into a string :
f2m616nn this hash represents the lat & long 46.770, -71.304 f2m616nn, if you shorten the string to
f2m61 it still refers to the same lat & long but with less precisionsf2m61.
A 4 digits hash leads to 19545 meters precision, to perfom a lookup around a position you simply query for the 8 adjacent blocks.A Geohash library for Go.
Here you would store all of your data points matching a geohash to the same set.
Problem there is no such thing as a set in LevelDB.
But there is a cursor so you can seek to a position then iterate over the next or previous one (byte ordered).
So your data could be stored that way: 4 digits geohash + a uniq id.
Then you can perform proximity lookup by searching for the 8 adjacents hashes from the position your are looking with a precision of 20km, good but not very flexible.
We can have a more generic solution, first we need a key a simple int64 uniq id…”
“I recently had to process data about places, or points of interest, around the globe. It was intuitive to me to try organize these records by their location. The standard way to group hadoop records is to make the records in the same group share the key prefix. I needed to somehow convert a latitude, longitude in a string of characters and that is when found Geohash. It is a well known dimensionality reduction technique that transforms the two dimension spatial point (latitude,longitude) into a alphanumerical string, or hash.
I’ll describe the details of the points of interest processing in a future post. In this post, I will describe Geohash visually because I believe it is easier for some people (like myself) to understand and it would had saved me a some time had anyone else done it…”
“This is a project to port Android open source project to x86 platform, formerly known as “patch hosting for android x86 support“. The original plan is to host different patches for android x86 support from open source community. A few months after we created the project, we found out that we could do much more than just hosting patches. So we decide to create our code base to provide support on different x86 platforms, and set up a git server to host it…”
It was mainly designed for fuzzing/evil testing purposes, when toxy becomes particularly useful to cover fault tolerance and resiliency capabilities of a system, especially in service-oriented architectures, where toxy may act as intermediate proxy among services.
toxy allows you to plug in poisons, optionally filtered by rules, which essentially can intercept and alter the HTTP flow as you need, performing multiple evil actions in the middle of that process, such as limiting the bandwidth, delaying TCP packets, injecting network jitter latency or replying with a custom error or status code.
Requires node.js +0.12 or io.js +1.6…”
“Perl Training Australia regularly sends out a useful tips about the Perl programming language to make your coding easier. These tips include Perl’s core features, useful modules, tricks and traps, and recent developments…”
“When testing an application in MySQL 5.6, I came across a few interesting issues. These weren’t necessarily changes in MySQL between version 5.5 and 5.6, but rather the packages I used to install MySQL 5.6…”
“This is a description of the Common Lisp ecosystem, as of August 2015, from the perspective of a user and contributor.
The purpose of this article is both to give an overview of the ecosystem, and to help drive consolidation in each domain.
Each application domain has recommendations for consolidating that part of the ecosystem, and pointers for interesting future work…”
“After you’ve ensured your web application is setup for a distributed environment, you can then decide on a strategy for load balancing. Nginx offers these strategies:
- Round Robin – Nginx switches which server to fulfill the request in order they are defined
- Least Connections – Request is assigned to the server with the least connections (and presumably the lowest load)
- Ip-Hash/Sticky Sessions – The Client’s IP address is hashed. Ther resulting hash is used to determine which server to send the request to. This also makes user sessions “sticky”. Subsequent requests from a specific user always get routed to the same server. This is one way to get around the issue of user sessions behaving as expected in a distributed environment.
- Weight – With any of the above strategies, you can assign weights to a server. Heavier-weighted servers are more likely to be selected to server a request. This is good if you want to use a partiuclarly powerful server more, or perhaps to use a server with newer or experimental specs/software installed less…”