thoughts…

rants and bookmarks about programming stuff…


Call me maybe: Redis & Reply to Aphyr attack to Sentinel

Redis is a fantastic data structure server, typically deployed as a shared heap. It provides fast access to strings, lists, sets, maps, and other structures with a simple text protocol. Since it runs on a single server, and that server is single-threaded, it offers linearizable consistency by default: all operations happen in a single, well-defined order. There’s also support for basic transactions, which are atomic and isolated from one another.

Because of this easy-to-understand consistency model, many users treat Redis as a message queue, lock service, session store, or even their primary database. Redis running on a single server is a CP system, so it is consistent for these purposes…”

http://aphyr.com/posts/283-call-me-maybe-redis

http://antirez.com/news/55


Redis as the primary data store? WTF?!

“Redis is a key-value in memory data store typically used for caches and other such mechanisms to speed up web applications. We however store all our data in Redis as our primary database.

The web is abound with warnings and cautionary tales about going this route. There are horror stories about lost data, hitting memory limits, or people unable to effectively manage the data within Redis, so you might be wondering “What on earth were you thinking?!” So here is our story, why we decided to use Redis anyway, and how we overcame those issues.

First of all, I want to stress that most applications shouldn’t even worry about the engineering hurdles involved with going this route. It was important for our use case, but we may very well be an edge case…”

http://moot.it/blog/technology/redis-as-primary-datastore-wtf.html


bitmapist: Powerful realtime analytics with Redis 2.6′s bitmaps and Python

bitmapist (GitHub) – a powerful realtime analytics library that can help you answer following questions:

  • Has user 123 been online today? This week? This month?
  • Has user 123 performed action “X”?
  • How many users have been active have this month? This hour?
  • How many unique users have performed action “X” this week?
  • How many % of users that were active last week are still active?
  • How many % of users that were active last month are still active this month?

This library is very easy to use and enables you to create your own reports easily…”

http://amix.dk/blog/post/19714


Solving monitoring state storage problems using Redis

Redis is an in-memory key-value data store that provides a small number of primitives suitable to the task of building monitoring systems. As a lot of us are hacking in this space I thought I’d write a blog post summarizing where I’ve been using it in a little Sensu like monitoring system I have been working on on and off.

There’s some monitoring related events coming up like MonitoringLove in Antwerp and Monitorama in Boston – I will be attending both and I hope a few members in the community will create similar background posts on various interesting areas before these events…”

http://www.devco.net/archives/2013/01/06/solving-monitoring-state-storage-problems-using-redis.php


A small rant about software reliability…

“I’m very serious about software reliability, and this is not just a good thing. It is good in a sense, as I tend to work to ensure that the software I release is solid. At the same time I think I take this issue a bit too personally: I get upset if I receive a crash report that I can’t investigate further for some reason, or that looks like almost impossible to me, or with an unhelpful stack trace. Guess what? This is a bad attitude because to deliver bugs free software is simply impossible. We are used to think in terms of labels: “stable”, “production ready”, “beta quality”. I think that these labels are actually pretty misleading if not put in the right perspective…”

http://antirez.com/news/43


Implements a powerful analytics library using Redis bitmaps

“This Python library makes it possible to implement real-time, highly scalable analytics that can answer following questions:

  • Has user 123 been online today? This week? This month?
  • Has user 123 performed action “X”?
  • How many users have been active have this month? This hour?
  • How many unique users have performed action “X” this week?
  • How many % of users that were active last week are still active?
  • How many % of users that were active last month are still active this month?…”

 

https://github.com/Doist/bitmapist


Redis Bit Operations Use Case at CopperEgg

“There’s always something interesting to work on at CopperEgg. Even the simplest things can get pretty interesting. Take billing for example. Seems like an easy thing to do at first, but once you realize the complexity of how we bill, you will see why it isn’t as simple as it seems.

We bill monthly, however we track usage hourly, a lot like Amazon Web Services pay-per-use model. In order to do that, you need to know when a service was used for any given hour so you can bill for that hour. We also want to know which hours we monitored a given system or website on a per customer basis. So we need to know the total hours of each service used over the month, we need to know which hours in the month a particular system/probe was being monitored, and how many hours per system/probe we monitored for that month. We also want to be able to show our customers throughout the month their usage to date.

Up until recently we had a process that would run and query for data (metrics) to see if a service was used during that time. That’s a bit nasty, since it puts constant load on our datastore just to find out if something sent us data in a particular timeframe. It also depended on other processes that we run, and makes billing depend on things that have nothing to do with billing. We wanted to split that out…”

http://copperegg.com/redis-bit-operations-use-case-at-copperegg/


Hipache: a distributed HTTP and websocket proxy

“Hipache is a distributed proxy designed to route high volumes of http and websocket traffic to unusually large numbers of virtual hosts, in a highly dynamic topology where backends are added and removed several times per second. It is particularly well-suited for PaaS (platform-as-a-service) and other environments that are both business-critical and multi-tenant.

Hipache was originally developed at dotCloud, a popular platform-as-a-service, to replace its first-generation routing layer based on a heavily instrumented nginx deployment. It currently serves production traffic for tens of thousands of applications hosted on dotCloud. Hipache is based on the node-http-proxy library…”

https://github.com/dotcloud/hipache

Follow

Get every new post delivered to your Inbox.

Join 514 other followers