“I want to make solid-state drives (SSDs) the optimal storage solution for my key-value store project. For that reason, I had to make sure I fully understood how SSDs work, so that I can optimize my hash table implementation to suit their internal characteristics. There is a lot of incomplete and contradictory information out there, and finding trustworthy information about SSDs was not an easy task. I had to do some substantial reading to find the proper publications and benchmarks in order to convince myself that, if I had to be coding for SSDs, I would know what I was doing.
Then I figured that since I had done all the research, it would be useful to share the conclusions I had reached. My intent was to transform all the information already available into practical knowledge. I ended up writing a 30-page article, not very suitable for the format of a blog. I have therefore decided to split what I had written into logical parts that can be digested independently. The full Table of Contents is available at the bottom of this article.
The most remarkable contribution is Part 6, a summary of the whole “Coding for SSDs” article series, that I am sure programmers who are in a rush will appreciate. This summary covers the basics of SSDs along with all of the recommended access patterns on how reads and writes should be implemented to get the best performance out of solid-state drives.
Another important detail is that “Coding for SSDs” is independent from my key-value store project (IKVS series), and therefore, no prior knowledge of the IKVS articles is needed. I am planning on writing an article for the IKVS series, on how hash table can be implemented to take advantage of the internal characteristics of SSDs, though I have no precise date for that yet…”
“I just turned in my next course for Pluralsight: RethnkDB Fundamentals – almost in time for their announcement that 2.0 has been released. This is very exciting – it’s a “production-ready” release which means that all the work they’ve done over the last 3+ years is ready to go, and it’s mind-blowing…”
“since is a unix utility similar to tail. Unlike tail, since only shows the lines appended since the last time. It is useful to monitor growing log files…”
“HAProxy: Cornerstone of Reliable Websites
One primary goal of the infrastructure teams here at Yelp is to get as close to zero downtime as possible. This means that when users make requests for http://www.yelp.com we want to ensure that they get a response, and that they get a response as fast as possible. One way we do that at Yelp is by using the excellent HAProxy load balancer. We use it everywhere: for our external load balancing, internal load balancing, and with our move to a Service Oriented Architecture, we find ourselves running HAProxy on every machine at Yelp as part of SmartStack.
We love the flexibility that SmartStack gives us in developing our SOA, but that flexibility comes at a cost. When services or service backends are added or permanently removed, HAProxy has to reload across our entire infrastructure. These reloads can cause reliability problems because while HAProxy is top notch at not dropping traffic while it is running, it can (and does) drop traffic during reloads…”
“Over the coming month I will be architecting, building and testing a modular, high performance SSD-only storage solution.
I’ll be documenting my progress / findings along the way and open sourcing all the information as a public guide.
With recent price drops and durability improvements in solid state storage now is better time than any to ditch those old magnets.
Modular server manufacturers such as SuperMicro have spent large on R&D thanks to the ever growing requirements from cloud vendors that utilise their hardware…”