Plumbing At Scale – Event Sourcing and Stream Processing Pipelines at Grab

As custodians and builders of the streaming platform at Grab operating at massive scale (think terabytes of data ingress each hour), the Coban team’s mission is to provide a NoOps, managed platform for seamless, secure access to event streams in real-time, for every team at Grab.

Coban Sewu Waterfall In Indonesia
Coban Sewu Waterfall In Indonesia. (Streams, get it?)

Streaming systems are often at the heart of event-driven architectures, and what starts as a need for a simple message bus for asynchronous processing of events quickly evolves into one that requires a more sophisticated stream processing paradigms. Earlier this year, we saw common patterns of event processing emerge across our Go backend ecosystem, including:

  • Filtering and mapping stream events of one type to another
  • Aggregating events into time windows and materializing them back to the event log or to various types of transactional and analytics databases

Generally, a class of problems surfaced which could be elegantly solved through an event sourcing1 platform with a stream processing framework built over it, similar to the Keystone platform at Netflix2.

This article details our journey building and deploying an event sourcing platform in Go, building a stream processing framework over it, and then scaling it (reliably and efficiently) to service over 300 billion events a week.

https://engineering.grab.com/plumbing-at-scale

URL shortening service written in Go and React

Short backend is built on top of Uncle Bob’s Clean Architecture, the central objective of which is separation of concerns.

Short adopts Microservices Architecture to organize dependent services around business capabilities and to enable independent deployment of each service.

Short leverages Kubernetes to automate deployment, scaling, and management of containerized microservices.

Short is maintained by a small team of talented software engineers working at Google, Uber, and Vmware as a side project.

https://github.com/short-d/short

Make resilient Go net/http servers using timeouts, deadlines and context cancellation

When it comes to timeouts, there are two types of people: those who know how tricky they can be, and those who are yet to find out.

As tricky as they are, timeouts are a reality in the connected world we live in. As I am writing this, on the other side of the table, two persons are typing on their smartphones, probably chatting to people very far from them. All made possible because of networks.

Networks and all their intricacies are here to stay, and we, who write servers for the web, have to know how to use them efficiently and guard against their deficiencies.

Without further ado, let’s look at timeouts and how they affect our net/http servers.

https://ieftimov.com/post/make-resilient-golang-net-http-servers-using-timeouts-deadlines-context-cancellation/

Microservices in Golang

Getting started, gRPC: https://ewanvalentine.io/microservices-in-golang-part-1/
Docker and micro: https://ewanvalentine.io/microservices-in-golang-part-2/
Docker compose and datastores: https://ewanvalentine.io/microservices-in-golang-part-3/
Authentication and JWT: https://ewanvalentine.io/microservices-in-golang-part-4/
Event brokering: https://ewanvalentine.io/microservices-in-golang-part-5/
Web Clients: https://ewanvalentine.io/microservices-in-golang-part-6/
Terraform: https://ewanvalentine.io/microservices-in-golang-part-7/
Kubernetes: https://ewanvalentine.io/microservices-in-golang-part-8/
CircleCI: https://ewanvalentine.io/microservices-in-golang-part-9/
Summary: https://ewanvalentine.io/microservices-in-golang-part-10/

 

Go: g0, Special Goroutine

All the goroutines created in Go are under the management of an internal scheduler. The Go scheduler tries to give running time to all goroutines and keep all CPU busy with running goroutines when the current ones are blocked or terminated. It actually runs as a special goroutine.

Scheduling goroutine

Go limits the number of OS thread running thanks to GOMAXPROCS variable simultaneously. That means Go has to schedule and manage goroutines on each of the running threads. This role is delegated to a special goroutine, called g0, that is the first goroutine created for each OS thread:

Then, it will schedule ready goroutines to run on the threads.

https://medium.com/a-journey-with-go/go-g0-special-goroutine-8c778c6704d8

Building a BitTorrent client from the ground up in Go

BitTorrent is a protocol for downloading and distributing files across the Internet. In contrast with the traditional client/server relationship, in which downloaders connect to a central server (for example: watching a movie on Netflix, or loading the web page you’re reading now), participants in the BitTorrent network, called peers, download pieces of files from each other—this is what makes it a peer-to-peer protocol. We’ll investigate how this works, and build our own client that can find peers and exchange data between them.

diagram showing the difference between client/server (all clients connecting to one server) and peer-to-peer (peers connecting to each other) relationships

The protocol evolved organically over the past 20 years, and various people and organizations added extensions for features like encryption, private torrents, and new ways of finding peers. We’ll be implementing the original spec from 2001 to keep this a weekend-sized project.

https://blog.jse.li/posts/torrent/