Why Uber Engineering Switched from Postgres to MySQL

The early architecture of Uber consisted of a monolithic backend application written in Python that used Postgres for data persistence. Since that time, the architecture of Uber has changed significantly, to a model of microservices and new data platforms. Specifically, in many of the cases where we previously used Postgres, we now use Schemaless, a novel database sharding layer built on top of MySQL. In this article, we’ll explore some of the drawbacks we found with Postgres and explain the decision to build Schemaless and other backend services on top of MySQL.

 

The Architecture of Postgres

We encountered many Postgres limitations:

  • Inefficient architecture for writes
  • Inefficient data replication
  • Issues with table corruption
  • Poor replica MVCC support
  • Difficulty upgrading to newer releases

https://eng.uber.com/mysql-migration/

 

Advertisements

AWS Workshop for Kubernetes

This is a self-paced workshop designed for Development and Operations teams who would like to leverage Kubernetes on Amazon Web Services (AWS).

This workshop provides instructions to create, manage, and scale a Kubernetes cluster on AWS, as well as how to deploy applications, scale them, run stateless and stateful containers, perform service discovery between different microservices, and other similar concepts.

It also shows deep integration with several AWS technologies.

We recommend at least 2 hours to complete the workshop.

Simple deployment to Amazon EKS

Recently, Amazon announced that Elastic Container Service for Kubernetes (EKS) is generally available. Previously only open to a few folks via request access, EKS is now available to everyone, allowing all users to get managed Kubernetes clusters on AWS. In light of this development, we’re excited to announce official support for EKS with GitLab.

GitLab is designed for Kubernetes. While you can use GitLab to deploy anywhere, from bare metal to VMs, when you deploy to Kubernetes, you get access to the most powerful features. In this post, I’ll walk through our Kubernetes integration, highlight a few key features, and discuss how you can use the integration with EKS.

https://about.gitlab.com/2018/06/06/eks-gitlab-integration/

EKS vs. ECS: orchestrating containers on AWS

AWS announced Kubernetes-as-a-Service at re:Invent in November 2017: Elastic Container Service for Kubernetes (EKS). Since yesterday, EKS is generally available. I discussed ECS vs. Kubernetesbefore EKS was a thing. Therefore, I’d like to take a second attempt and compare EKS with ECS.

Before comparing the differences, let us start with what EKS and ECS have in common. Both solutions are managing containers distributed among a fleet of virtual machines. Managing containers includes:

  • Monitoring and replacing failed containers.
  • Deploying new versions of your containers.
  • Scaling the number of containers based on load.

What are the differences between EKS and ECS?

https://cloudonaut.io/eks-vs-ecs-orchestrating-containers-on-aws/

How To Scrape Web Pages with Beautiful Soup and Python 3

Many data analysis, big data, and machine learning projects require scraping websites to gather the data that you’ll be working with. The Python programming language is widely used in the data science community, and therefore has an ecosystem of modules and tools that you can use in your own projects. In this tutorial we will be focusing on the Beautiful Soup module.

Beautiful Soup, an allusion to the Mock Turtle’s song found in Chapter 10 of Lewis Carroll’s Alice’s Adventures in Wonderland, is a Python library that allows for quick turnaround on web scraping projects. Currently available as Beautiful Soup 4 and compatible with both Python 2.7 and Python 3, Beautiful Soup creates a parse tree from parsed HTML and XML documents (including documents with non-closed tags or tag soup and other malformed markup).

In this tutorial, we will collect and parse a web page in order to grab textual data and write the information we have gathered to a CSV file.

https://www.digitalocean.com/community/tutorials/how-to-scrape-web-pages-with-beautiful-soup-and-python-3

Python Application Layouts: A Reference

Python, though opinionated on syntax and style, is surprisingly flexible when it comes to structuring your applications.

On the one hand, this flexibility is great: it allows different use cases to use structures that are necessary for those use cases. On the other hand, though, it can be very confusing to the new developer.

The Internet isn’t a lot of help either—there are as many opinions as there are Python blogs. In this article, I want to give you a dependable Python application layout reference guide that you can refer to for the vast majority of your use cases.

You’ll see examples of common Python application structures, including command-line applications (CLI apps), one-off scripts, installable packages, and web application layouts with popular frameworks like Flask and Django.