A Practical Introduction to Deep Learning with Caffe and Python

Deep learning is the new big trend in machine learning. It had many recent successes in computer vision, automatic speech recognition and natural language processing.

The goal of this blog post is to give you a hands-on introduction to deep learning. To do this, we will build a Cat/Dog image classifier using a deep learning algorithm called convolutional neural network (CNN) and a Kaggle dataset.

This post is divided into 2 main parts. The first part covers some core concepts behind deep learning, while the second part is structured in a hands-on tutorial format.

In the first part of the hands-on tutorial (section 4), we will build a Cat/Dog image classifier using a convolutional neural network from scratch. In the second part of the tutorial (section 5), we will cover an advanced technique for training convolutional neural networks called transfer learning. We will use some Python code and a popular open source deep learning framework called Caffe to build the classifier. Our classifier will be able to achieve a classification accuracy of 97%.

By the end of this post, you will understand how convolutional neural networks work, and you will get familiar with the steps and the code for building these networks.

The source code for this tutorial can be found in this github repository.

http://adilmoujahid.com/posts/2016/06/introduction-deep-learning-python-caffe/

Tensorflow Image Recognition Tutorial

This tutorial shows how we can use MLDB’s TensorFlow integration to do image recognition. TensorFlow is Google’s open source deep learning library.

We will load the Inception-v3 model to generate discriptive labels for an image. The Inception model is a deep convolutional neural network and was trained on the ImageNet Large Visual Recognition Challenge dataset, where the task was to classify images into 1000 classes.

To offer context and a basis for comparison, this notebook is inspired by TensorFlow’s Image Recognition tutorial.

https://docs.mldb.ai/ipy/notebooks/_tutorials/_latest/Tensorflow%20Image%20Recognition%20Tutorial.html

Series of books diving deep into the core mechanisms of the JavaScript language

This is a series of books diving deep into the core mechanisms of the JavaScript language. The first edition of the series is now complete.

         

Please feel free to contribute to the quality of this content by submitting PR’s for improvements to code snippets, explanations, etc. While typo fixes are welcomed, they will likely be caught through normal editing processes, and are thus not necessarily as important for this repository.

To read more about the motivations and perspective behind this book series, check out the Preface.

Titles

https://github.com/getify/You-Dont-Know-JS

Close Encounters of The Java Memory Model Kind

Two years ago I painfully researched and built JMM Pragmatics talk and transcript, hoping it would highlight the particular dark corners of the Java Memory Models for those who cannot afford to spend years studying the formalisms, and deriving the actionable insights from them. JMM Pragmatics has helped many people, but there is still lots of confusion about what Memory Model guarantees, and what it does not.

In this post, we will try to follow up on particular misunderstandings about Java Memory Model, hopefully on the practical examples. The examples use the APIs from the jcstress suite, which are concise enough for a reader, and are runnable.

 

cart twitter updates

This is a fairly large piece of writing. In this age of attention-scarce Internet it would normally be presented as a series of posts, one section each. Let’s pretend we did that 🙂 If you run out of time/patience to read through, bookmark, and restart from the section you’ve left at.

Most examples in this post came from public and private discussions we’ve had over the years. I would not claim they cover all memory model abuses. If you have an interesting case that is not covered here, don’t hesitate to drop me a mail, and we can see if it fits the narrative somewhere. concurrency-interest is the mailing list where you can discuss interesting cases in public.

http://shipilev.net/blog/2016/close-encounters-of-jmm-kind/

Lighting the way to deep machine learning

Open source Torchnet helps researchers and developers build rapid and reusable prototypes of learning systems in Torch.

Building rapid and clean prototypes for deep machine-learning operations can now take a big step forward with Torchnet, a new software toolkit that fosters rapid and collaborative development of deep learning experiments by the Torch community.

Introduced and open-sourced this week at the International Conference on Machine Learning (ICML) in New York, Torchnet provides a collection of boilerplate code, key abstractions, and reference implementations that can be snapped together or taken apart and then later reused, substantially speeding development. It encourages a modular programming approach, reducing the chance of bugs while making it easy to use asynchronous, parallel data loading and efficient multi-GPU computations.

The new toolkit builds on the success of Torch, a framework for building deep learning models by providing fast implementations of common algebraic operations on both CPU (via OpenMP/SSE) and GPU (via CUDA).

https://code.facebook.com/posts/580706092103929

Python Concurrency

multiprocessing Basics

The simplest way to spawn a second is to instantiate a Process object with a target function and call start() to let it begin working…

https://pymotw.com/2/multiprocessing/basics.html

threading — Manage Concurrent Operations Within a Process

Using threads allows a program to run multiple operations concurrently in the same process space…

https://pymotw.com/3/threading/index.html#module-threading

asyncio — Asynchronous I/O, event loop, and concurrency tools

The asyncio module provides tools for building concurrent applications using coroutines. While the threading module implements concurrency through application threads and multiprocessing implements concurrency using system processes, asyncio uses a single-threaded, single-process approach in which parts of an application cooperate to switch tasks explicitly at optimal times. Most often this context switching occurs when the program would otherwise block waiting to read or write data, but asyncio also includes support for scheduling code to run at a specific future time, to enable one coroutine to wait for another to complete, for handling system signals, and for recognizing other events that may be reasons for an application to change what it is working on…

https://pymotw.com/3/asyncio/

Lessons learned from three container-management systems over a decade

Though widespread interest in software containers is a relatively recent phenomenon, at Google we have been managing Linux containers at scale for more than ten years and built three different container-management systems in that time. Each system was heavily influenced by its predecessors, even though they were developed for different reasons. This article describes the lessons we’ve learned from developing and operating them.

The first unified container-management system developed at Google was the system we internally call Borg.7 It was built to manage both long-running services and batch jobs, which had previously been handled by two separate systems: Babysitter and the Global Work Queue. The latter’s architecture strongly influenced Borg, but was focused on batch jobs; both predated Linux control groups. Borg shares machines between these two types of applications as a way of increasing resource utilization and thereby reducing costs. Such sharing was possible because container support in the Linux kernel was becoming available (indeed, Google contributed much of the container code to the Linux kernel), which enabled better isolation between latency-sensitive user-facing services and CPU-hungry batch processes.

As more and more applications were developed to run on top of Borg, our application and infrastructure teams developed a broad ecosystem of tools and services for it. These systems provided mechanisms for configuring and updating jobs; predicting resource requirements; dynamically pushing configuration files to running jobs; service discovery and load balancing; auto-scaling; machine-lifecycle management; quota management; and much more. The development of this ecosystem was driven by the needs of different teams inside Google, and the result was a somewhat heterogeneous, ad-hoc collection of systems that Borg’s users had to configure and interact with, using several different configuration languages and processes. Borg remains the primary container-management system within Google because of its scale, breadth of features, and extreme robustness.

Omega,6 an offspring of Borg, was driven by a desire to improve the software engineering of the Borg ecosystem. It applied many of the patterns that had proved successful in Borg, but was built from the ground up to have a more consistent, principled architecture. Omega stored the state of the cluster in a centralized Paxos-based transaction-oriented store that was accessed by the different parts of the cluster control plane (such as schedulers), using optimistic concurrency control to handle the occasional conflicts. This decoupling allowed the Borgmaster’s functionality to be broken into separate components that acted as peers, rather than funneling every change through a monolithic, centralized master. Many of Omega’s innovations (including multiple schedulers) have since been folded into Borg.

The third container-management system developed at Google was Kubernetes.4 It was conceived of and developed in a world where external developers were becoming interested in Linux containers, and Google had developed a growing business selling public-cloud infrastructure. Kubernetes is open source—a contrast to Borg and Omega, which were developed as purely Google-internal systems. Like Omega, Kubernetes has at its core a shared persistent store, with components watching for changes to relevant objects. In contrast to Omega, which exposes the store directly to trusted control-plane components, state in Kubernetes is accessed exclusively through a domain-specific REST API that applies higher-level versioning, validation, semantics, and policy, in support of a more diverse array of clients. More importantly, Kubernetes was developed with a stronger focus on the experience of developers writing applications that run in a cluster: its main design goal is to make it easy to deploy and manage complex distributed systems, while still benefiting from the improved utilization that containers enable.

This article describes some of the knowledge gained and lessons learned during Google’s journey from Borg to Kubernetes.

http://queue.acm.org/detail.cfm?id=2898444