Important announcement: Missinglink has shut down. Click here to learn more.

Deep Learning Frameworks Cover

TensorFlow

TensorFlow Docker

Containers help improve security, reproducibility, and scalability in software development, data science, and deep learning. Using containers is an excellent way to create easy-to-deploy, repeatable environments for your deep learning projects. With Docker containers as the development environment, you don’t need to install and set up the environment for the various frameworks. Experiments are guaranteed to run everywhere and will run the same every time.

 

In this article we’ll explain how to deploy TensorFlow using Docker containers, and how to leverage MissingLink’s deep learning platform, an AI automation platform based on Docker, which lets you seamlessly manage data and resources for thousands of experiments.

What is Docker and How is it Used with TensorFlow?

Docker is a platform used to develop, deploy, and run applications by using the benefits of containerization. It ensures that all applications remain isolated in a container and packaged with all their dependencies and libraries.

 

Docker containers are used to create virtual environments, which can run TensorFlow. Within these environments, TensorFlow can, for example, share resources with its host machine, access directories, use a GPU, and connect to the internet, while remaining separate from the rest of the system.

 

Advantages of Using a Docker Image to set up TensorFlow

  • Dependencies and requirements are satisfied for all the different frameworks
  • Getting the right Anaconda distribution and the correct version of Python
  • Setting up the paths and the correct versions of different packages
  • Ensuring the installation does not interfere with other Python-based installations.

 

You can also test each release of the TensorFlow Docker images in these virtual environments. Using Docker also makes sharing projects with others a painless process. You don’t have to worry about environments not being compatible, missing dependencies or even platform conflicts.


Using Docker for Deep Learning in Research Environments

Many modern deep learning researchers rely on more than one framework. Running several deep learning frameworks or multiple versions of a framework on a single machine can be challenging.

Docker provides ready-to-go containers for each framework, which gives you the flexibility to experiment, without the risk of damaging your current environment. With Docker, the frameworks are entirely self-contained. If something is not working correctly, simply remove the entire container and start over, by performing a “docker run” command.

 

If you are only using one framework, like TensorFlow, in an experiment, it is probably already containerized. This means you can simply execute TensorFlow using the “docker run” command and it will not compromise your local workstation. You can simply download it, test it, and use it, and if the experiment doesn’t fit your needs, discard the container and move on


Using Docker for Deep Learning in Development Environments

As with any development project, it is important to make experiments and results reproducible. For deep learning, you should implement practices that effectively track code and ensure the training of data, weights, statistics, and files. You should also ensure this code and data can be rerun and reused with subsequent experiments. When you use containerized environments within Docker and images from Docker hub, you can achieve reproducible results in your deep learning experiments.

 

Containerized environments and images can be advantageous when you deploy deep learning at large organizations. You can also effectively use containerized environments for distributed development environments where deep learning talent is spread across different organizations, departments, or geographical regions. Also, you can manage and customize your deployment further by using orchestration tools like Kubernetes or Docker Swarm.


Supercharging Experiment Management and Reproduction with MissingLink

MissingLink abstracts away the complexity of managing Docker or Kubernetes, letting you manage and run experiments on multiple machines from a simple dashboard and CLI.

 

missinglink screenshot


Quick Tutorial: Real-Time TensorFlow Object Recognition with, Docker, and a GPU

The following tutorial steps are summarized. See the full tutorial by Reda Boumahdi.

 

In this tutorial, we package all the necessary tools needed for object recognition. The goal is to be able to run any computer vision algorithm on any machine within minutes.

 

1. Install Docker

> curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
> sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
> sudo apt-get update
> apt-cache policy docker-ce
> sudo apt-get install -y docker-ce
> sudo systemctl status docker

 

2. Install nvidia-docker to leverage NVIDIA GPUs

> wget https://github.com/NVIDIA/nvidia-docker/releases/download/v1.0.1/nvidia-docker_1.0.1-1_amd64.deb
> sudo dpkg -i nvidia-docker*.deb

2.1 Test your Installation

> sudo nvidia-docker run --rm nvidia/cuda nvidia-smi

 

3. Run a Jupyter Notebook with TensorFlow powered by GPU and OpenCv

> sudo nvidia-docker run --rm --name tf1 -p 8888:8888 -p 6006:6006 redaboumahdi/image_processing:gpu jupyter notebook --allow-root

Then you will need to navigate to your localhost and use the port 8888 and paste your token to identify and access your Jupyter notebooks.

 

4. Real-Time Object Recognition

The following code tracks objects over frames with a webcam. It is a sample of code taken from the following GitHub repository.

 

First of all, we need to open access to the xserver to our docker image. There are different ways of doing so. We will open access to xserver to anyone.

> xhost +local:root

Then we will bash to our Docker image using this command:

> sudo docker run -p 8888:8888 --device /dev/video0 --env="DISPLAY" --volume="/tmp/.X11-unix:/tmp/.X11-unix:rw" -it image_processing bash

We will need to clone the github repository, which is a real-time object detector:

> git clone https://github.com/datitran/object_detector_app.git && cd object_detector_app/

Finally, you can launch the python code:

> python object_detection_app.py

 


Scaling Up TensorFlow Docker Deployments

Train Deep Learning Models 20X Faster

Let us show you how you can:

  • Run experiments across hundreds of machines
  • Easily collaborate with your team on experiments
  • Reproduce experiments with one click
  • Save time and immediately understand what works and what doesn’t

MissingLink is the most comprehensive deep learning platform to manage experiments, data, and resources more frequently, at scale and with greater confidence.

Request your personal demo to start training models faster

    Thank you!
    We will be in touch with more information in one business day.
    In the meantime, why not check out how Nanit is using MissingLink to streamline deep learning training and accelerate time to Market.