This post originated from an RSS feed registered with .NET Buzz by Lutz Roeder. Original Post: TensorFlow on Azure GPU Feed Title: Lutz Roeder's Weblog Feed URL: https://feeds.feedburner.com/lutzroeder Feed Description: ... Latest .NET Buzz Posts Latest .NET Buzz Posts by Lutz Roeder Latest Posts From Lutz Roeder's Weblog

Training neural networks (deep learning) is very compute-intensive. Fast GPUs can make those sessions, which sometimes take hours, days or weeks go orders of magnitude faster. However, laptops usually don't come with the fastest GPUs and having to maintain a desktop machine only to occasionally run deep learning tasks is extra hassle.

Cloud providers now offer virtual machines (VMs) with GPUs which run in data centers and can be used by anybody on an hourly basis. Below is a quick tutorial that walks through setting up a VM in Microsoft Azure with the necessary drivers to train neural networks using TensorFlow.

First, if you haven't done so already, create an Azure account, install the Azure 2.0 command line interface (CLI)...

sudo pip install azure-cli

... and follow the login procedure:

az login

Azure manages resources (virtual machines, storage etc.) via resource groups. GPU virtual machine instances are currently available in the East US region. If you already have a group for that region feel free to use it, otherwise create a new resource group:

az group create --name tensorflow --location EastUS

We will connect to the machine via SSH and need to create a key pair:

ssh-keygen -f ~/.ssh/tensorflow_id_rsa -t rsa -b 2048 -C '' -N ''

Next, we create the actual virtual machine running Ubuntu 16.04. We choose the cheapest and least powerful GPU size (NC6) and downgrade from premium (SSD) to standard storage (HDD) as the former is not supported for NC instances yet.

az vm create --resource-group tensorflow --name tensorflow --image Canonical:UbuntuServer:16.04-LTS:latest --size Standard_NC6 --storage-type Standard_LRS --admin-username tensorflow --ssh-key-value ~/.ssh/tensorflow_id_rsa.pub

Once completed, the command will print the IP address for the newly created machine:

{
  "publicIpAddress": "127.0.0.1",
  "resourceGroup": "tensorflow"
}

The VM is now running in a data center (and charging for cycles). The following commands can be used to shut down and restart anytime:

az vm stop  --resource-group tensorflow --name tensorflow
az vm start --resource-group tensorflow --name tensorflow

Connect to the machine via SSH using the IP address printed above (type 'yes', if asked to continue):

ssh tensorflow@127.0.0.1 -i ~/.ssh/tensorflow_id_rsa

Next, download the CUDA package from Nvidia, ...

wget https://developer.nvidia.com/compute/cuda/8.0/prod/local_installers/cuda-repo-ubuntu1604-8-0-local_8.0.44-1_amd64-deb

... make it known to apt-get and install the CUDA Toolkit:

sudo dpkg -i cuda-repo-ubuntu1604-8-0-local_8.0.44-1_amd64-deb
sudo apt-get update
sudo apt-get install -y cuda

Now we can check the status of the GPU(s) by running...

nvidia-smi

Next, we download cuDNN...

wget http://developer.download.nvidia.com/compute/redist/cudnn/v5.1/cudnn-8.0-linux-x64-v5.1.tgz

... unzip, copy the lib64 and include folders:

tar -zxf cudnn-8.0-linux-x64-v5.1.tgz
sudo cp cuda/lib64/* /usr/local/cuda/lib64/
sudo cp cuda/include/* /usr/local/cuda/include/
sudo rm -R cuda

Time to do some clean up and remove the downloaded archives:

rm cuda-repo-ubuntu1604-8-0-local_8.0.44-1_amd64-deb
rm cudnn-8.0-linux-x64-v5.1.tgz

The final step is to install Pip and the GPU version of TensorFlow:

sudo apt-get install -y python-pip python-dev
sudo pip install -y tensorflow-gpu

We can now start a Python console and create a TensorFlow session:

python
>>> import tensorflow as tf
>>> session = tf.Session()

If everything went well, it will recognize the Tesla K80 GPU:

I tensorflow/core/common_runtime/gpu/gpu_device.cc:885]
Found device 0 with properties: 
name: Tesla K80
major: 3 minor: 7 memoryClockRate (GHz) 0.8235
pciBusID b0b5:00:00.0
Total memory: 11.17GiB
Free memory: 11.11GiB

Remember to stop the VM when done to avoid using cycles:

az vm stop --resource-group tensorflow --name tensorflow

Once no longer needed, you can delete the virtual machine by running:

az vm delete --resource-group tensorflow --name tensorflow