Deep Learning software packages

IBM provides software packages for several Deep Learning frameworks and supporting libraries:

Caffe - IBM, NVIDIA, and BVLC variants
Torch
Theano
OpenBLAS

All the packages are intended for use with Ubuntu 16.04 on POWER with NVIDIA CUDA 8.0 and cuDNN v5.1 packages.

System set up

Operating System

The Deep Learning framework packages require Ubuntu 16.04 for IBM POWER8. Ubuntu installation images can be downloaded from:

http://www.ubuntu.com/download/server/power8

NVIDIA components

The Deep Learning packages require NVIDIA CUDA 8.0 and cuDNN 5.1, which can be installed as follows:

Download and install NVIDIA CUDA 8.0 from https://developer.nvidia.com/cuda-downloads

Select Operating System: Linux
Select Architecture: ppc64le
Select Distribution Ubuntu
Select Version 16.04
Select the Installer Type that best fits your needs
Follow the Linux installation instructions in the CUDA Quick Start Guide linked from the download page, including the steps describing how to set up the CUDA development environment by updating PATH and LD_LIBRARY_PATH.

Download NVIDIA cuDNN 5.1 for CUDA 8.0 Power8 Deb packages from https://developer.nvidia.com/cudnn

cuDNN v5.1 Runtime Library for Ubuntu16.04 Power8 (Deb)
cuDNN v5.1 Developer Library for Ubuntu16.04 Power8 (Deb)
cuDNN v5.1 Code Samples and User Guide Power8 (Deb)

Registration in NVIDIA's Accelerated Computing Developer Program is required.

Install the cuDNN v5.1 packages
```
$ sudo dpkg -i libcudnn5*deb
```

NVIDIA GPU driver update

The Deep Learning framework packages will work with the version 361 GPU driver that ships with CUDA 8 but IBM recommends using driver 361.93.03 or higher.

NVIDIA driver updates for POWER8 are available from https://www.nvidia.com (select DRIVERS then All NVIDIA Drivers).

Software repository Setup

The Deep Learning packages are published as an Ubuntu package that sets up an intallation repositoriy on the local machine. The repository can be enabled as follows:

Download the latest mldl-repo-local .deb file from https://download.boulder.ibm.com/ibmdl/pub/software/server/mldl/
Install the repository package:
```
$ sudo dpkg -i mldl-repo-local*.deb
```
Update the package cache
```
$ sudo apt-get update
```

Tuning

Recommended settings for optimal Deep Learning performance on the S822LC for High Performance Computing:

Performance Governor

   $ sudo apt-get install linux-tools-common cpufrequtils
   $ sudo cpupower -c all frequency-set -g performance

GPU persistent mode

Use nvidia-persistenced (http://docs.nvidia.com/deploy/driver-persistence/index.html) or
```
   $ sudo nvidia-smi -pm ENABLED
```
GPU memory,sm frequency
```
   $ sudo nvidia-smi -ac 715,1480
```

Installing the Deep Learning Frameworks

Installing all frameworks at once

All the Deep Learning frameworks can be installed at once using the power-mldl meta-package:

    $ sudo apt-get install power-mldl

Installing frameworks individually

The Deep Learning frameworks can be installed individually if preferred. The framework packages are:

caffe-ibm - IBM Optimized version of the Berkeley Vision and Learning Center (BVLC) upstream Caffe, v1.0.0rc3
caffe-bvlc - Berkeley Vision and Learning Center (BVLC) upstream Caffe v1.0.0rc3
caffe-nv - NVIDIA fork of Caffe, v0.14.5
theano - Theano, v0.8.2
torch - Torch, v7

Each can be installed with:

    $ sudo apt-get install <framework>

Installation notes

Framework installation may pull in just a few or many (up to a few hundred) pre-requisite packages depending on currently installed packages.

Getting started with MLDL Frameworks

General setup

Each framework package provides a shell script to simplify environmental setup.

We recommend that users update their shell rc file (e.g. .bashrc) to source the desired setup scripts. For example:

    source /opt/DL/<framework>/bin/<framework>-activate

Each frame also provides a test script to verify basic function:

    $ <framework>-test

Packages for both upstream BVLC Caffe (/opt/DL/caffe-bvlc), IBM optimized BVLC Caffe (/opt/DL/caffe-ibm), and NVIDIA's fork of Caffe (/opt/DL/caffe-nv) are provided. You can choose which of those packages is the system default (/opt/DL/caffe) using Ubuntu's alternatives system:

    $ sudo update-alternatives --config caffe
    There are 3 choices for the alternative caffe (providing /opt/DL/caffe).

      Selection    Path                Priority   Status
    ------------------------------------------------------------
    * 0            /opt/DL/caffe-ibm   100        auto mode
      1            /opt/DL/caffe-bvlc   50        manual mode
      1            /opt/DL/caffe-ibm   100        manual mode
      2            /opt/DL/caffe-nv     75        manual mode

    Press enter to keep the current choice[*], or type selection number:

Users can activate the system default caffe:

    source /opt/DL/caffe/bin/caffe-activate

Or they can activate a specific variant. For example:

    source /opt/DL/caffe-bvlc/bin/caffe-activate

Attempting to activate both Caffe packages in a single login session will cause unpredictable behavior.

Caffe samples and examples

Each Caffe package includes example scripts and sample models, etc. A script is provided to copy the sample content into a specified directory:

    $ caffe-install-samples <somedir>

More info

Visit Caffe's website at http://caffe.berkeleyvision.org/ for tutorials and example programs that you can run to get started.

Here are links to a couple of the example programs from Caffe's website:

LeNet MNIST Tutorial - Train a neural network to understand handwritten digits
CIFAR-10 tutorial - Train a convolutional neural network to classify small images

Getting started with Theano

Here are some links to help you get started with Theano:

Visit Theano's website for the latest from Theano.

Getting started with Torch

The Torch Cheatsheet contains lots of info for people new to Torch, including tutorials and examples.

Here is a Torch tutorial that includes the code for several example programs:

Torch Tutorial (code for the tutorial)

Visit Torch's website for the latest from Torch.