{"id":3457,"date":"2018-11-16T00:07:32","date_gmt":"2018-11-16T00:07:32","guid":{"rendered":"https:\/\/www.appservgrid.com\/paw92\/?p=3457"},"modified":"2018-11-17T15:04:57","modified_gmt":"2018-11-17T15:04:57","slug":"how-to-do-deep-machine-learning-tasks-inside-kvm-guests-with-a-passed-through-nvidia-gpu","status":"publish","type":"post","link":"https:\/\/www.appservgrid.com\/paw92\/index.php\/2018\/11\/16\/how-to-do-deep-machine-learning-tasks-inside-kvm-guests-with-a-passed-through-nvidia-gpu\/","title":{"rendered":"How to Do Deep Machine Learning Tasks Inside KVM Guests with a Passed-through NVIDIA GPU"},"content":{"rendered":"

This article shows how to run deep machine learning tasks in a SUSE Linux Enterprise Server 15 KVM guest. In a first step, you will learn how to do the train\/test tasks using CPU and GPU separately. After that, we can compare the performance differences.<\/p>\n

Preparation<\/h2>\n

But first of all, we need to do some preparation work before building both the Caffe<\/a> and the TensorFlow<\/a> frameworks with GPU support.<\/p>\n

1- Enable vt-d in the host bios and ensure the kernel parameter \u2018intel_iommu=on\u2019 is enabled.<\/p>\n

\"\"<\/p>\n

2- Pass the nv970GTX<\/em> on to the SUSE Linux Enterprise Server 15 KVM guest through libvirt.<\/p>\n

Note:
\n* If there are multiple devices in the same iommu group, you need to pass all of them on to the guest.
\n* What is passed-through is the 970GTX<\/em> physical function, not a vGPU<\/em> instance, because 970GTX<\/em> is not vGPU<\/em> capable.<\/p>\n

3- Disable the visibility of KVM to the guest by hiding the KVM signature. Otherwise, the newer public NVIDIA drivers and tools refuse to work (Please refer to qemu commit#f522d2a for the details).<\/p>\n

\"\"<\/p>\n

4- Install the official NVIDIA display driver in the guest:<\/p>\n

\"\"
\n\"\"<\/p>\n

5- Install Cuda 10<\/em><\/a>, cuDNN 7.3.1<\/em><\/a> and NCCL 2.3.5<\/em><\/a> in the guest:<\/p>\n

\"\"<\/p>\n

\"\"<\/p>\n

\"\"<\/p>\n

\"\"<\/p>\n

Build the Frameworks<\/h2>\n

Now it\u2019s time to build the TensorFlow framework with GPU support and the Caffe framework.<\/p>\n

As the existing whl package of TensorFlow 1.11 doesn\u2019t support Cuda 10<\/em> yet, I built TensorFlow 1.12<\/a> from the official Git source.<\/p>\n

\"\"<\/p>\n

\"\"<\/p>\n

\"\"<\/p>\n

As next step, build a whl package and install it.<\/p>\n

\"\"<\/p>\n

Now let\u2019s create a simple example to test the TensorFlow GPU in the guest:<\/p>\n

\"\"<\/p>\n

Through the nvidia-smi command, you can see the process information on GPU0 while the example code is running.<\/p>\n

\"\"<\/p>\n

Next, let\u2019s build the Caffe framework from the source, and the Caffe python wrapper.<\/p>\n

\"\"<\/p>\n

\"\"<\/p>\n

\"\"<\/p>\n

\"\"<\/p>\n

The setup is done!<\/p>\n

Examples<\/h2>\n

Now let\u2019s try to execute some deep learning tasks.<\/p>\n

Example 1.1: This is a Caffe built-in example. Please refer to http:\/\/caffe.berkeleyvision.org\/gathered\/examples\/mnist.html<\/a> to learn more.<\/p>\n

Let\u2019s use GPU0 in a guest to train this LeNET<\/em><\/a> model.<\/p>\n

\"\"<\/p>\n

During the training progress, we should see that the loss rate presents the downward trend all the time along with continuous iteration. But as the output is too long, I will not show it here.<\/p>\n

\"\"<\/p>\n

We got four files at the given folder after the training is done. This is because I set up the system to save the model and the training status every 5000 times. This means we get 2 files after 5000 iterations and 2 files after 10000 iterations.<\/p>\n

Now we got a trained model. Let\u2019s test it with 10000 test images to see how good the accuracy is.<\/p>\n

\"\"<\/p>\n

\"\"<\/p>\n

See? The accuracy is 0.9844. It is an acceptable result.<\/p>\n

Example 1.2: Now let\u2019s re-train a LeNET<\/em> model using CPU instead of GPU \u2013 and let\u2019s see what happens.<\/p>\n

\"\"<\/p>\n

\"\"<\/p>\n

\"\"<\/p>\n

\"\"<\/p>\n

When we compare the GPU and the CPU, we can see that there are huge performance differences, while we train\/test the LeNET<\/em> with the mnist dataset.<\/p>\n

We know that the traditional LeNET<\/em> convolutional neural network (CNN) contains seven layers. Except for the input layer, the MNIST<\/em> database contains 60,000 training images and 10,000 testing images. That means the performance differences become more between the training by CPU and the training by GPU when using deeper neural network layers.<\/p>\n

Example 2.1: This example is a TensorFlow built-in example. Let\u2019s do a very simple mnist classifier using the same mnist dataset.<\/p>\n

\"\"<\/p>\n

Here we go: As no convolutional layers are involved, the time consumed is quite short. It is only 8.5 seconds. But the accuracy is 0.92, which is not good enough.<\/p>\n

If you want, you can check all details through the TensorBoard.<\/p>\n

\"\"<\/p>\n

\"\"<\/p>\n

\"\"<\/p>\n

Example 2.2: Now we create a network with five layers CNN which is similar to the LeNET<\/em>. Let\u2019s re-train the system through GPU0 based on the TensorFlow framework.<\/p>\n

\"\"<\/p>\n

\"\"<\/p>\n

You can see now that the accuracy is 0.99 \u2013 it got much better, and the time consumed is only 2m 16s.<\/p>\n

Example 2.3<\/b>: Finally, let\u2019s redo example 2.2 with CPU instead of GPU0, to check the performance differences.<\/p>\n

\"\"<\/p>\n

\"\"<\/p>\n

With 0.99, the accuracy is really good now. But the time consumed is 19m 53s, which is way longer than the time consumed in example 2.2.<\/p>\n

Summary<\/h2>\n

Finally, let\u2019s summarize our test results:<\/p>\n