TensorFlow is a widespread software library for numerical computation using data flow graphs. It is very common on machine learning and deep neural networks projects. Therefore, today we are going to see how to install it on POWER with CPU only configuration.
Before installing TensorFlow, there are a couple of details we have to pay attention to:
- Due to Bazel, one of TF dependencies, the operating system must be Ubuntu 14.04 or Ubuntu 16.04.
- We are going to use Python 2.7, since TF doesn’t seem to be supported by Python 3.5 on POWER.
You can use the commands below to solve most of the dependencies:
Bazel is one of the TF dependencies, but its installation is less intuitive than the others due to its community not officially supporting POWER architecture. That said, we must compile it from the Source. First of all, we need to install its own dependencies by the following commands:
It is also important to add enviroment variables on .bashrc for JDK.
For compiling Bazel, we are going to download and unpack its distribution archive (the zip file from the release page https://github.com/bazelbuild/bazel/releases. The .sh is not compatible with ppc64le) and compile it.
As we can see, this tutorial was tested with bazel 0.11.1, but feel free to try other version and see if it works properly.
Also, if you are having any trouble about lack of resources, you can take a look on ‘Build issues and Support Websites’ to see if there’s any link that could help you. Anticipating: if you don’t have memory enough and your Bazel can’t complete the compile step, you might have a problem with the garbage collector of JAVA (and there’s a link which explains how to deal with it).
Since we are going to use the current version of TF, we need to clone it from the official GitHub and execute the configuration script.
On this step, we have to specify the pathname of all relevant TF dependencies and other build configuration options. On most of them we can use the answers suggested on each question. Here, I will show how it was done for this tutorial. (Yours might be a little different, depending on the pathnames)
Using python library path: /usr/local/lib/python2.7/dist-packages
#Y/N Answers given: All of them as suggested in each question.
Please specify optimization flags to use during compilation when bazel option “–config=opt” is specified [Default is -march=native]: -mcpu=native Configuration finished
To build and install TF, we use:
By this moment, your TF must be working. Remember not to import it into its own directory: you have to chance directory before executing Python.
Build Issues and Support Websites:
While testing this tutorial, I could separate some useful issues reports and links to help some of the troubles you might have on the way.
- https://github.com/tensorflow/tensorflow/issues/14540 It solves a protobuf problem I had. It seems pretty common on PPC TF installation.
- https://github.com/tensorflow/tensorflow/issues/349 This one is about local resources. If you are running out of memory (your build fails on C++ compilation rules), you have to specify your resources on the command line when you build TF. On the tutorial, it is already done.
- https://www.tensorflow.org/install/install_sources An official tutorial about how to install TF from Sources
- https://docs.bazel.build/versions/master/install-compile-source.html An official tutorial about how to install Bazel from Sources.
- https://www.ibm.com/developerworks/community/blogs/fe313521-2e95-46f2-817d-44a4f27eba32/entry/Building_TensorFlow_on_OpenPOWER_Linux_Systems?lang=en IBM source about Tensorflow installation. Provides interesting information about bazel installation on PPC and how to install TF with GPU support. It also points to an IBM Bazel modified to PPC (which we are not using in this tutorial, but you can take a look on it).
- https://github.com/tensorflow/tensorflow/issues/7979#issuecomment-283559640 An issue about enviroment variables: on the configuration step, if it does not recognize some of the TF variables, this might help you to solve the problem.
- https://github.com/bazelbuild/bazel/issues/1308 An issue about Bazel: “The system is out of resources”. You might need to add a command line on compile file to change the garbage collector size. On the issue on git, it’s suggested to change it to 384, but, at least on one of the computers I tried to compile, I needed to change it to 512 (in other words, change -J-Xmx384m on the solution line to -J-Xmx512m). It’s important to see that ideally we should not have to change the source code, but it solves the problem. Another option is to increase the memory of your system if it’s possible (recommended).