python - Install Tensorflow with Quantization Support

Question

This is a follow-up of another question by me : Error with 8-bit Quantization in Tensorflow

Basically, I would like to install the Tensorflow with 8-bit quantization support. Currently, I installed Tensorflow 0.9 with pip installation method on CentOS 7 machine (without GPU support).

I could compile and run the codes as given in Pete Warden's blog post. But, I can't import the functions given in Pete Warden's reply. I would like to add the quantization support. I couldn't find any details about the quantization part in the Tensorflow documentation also.

Can anybody share the details on how to do it?

Answer 1

For time-being, I could figure out a method to do this. But still waiting for official method from any TensorFlow developers.

1. First install the tensorflow ( I tried both source installation as well as PIP installation, both are fine)
2. Get the tensorflow source from the Github repo and go to the tensorflow root directory (I would call it tensorflow_root.
3. Now compile the quantization script as given in Pete Warden's blog

bazel build tensorflow/contrib/quantization/tools:quantize_graph

This wil create ops libraries for quantized versions. Go to tensorflow_root/bazel-bin/tensorflow/contrib/quantization and you should see two library files : _quantized_ops.so and kernels/_quantized_kernels.so

4. Now in your script, along with tensorflow, you should import these two library files also, using a dedicated tensorflow function

You can do it using tf.load_op_library() function

import tensorflow as tf
qops = tf.load_op_library('[tensorflow_root]/bazel-bin/tensorflow/contrib/quantization/_quantized_ops.so')
qkernelops = tf.load_op_library('[tensorflow_root]/bazel-bin/tensorflow/contrib/quantization/kernels/_quantized_kernels.so')