Example of CPU inference using a uint8 network?

[skoppula]

Hi,

I notice that on the v0.5 roadmap uint8 support was proposed, added, and checked off: https://github.com/dmlc/tvm/issues/1596 (‘Intel cpu support’)

Where can I find an example that demonstrates this?

Thanks!

Also related, but can’t seem to find any information on these threads:

INT8 Quantization - Code generation for backends

For background on quantization, please read this link (INT8 quantization proposal). This thread only focuses on implementation of quantized layers in TVM. High-level overview Hardware vendors are adding support for optimized INT8 operations in the hardware (Intel (https://software.intel.com/en-us/articles/lower-numerical-precision-deep-learning-inference-and-training), Nvidia (https://devblogs.nvidia.com/mixed-precision-programming-cuda-8/)). To take full advantage of the hardware, we need to …

INT8 quantization proposal

Good point @tqchen I have added separate threads. This thread can act as a background post. Quantizing models (INT8 quantization - Quantizing models) Code generation for backend (INT8 Quantization - Code generation for backends)

[thierry]

Support in in progress, as far as I know @tqchen @ziheng