Quantization (relay.quantize.qconfig) not working


#1

I used the relay quantization tool in the following way:

with relay.quantize.qconfig(nbit_input=16,
                                    nbit_weight=16,
                                    nbit_activation=16,
                                    dtype_input="int16",
                                    dtype_weight="int16",
                                    dtype_activation="int16",
                                    global_scale=8.0,
                                    skip_conv_layers=None,
                                    round_for_shift=True,
                                    store_lowbit_output=False,
                                    debug_enabled_ops=None,
                                    ):
    print("   ", relay.quantize.current_qconfig())
    mod["main"] = relay.quantize.quantize(mod["main"], params)

with relay.build_config(opt_level=3):
    graph, lib, params = relay.build(mod, target=target, params=params)
from tvm.contrib import graph_runtime
dtype = 'float32'
m = graph_runtime.create(graph, lib, ctx)
# set inputs
for i in range(20):
    m.set_input('x', x[i:i+1])
    m.set_input(**params)
    # execute
    m.run()
    # get outputs
    tvm_output = m.get_output(0, tvm.nd.empty(((1, 3)), 'float32'))
    print(tvm_output)

from print(tvm_output) function above I got the same output for every different image which are as follows:

[[ 3.4990034 18.61744    6.8425226]]
[[ 3.4990034 18.61744    6.8425226]]
[[ 3.4990034 18.61744    6.8425226]]
[[ 3.4990034 18.61744    6.8425226]]....

But if I remove the quantization function then I got the correct outputs with different values for different images as below:

[[2.6558046e+01 2.5423976e-02 1.7294771e+01]]
[[2.6697420e+01 1.6384061e-02 1.9536079e+01]]
[[2.6694643e+01 1.6997138e-02 1.8962639e+01]]
[[2.6695280e+01 1.6843578e-02 1.8819904e+01]]...

Can anybody please comment on what could be the issue?
relay.quantize.qconfig should at least produce different values for different inputs.