[Quantization] Problems with recent refactoring changes in the quantization pass


#1

Hi,

After the last changes in the quantization in the following commit, I am facing some issues:

  1. I get the following error:
  File "/home/tvm/tvm/python/tvm/relay/quantize/_partition.py", line 136, in add_partition_function
    if 'cuda' in _target.current_target().keys:
AttributeError: 'NoneType' object has no attribute 'keys'
  1. If I remove the problematic code in 1, the accuracy of a quantized model, which was previously working, drops to the point that the output of the model is not valid anymore.

@vinx13 can you have a look into this? I saw that the mentioned commit touches some aspects of accuracy.

Thanks


#2

@tico I can confirm this issue, cc @ziheng


#3

should be fixed by https://github.com/dmlc/tvm/pull/3792


#4

What’s the model with accuracy drop?


#5

I tested calibration for resnet18 v1, using non-power-2 scale, acc-top1 is 0.51, it raised error when using power-2 scale


#6

resnet18_v1 should be fine with configure here: https://github.com/dmlc/tvm/blob/master/tests/python/nightly/quantization/test_quantization_accuracy.py#L141


#7

Likely there are overflow with int8 addition when custom scales are used
After I commented out


and

resnet18 v1 works fine (acc 0.69)