[Quantization] Problems with recent refactoring changes in the quantization pass



After the last changes in the quantization in the following commit, I am facing some issues:

  1. I get the following error:
  File "/home/tvm/tvm/python/tvm/relay/quantize/_partition.py", line 136, in add_partition_function
    if 'cuda' in _target.current_target().keys:
AttributeError: 'NoneType' object has no attribute 'keys'
  1. If I remove the problematic code in 1, the accuracy of a quantized model, which was previously working, drops to the point that the output of the model is not valid anymore.

@vinx13 can you have a look into this? I saw that the mentioned commit touches some aspects of accuracy.



@tico I can confirm this issue, cc @ziheng


should be fixed by https://github.com/dmlc/tvm/pull/3792


What’s the model with accuracy drop?


I tested calibration for resnet18 v1, using non-power-2 scale, acc-top1 is 0.51, it raised error when using power-2 scale


resnet18_v1 should be fine with configure here: https://github.com/dmlc/tvm/blob/master/tests/python/nightly/quantization/test_quantization_accuracy.py#L141


Likely there are overflow with int8 addition when custom scales are used
After I commented out


resnet18 v1 works fine (acc 0.69)