[Quantization] Selectively skipping layer(s) for quantization

[robeastbme]

Hello,

I’m currently facing a limitation with a model that TVM can’t fully quantize. My workaround is to skip layers that cause problems using the skip_conv_layers parameter. After a lot of experimentation, I have noticed that I have to remove the first 9 layers so that the rest of the model can be quantized: skip_conv_layers = [0, 1, 2, 3, 4, 5, 6, 7, 8]. However, after looking at the IR, it can be seen that only layers 5,6,7,8 are problematic. So I wanted to re-enable the quantization of the first layers, but I receive the following error:

  File "/home/anonymus/tvm/src/relay/quantize/realize.cc", line 382
TVMError: Check failed: !new_args[0]->IsInstance<TempExprNode>() && !new_args[1]->IsInstance<TempExprNode>(): 

I have tried many combinations, e.g., removing only 0 from the list, or 0 and 1, etc… but so far every combination lead to the previous error. Since I’m just entering the topic of quantization, I can’t really tell what could lead to this error.

Does anyone have an idea? Any tips & hints are more then welcome!

Thank you & Best regards, Robert

[robeastbme]

Could you @vinx13 maybe shed light on what is possibly going wrong here?

[adb]

@robeastbme

However, after looking at the IR, it can be seen that only layers 5,6,7,8 are problematic

Could you elaborate a bit more on this? What kind of failure are you seeing?

[robeastbme]

Hi @adb, thank you for looking into this! I have opened another thread on that particular issue:

[Quantization] Quantizer fails with specific layers

Hello, while trying to quantize a pretty huge model, I’m facing some limitations with the quantizer. When reaching certain layers, I receive the following error: /home/anonymus/tvm/src/relay/quantize/quantize.cc:49: Check failed: data->shape.size() != 0 (0 vs. 0) : Input shape cannot be empty My current workaround is to use the skip_conv_layers parameter to switch off the quantization of ‘problematic’ layers, which approach works fine - the rest of the model can be quantized. Here is an exce…

What do you think? Best regards, Robert

[okkk]

hello i’m deploying a mobilenetv3 model on pynq-z2 using VTA, i have the same problem when using quantization, i’m trying to skip the layers with zero input, but i can’t tell which layers are wrong, can you give me I have some suggestions? Looking forward to your reply！