Hello,
while trying to quantize a pretty huge model, I’m facing some limitations with the quantizer. When reaching certain layers, I receive the following error:
/home/anonymus/tvm/src/relay/quantize/quantize.cc:49: Check failed: data->shape.size() != 0 (0 vs. 0) : Input shape cannot be empty
My current workaround is to use the skip_conv_layers parameter to switch off the quantization of ‘problematic’ layers, which approach works fine - the rest of the model can be quantized.
Here is an excerpt of the IR around the operations that generate the previous error:
...
%60 = relay.op.annotation.simulated_quantize(meta[relay.Constant][14], %dom_scale5, %clip_min5, %clip_max5, kind=2);
%61 = nn.conv2d(%59, %60, padding=[1, 1, 1, 1], channels=64, kernel_size=[3, 3], data_layout="NHWC", kernel_layout="HWIO");
%62 = relay.op.annotation.simulated_quantize(%61, %dom_scale6, %clip_min6, %clip_max6, kind=1);
%63 = annotation.cast_hint(%62, meta[relay.attrs.CastHintAttrs][1]);
%64 = annotation.stop_fusion(%63);
%65 = add(%58, %64);
%66 = relay.op.annotation.simulated_quantize(0.2f, %dom_scale7, %clip_min7, %clip_max7, kind=2) an internal invariant was violated while typechecking your program [10:15:12] /home/anonymus/tvm/src/relay/quantize/quantize.cc:49: Check failed: data->shape.size() != 0 (0 vs. 0) : Input shape cannot be empty
; ;
%67 = multiply(%65, %66);
%68 = relay.op.annotation.simulated_quantize(0.5f, %dom_scale8, %clip_min8, %clip_max8, kind=2) an internal invariant was violated while typechecking your program [10:15:12] /home/anonymus/tvm/src/relay/quantize/quantize.cc:49: Check failed: data->shape.size() != 0 (0 vs. 0) : Input shape cannot be empty
; ;
%69 = add(%67, %68);
...
It seems to me that certain add/multiply combinations generate this issue, which have apparently a constant parameter to the relay.op.annotation.simulated_quantize(...) function.
Would anyone have an idea what could cause this? Any tips & hints are more than welcome!
Best regards, Robert
