Seems Convolution ops bug in relay that it does not support input and weight with mix data type uint8 and int8

guixiang.he · April 12, 2019, 3:09am

Because of the project reason, we need quantize convolution input from float32 to uint8, and weight from float32 to int8 using tvm relay quantize code. But it will appears following bug:

tvm/src/relay/pass/type_solver.cc:99: Check failed: resolved.defined() Unable to unify parent types: TensorType([64, 3, 7, 7], uint8) and TensorType([64, 3, 7, 7], int8)

and after changing

github.com

dmlc/tvm/blob/5a27632e274fff57087ed0b6eb2856b6e5946cfb/src/relay/pass/quantize.cc#L281


}
const auto* lhs = new_args[0].as<QRealizeIntExprNode>();
CHECK(lhs);
const auto* rhs = new_args[1].as<QRealizeIntExprNode>();
CHECK(rhs);


Expr ldata = lhs->data;
if (lhs->dtype != cfg->dtype_input) {
  ldata = Cast(ldata, cfg->dtype_input);
}
Expr rdata = Cast(rhs->data, cfg->dtype_weight);


const auto ref_attrs = ref_call->attrs.as<Conv2DAttrs>();
auto attrs = make_node<Conv2DAttrs>();
*attrs = *ref_attrs;
DataType out_dtype = cfg->dtype_activation;
attrs->out_dtype = out_dtype;


Expr ret = CallNode::make(ref_call->op,
  {ldata, rdata}, Attrs(attrs), ref_call->type_args);
Expr dom_scale = FoldConstant(Multiply(lhs->dom_scale, rhs->dom_scale));

from " reporter->Assign(types[1], TensorTypeNode::make(wshape, data->dtype));" to “reporter->Assign(types[1], TensorTypeNode::make(wshape, weight->dtype));”, the infer_type can pass, but it appears a new error, just like below. I will create a new question, thank you very much!

Log:
Traceback (most recent call last):
File “/home/ai/solomon/workspace/code/tvm/python/tvm/relay/backend/compile_engine.py”, line 76, in lower
return _backend._CompileEngineLower(self, key)
File “/home/ai/solomon/workspace/code/tvm/python/tvm/_ffi/_ctypes/function.py”, line 185, in call
ctypes.byref(ret_val), ctypes.byref(ret_tcode)))
File “/home/ai/solomon/workspace/code/tvm/python/tvm/_ffi/base.py”, line 71, in check_call
raise TVMError(py_str(_LIB.TVMGetLastError()))
tvm._ffi.base.TVMError: TVMCall CFunc Error:
Traceback (most recent call last):
File “/home/ai/solomon/workspace/code/tvm/python/tvm/_ffi/_ctypes/function.py”, line 55, in cfun
rv = local_pyfunc(*pyargs)
File “/home/ai/solomon/workspace/code/tvm/python/tvm/relay/op/nn/_nn.py”, line 337, in compute_contrib_conv2d_NCHWc
data_layout, out_layout, out_dtype)
File “”, line 2, in conv2d_NCHWc
File “/home/ai/solomon/workspace/code/tvm/python/tvm/target.py”, line 356, in dispatch_func
return dispatch_dict[k](*args, **kwargs)
File “”, line 2, in config_dispatcher
File “/home/ai/solomon/workspace/code/tvm/python/tvm/autotvm/task/dispatcher.py”, line 199, in dispatch_func
return dispatch_dict[‘direct’](cfg, *args, **kwargs)
File “/home/ai/solomon/workspace/code/tvm/python/tvm/autotvm/task/topi_integration.py”, line 267, in template_call
node = f(cfg, *args, **kwargs)
File “/home/ai/solomon/workspace/code/tvm/topi/python/topi/x86/conv2d.py”, line 377, in _declaration_conv_NCHWc
oc_chunk, _, kernel_height, kernel_width, _, oc_bn, _ = get_const_tuple(kernel.shape)
ValueError: not enough values to unpack (expected 7, got 6)

How to fix the bug? thank you very much.

eqy · April 12, 2019, 5:18am

The second problem should not be related to the first. Are you using opt_level=3 before running the quantization pass?

Either way, you can try to workaround this by using opt_level < 3 to disable the layout transformation.

guixiang.he · April 12, 2019, 8:03am

I modify the “AlterOpLayout”: from 3 to 4, and using opt_level = 3 after running the quantize pass(firstly, optimize(opt_level = 3); Then, quantize; Eventually, relay.build(opt_level = 3)). and it can pass now, thank you very much.