Fail to load ONNX ResNet-50

Questions
#1

I start with Tutorial: compile onnx models and modify CNN model to ResNet-50. However, I got a screen of C++ stack errors.

Code to re-produce

import os, os.path as osp
import tarfile

import onnx
import numpy as np
import tvm
import tvm.relay as relay
from tvm.contrib.download import download_testdata, download

# https://github.com/onnx/models/tree/master/resnet50
model_url = "https://s3.amazonaws.com/download.onnx/models/opset_9/resnet50.tar.gz"
model_path = download_testdata(model_url, 'resnet50.tar.gz', module='onnx')
# now you have super_resolution.onnx on disk
tar = tarfile.open(model_path)
tar.extractall(osp.dirname(model_path))
tar.close()

mpath = model_path.replace(".tar.gz", "/model.onnx")
onnx_model = onnx.load(mpath)


from PIL import Image
img_url = 'https://github.com/dmlc/mxnet.js/blob/master/data/cat.png?raw=true'
img_path = download_testdata(img_url, 'cat.png', module='data')
img = Image.open(img_path).resize((224, 224))
img_ycbcr = img.convert("YCbCr")  # convert to YCbCr
img_y, img_cb, img_cr = img_ycbcr.split()
x = np.array(img_y)[np.newaxis, np.newaxis, :, :]

target = 'llvm'

input_name = 'gpu_0/data_0' # the input layer
shape_dict = {input_name: x.shape}
sym, params = relay.frontend.from_onnx(onnx_model, shape_dict)

with relay.build_config(opt_level=1):
    intrp = relay.build_module.create_executor('graph', sym, tvm.cpu(0), target)

dtype = 'float32'
tvm_output = intrp.evaluate(sym)(tvm.nd.array(x.astype(dtype)), **params).asnumpy()

Error logs

Any advice to solve this bug?

1 Like
#2

moved from https://github.com/dmlc/tvm/issues/3307

#3

Thanks for reporting this. The error did reveal one thing we should improve https://github.com/dmlc/tvm/blob/master/src/relay/op/nn/nn.cc#L51 to add https://github.com/dmlc/tvm/blob/master/src/relay/op/nn/convolution.cc#L45

The error hint means the type inference didn’t get to infer the type of the input around avg_pool. It would be great if you can help dig a bit into it.

#4

I found this bug was actually caused by incorrect input data. In SR tutorial, the input has size [1x1x224x224] but the channel number should be 3 for imagenet classification.

By switching the pre-processing from

img = Image.open(img_path).resize((224, 224))
img_ycbcr = img.convert("YCbCr")  # convert to YCbCr
img_y, img_cb, img_cr = img_ycbcr.split()
x = np.array(img_y)[np.newaxis, np.newaxis, :, :]

to

img = Image.open(img_path).resize((224, 224))
x = np.array(img).transpose(2, 0, 1)[np.newaxis, :, :, :]

The code executes without error.

#5

And interestingly, I notice with the wrong input shape, only conv and pooling layers raise the error.
It makes sense for conv layer since channel number mismatches, but for pooling layer, I am curious why it also fails to infer the shape.

After some exploration, I find the nullptr checks for nn, convolution, pooling are different.

It seems Pool2DRel does not return any value when data == nullptr.

#6

In my understanding, if there is any inconsistency in the previous layer (e.g., data shape, weight size), the data will become nullptr and warn the following layers. However, it may cause ambiguity sometime. In my example, the error is not related to the pooling layer. Why not let the program abort at the first place where inference error occurs?

#7

You are right and a PR is more than welcomed to fix this part

#8

There is certainly a tradeoff about deferring the inference vs where it actually first occurs. It acts like a message prop algorithm that can go both ways. The current error reporter tries to keep inferencing until get to a point that we can not.

Do you think if there is a particular place where the error should be reporting? would be great to see suggestions about what the error message should be like. Perhaps outputting the partial type inference result would be helpful here.

cc @jroesch

[solved][mxnet] nullptr with nn.max_pool2d
#9

Pull request created https://github.com/dmlc/tvm/pull/3322

From my personal experience, if there is an error in DAG (or relay), I would prefer to see the exact position of first error instead of all potential errors across the graph.

For example, current error trace

fn (%gpu_0/data_0: Tensor[(1, 1, 224, 224), float32], %gpu_0/conv1_w_0: Tensor[(64, 3, 7, 7), float32], %gpu_0/res_conv1_bn_s_0: Tensor[(64,), float32], %gpu_0/res_conv1_bn_b_0: Tensor[(64,), float32], %gpu_0/res_conv1_bn_rm_0: Tensor[(64,), float32], %gpu_0/res_conv1_bn_riv_0: Tensor[(64,), float32], %gpu_0/res2_0_branch2a_w_0: Tensor[(64, 64, 1, 1), float32], %gpu_0/res2_0_branch2a_bn_s_0: Tensor[(64,), float32], %gpu_0/res2_0_branch2a_bn_b_0: Tensor[(64,), float32], %gpu_0/res2_0_branch2a_bn_rm_0: Tensor[(64,), float32], %gpu_0/res2_0_branch2a_bn_riv_0: Tensor[(64,), float32], %gpu_0/res2_0_branch2b_w_0: Tensor[(64, 64, 3, 3), float32], %gpu_0/res2_0_branch2b_bn_s_0: Tensor[(64,), float32], %gpu_0/res2_0_branch2b_bn_b_0: Tensor[(64,), float32], %gpu_0/res2_0_branch2b_bn_rm_0: Tensor[(64,), float32], %gpu_0/res2_0_branch2b_bn_riv_0: Tensor[(64,), float32], %gpu_0/res2_0_branch2c_w_0: Tensor[(256, 64, 1, 1), float32], %gpu_0/res2_0_branch2c_bn_s_0: Tensor[(256,), float32], %gpu_0/res2_0_branch2c_bn_b_0: Tensor[(256,), float32], %gpu_0/res2_0_branch2c_bn_rm_0: Tensor[(256,), float32], %gpu_0/res2_0_branch2c_bn_riv_0: Tensor[(256,), float32], %gpu_0/res2_0_branch1_w_0: Tensor[(256, 64, 1, 1), float32], %gpu_0/res2_0_branch1_bn_s_0: Tensor[(256,), float32], %gpu_0/res2_0_branch1_bn_b_0: Tensor[(256,), float32], %gpu_0/res2_0_branch1_bn_rm_0: Tensor[(256,), float32], %gpu_0/res2_0_branch1_bn_riv_0: Tensor[(256,), float32], %gpu_0/res2_1_branch2a_w_0: Tensor[(64, 256, 1, 1), float32], %gpu_0/res2_1_branch2a_bn_s_0: Tensor[(64,), float32], %gpu_0/res2_1_branch2a_bn_b_0: Tensor[(64,), float32], %gpu_0/res2_1_branch2a_bn_rm_0: Tensor[(64,), float32], %gpu_0/res2_1_branch2a_bn_riv_0: Tensor[(64,), float32], %gpu_0/res2_1_branch2b_w_0: Tensor[(64, 64, 3, 3), float32], %gpu_0/res2_1_branch2b_bn_s_0: Tensor[(64,), float32], %gpu_0/res2_1_branch2b_bn_b_0: Tensor[(64,), float32], %gpu_0/res2_1_branch2b_bn_rm_0: Tensor[(64,), float32], %gpu_0/res2_1_branch2b_bn_riv_0: Tensor[(64,), float32], %gpu_0/res2_1_branch2c_w_0: Tensor[(256, 64, 1, 1), float32], %gpu_0/res2_1_branch2c_bn_s_0: Tensor[(256,), float32], %gpu_0/res2_1_branch2c_bn_b_0: Tensor[(256,), float32], %gpu_0/res2_1_branch2c_bn_rm_0: Tensor[(256,), float32], %gpu_0/res2_1_branch2c_bn_riv_0: Tensor[(256,), float32], %gpu_0/res2_2_branch2a_w_0: Tensor[(64, 256, 1, 1), float32], %gpu_0/res2_2_branch2a_bn_s_0: Tensor[(64,), float32], %gpu_0/res2_2_branch2a_bn_b_0: Tensor[(64,), float32], %gpu_0/res2_2_branch2a_bn_rm_0: Tensor[(64,), float32], %gpu_0/res2_2_branch2a_bn_riv_0: Tensor[(64,), float32], %gpu_0/res2_2_branch2b_w_0: Tensor[(64, 64, 3, 3), float32], %gpu_0/res2_2_branch2b_bn_s_0: Tensor[(64,), float32], %gpu_0/res2_2_branch2b_bn_b_0: Tensor[(64,), float32], %gpu_0/res2_2_branch2b_bn_rm_0: Tensor[(64,), float32], %gpu_0/res2_2_branch2b_bn_riv_0: Tensor[(64,), float32], %gpu_0/res2_2_branch2c_w_0: Tensor[(256, 64, 1, 1), float32], %gpu_0/res2_2_branch2c_bn_s_0: Tensor[(256,), float32], %gpu_0/res2_2_branch2c_bn_b_0: Tensor[(256,), float32], %gpu_0/res2_2_branch2c_bn_rm_0: Tensor[(256,), float32], %gpu_0/res2_2_branch2c_bn_riv_0: Tensor[(256,), float32], %gpu_0/res3_0_branch2a_w_0: Tensor[(128, 256, 1, 1), float32], %gpu_0/res3_0_branch2a_bn_s_0: Tensor[(128,), float32], %gpu_0/res3_0_branch2a_bn_b_0: Tensor[(128,), float32], %gpu_0/res3_0_branch2a_bn_rm_0: Tensor[(128,), float32], %gpu_0/res3_0_branch2a_bn_riv_0: Tensor[(128,), float32], %gpu_0/res3_0_branch2b_w_0: Tensor[(128, 128, 3, 3), float32], %gpu_0/res3_0_branch2b_bn_s_0: Tensor[(128,), float32], %gpu_0/res3_0_branch2b_bn_b_0: Tensor[(128,), float32], %gpu_0/res3_0_branch2b_bn_rm_0: Tensor[(128,), float32], %gpu_0/res3_0_branch2b_bn_riv_0: Tensor[(128,), float32], %gpu_0/res3_0_branch2c_w_0: Tensor[(512, 128, 1, 1), float32], %gpu_0/res3_0_branch2c_bn_s_0: Tensor[(512,), float32], %gpu_0/res3_0_branch2c_bn_b_0: Tensor[(512,), float32], %gpu_0/res3_0_branch2c_bn_rm_0: Tensor[(512,), float32], %gpu_0/res3_0_branch2c_bn_riv_0: Tensor[(512,), float32], %gpu_0/res3_0_branch1_w_0: Tensor[(512, 256, 1, 1), float32], %gpu_0/res3_0_branch1_bn_s_0: Tensor[(512,), float32], %gpu_0/res3_0_branch1_bn_b_0: Tensor[(512,), float32], %gpu_0/res3_0_branch1_bn_rm_0: Tensor[(512,), float32], %gpu_0/res3_0_branch1_bn_riv_0: Tensor[(512,), float32], %gpu_0/res3_1_branch2a_w_0: Tensor[(128, 512, 1, 1), float32], %gpu_0/res3_1_branch2a_bn_s_0: Tensor[(128,), float32], %gpu_0/res3_1_branch2a_bn_b_0: Tensor[(128,), float32], %gpu_0/res3_1_branch2a_bn_rm_0: Tensor[(128,), float32], %gpu_0/res3_1_branch2a_bn_riv_0: Tensor[(128,), float32], %gpu_0/res3_1_branch2b_w_0: Tensor[(128, 128, 3, 3), float32], %gpu_0/res3_1_branch2b_bn_s_0: Tensor[(128,), float32], %gpu_0/res3_1_branch2b_bn_b_0: Tensor[(128,), float32], %gpu_0/res3_1_branch2b_bn_rm_0: Tensor[(128,), float32], %gpu_0/res3_1_branch2b_bn_riv_0: Tensor[(128,), float32], %gpu_0/res3_1_branch2c_w_0: Tensor[(512, 128, 1, 1), float32], %gpu_0/res3_1_branch2c_bn_s_0: Tensor[(512,), float32], %gpu_0/res3_1_branch2c_bn_b_0: Tensor[(512,), float32], %gpu_0/res3_1_branch2c_bn_rm_0: Tensor[(512,), float32], %gpu_0/res3_1_branch2c_bn_riv_0: Tensor[(512,), float32], %gpu_0/res3_2_branch2a_w_0: Tensor[(128, 512, 1, 1), float32], %gpu_0/res3_2_branch2a_bn_s_0: Tensor[(128,), float32], %gpu_0/res3_2_branch2a_bn_b_0: Tensor[(128,), float32], %gpu_0/res3_2_branch2a_bn_rm_0: Tensor[(128,), float32], %gpu_0/res3_2_branch2a_bn_riv_0: Tensor[(128,), float32], %gpu_0/res3_2_branch2b_w_0: Tensor[(128, 128, 3, 3), float32], %gpu_0/res3_2_branch2b_bn_s_0: Tensor[(128,), float32], %gpu_0/res3_2_branch2b_bn_b_0: Tensor[(128,), float32], %gpu_0/res3_2_branch2b_bn_rm_0: Tensor[(128,), float32], %gpu_0/res3_2_branch2b_bn_riv_0: Tensor[(128,), float32], %gpu_0/res3_2_branch2c_w_0: Tensor[(512, 128, 1, 1), float32], %gpu_0/res3_2_branch2c_bn_s_0: Tensor[(512,), float32], %gpu_0/res3_2_branch2c_bn_b_0: Tensor[(512,), float32], %gpu_0/res3_2_branch2c_bn_rm_0: Tensor[(512,), float32], %gpu_0/res3_2_branch2c_bn_riv_0: Tensor[(512,), float32], %gpu_0/res3_3_branch2a_w_0: Tensor[(128, 512, 1, 1), float32], %gpu_0/res3_3_branch2a_bn_s_0: Tensor[(128,), float32], %gpu_0/res3_3_branch2a_bn_b_0: Tensor[(128,), float32], %gpu_0/res3_3_branch2a_bn_rm_0: Tensor[(128,), float32], %gpu_0/res3_3_branch2a_bn_riv_0: Tensor[(128,), float32], %gpu_0/res3_3_branch2b_w_0: Tensor[(128, 128, 3, 3), float32], %gpu_0/res3_3_branch2b_bn_s_0: Tensor[(128,), float32], %gpu_0/res3_3_branch2b_bn_b_0: Tensor[(128,), float32], %gpu_0/res3_3_branch2b_bn_rm_0: Tensor[(128,), float32], %gpu_0/res3_3_branch2b_bn_riv_0: Tensor[(128,), float32], %gpu_0/res3_3_branch2c_w_0: Tensor[(512, 128, 1, 1), float32], %gpu_0/res3_3_branch2c_bn_s_0: Tensor[(512,), float32], %gpu_0/res3_3_branch2c_bn_b_0: Tensor[(512,), float32], %gpu_0/res3_3_branch2c_bn_rm_0: Tensor[(512,), float32], %gpu_0/res3_3_branch2c_bn_riv_0: Tensor[(512,), float32], %gpu_0/res4_0_branch2a_w_0: Tensor[(256, 512, 1, 1), float32], %gpu_0/res4_0_branch2a_bn_s_0: Tensor[(256,), float32], %gpu_0/res4_0_branch2a_bn_b_0: Tensor[(256,), float32], %gpu_0/res4_0_branch2a_bn_rm_0: Tensor[(256,), float32], %gpu_0/res4_0_branch2a_bn_riv_0: Tensor[(256,), float32], %gpu_0/res4_0_branch2b_w_0: Tensor[(256, 256, 3, 3), float32], %gpu_0/res4_0_branch2b_bn_s_0: Tensor[(256,), float32], %gpu_0/res4_0_branch2b_bn_b_0: Tensor[(256,), float32], %gpu_0/res4_0_branch2b_bn_rm_0: Tensor[(256,), float32], %gpu_0/res4_0_branch2b_bn_riv_0: Tensor[(256,), float32], %gpu_0/res4_0_branch2c_w_0: Tensor[(1024, 256, 1, 1), float32], %gpu_0/res4_0_branch2c_bn_s_0: Tensor[(1024,), float32], %gpu_0/res4_0_branch2c_bn_b_0: Tensor[(1024,), float32], %gpu_0/res4_0_branch2c_bn_rm_0: Tensor[(1024,), float32], %gpu_0/res4_0_branch2c_bn_riv_0: Tensor[(1024,), float32], %gpu_0/res4_0_branch1_w_0: Tensor[(1024, 512, 1, 1), float32], %gpu_0/res4_0_branch1_bn_s_0: Tensor[(1024,), float32], %gpu_0/res4_0_branch1_bn_b_0: Tensor[(1024,), float32], %gpu_0/res4_0_branch1_bn_rm_0: Tensor[(1024,), float32], %gpu_0/res4_0_branch1_bn_riv_0: Tensor[(1024,), float32], %gpu_0/res4_1_branch2a_w_0: Tensor[(256, 1024, 1, 1), float32], %gpu_0/res4_1_branch2a_bn_s_0: Tensor[(256,), float32], %gpu_0/res4_1_branch2a_bn_b_0: Tensor[(256,), float32], %gpu_0/res4_1_branch2a_bn_rm_0: Tensor[(256,), float32], %gpu_0/res4_1_branch2a_bn_riv_0: Tensor[(256,), float32], %gpu_0/res4_1_branch2b_w_0: Tensor[(256, 256, 3, 3), float32], %gpu_0/res4_1_branch2b_bn_s_0: Tensor[(256,), float32], %gpu_0/res4_1_branch2b_bn_b_0: Tensor[(256,), float32], %gpu_0/res4_1_branch2b_bn_rm_0: Tensor[(256,), float32], %gpu_0/res4_1_branch2b_bn_riv_0: Tensor[(256,), float32], %gpu_0/res4_1_branch2c_w_0: Tensor[(1024, 256, 1, 1), float32], %gpu_0/res4_1_branch2c_bn_s_0: Tensor[(1024,), float32], %gpu_0/res4_1_branch2c_bn_b_0: Tensor[(1024,), float32], %gpu_0/res4_1_branch2c_bn_rm_0: Tensor[(1024,), float32], %gpu_0/res4_1_branch2c_bn_riv_0: Tensor[(1024,), float32], %gpu_0/res4_2_branch2a_w_0: Tensor[(256, 1024, 1, 1), float32], %gpu_0/res4_2_branch2a_bn_s_0: Tensor[(256,), float32], %gpu_0/res4_2_branch2a_bn_b_0: Tensor[(256,), float32], %gpu_0/res4_2_branch2a_bn_rm_0: Tensor[(256,), float32], %gpu_0/res4_2_branch2a_bn_riv_0: Tensor[(256,), float32], %gpu_0/res4_2_branch2b_w_0: Tensor[(256, 256, 3, 3), float32], %gpu_0/res4_2_branch2b_bn_s_0: Tensor[(256,), float32], %gpu_0/res4_2_branch2b_bn_b_0: Tensor[(256,), float32], %gpu_0/res4_2_branch2b_bn_rm_0: Tensor[(256,), float32], %gpu_0/res4_2_branch2b_bn_riv_0: Tensor[(256,), float32], %gpu_0/res4_2_branch2c_w_0: Tensor[(1024, 256, 1, 1), float32], %gpu_0/res4_2_branch2c_bn_s_0: Tensor[(1024,), float32], %gpu_0/res4_2_branch2c_bn_b_0: Tensor[(1024,), float32], %gpu_0/res4_2_branch2c_bn_rm_0: Tensor[(1024,), float32], %gpu_0/res4_2_branch2c_bn_riv_0: Tensor[(1024,), float32], %gpu_0/res4_3_branch2a_w_0: Tensor[(256, 1024, 1, 1), float32], %gpu_0/res4_3_branch2a_bn_s_0: Tensor[(256,), float32], %gpu_0/res4_3_branch2a_bn_b_0: Tensor[(256,), float32], %gpu_0/res4_3_branch2a_bn_rm_0: Tensor[(256,), float32], %gpu_0/res4_3_branch2a_bn_riv_0: Tensor[(256,), float32], %gpu_0/res4_3_branch2b_w_0: Tensor[(256, 256, 3, 3), float32], %gpu_0/res4_3_branch2b_bn_s_0: Tensor[(256,), float32], %gpu_0/res4_3_branch2b_bn_b_0: Tensor[(256,), float32], %gpu_0/res4_3_branch2b_bn_rm_0: Tensor[(256,), float32], %gpu_0/res4_3_branch2b_bn_riv_0: Tensor[(256,), float32], %gpu_0/res4_3_branch2c_w_0: Tensor[(1024, 256, 1, 1), float32], %gpu_0/res4_3_branch2c_bn_s_0: Tensor[(1024,), float32], %gpu_0/res4_3_branch2c_bn_b_0: Tensor[(1024,), float32], %gpu_0/res4_3_branch2c_bn_rm_0: Tensor[(1024,), float32], %gpu_0/res4_3_branch2c_bn_riv_0: Tensor[(1024,), float32], %gpu_0/res4_4_branch2a_w_0: Tensor[(256, 1024, 1, 1), float32], %gpu_0/res4_4_branch2a_bn_s_0: Tensor[(256,), float32], %gpu_0/res4_4_branch2a_bn_b_0: Tensor[(256,), float32], %gpu_0/res4_4_branch2a_bn_rm_0: Tensor[(256,), float32], %gpu_0/res4_4_branch2a_bn_riv_0: Tensor[(256,), float32], %gpu_0/res4_4_branch2b_w_0: Tensor[(256, 256, 3, 3), float32], %gpu_0/res4_4_branch2b_bn_s_0: Tensor[(256,), float32], %gpu_0/res4_4_branch2b_bn_b_0: Tensor[(256,), float32], %gpu_0/res4_4_branch2b_bn_rm_0: Tensor[(256,), float32], %gpu_0/res4_4_branch2b_bn_riv_0: Tensor[(256,), float32], %gpu_0/res4_4_branch2c_w_0: Tensor[(1024, 256, 1, 1), float32], %gpu_0/res4_4_branch2c_bn_s_0: Tensor[(1024,), float32], %gpu_0/res4_4_branch2c_bn_b_0: Tensor[(1024,), float32], %gpu_0/res4_4_branch2c_bn_rm_0: Tensor[(1024,), float32], %gpu_0/res4_4_branch2c_bn_riv_0: Tensor[(1024,), float32], %gpu_0/res4_5_branch2a_w_0: Tensor[(256, 1024, 1, 1), float32], %gpu_0/res4_5_branch2a_bn_s_0: Tensor[(256,), float32], %gpu_0/res4_5_branch2a_bn_b_0: Tensor[(256,), float32], %gpu_0/res4_5_branch2a_bn_rm_0: Tensor[(256,), float32], %gpu_0/res4_5_branch2a_bn_riv_0: Tensor[(256,), float32], %gpu_0/res4_5_branch2b_w_0: Tensor[(256, 256, 3, 3), float32], %gpu_0/res4_5_branch2b_bn_s_0: Tensor[(256,), float32], %gpu_0/res4_5_branch2b_bn_b_0: Tensor[(256,), float32], %gpu_0/res4_5_branch2b_bn_rm_0: Tensor[(256,), float32], %gpu_0/res4_5_branch2b_bn_riv_0: Tensor[(256,), float32], %gpu_0/res4_5_branch2c_w_0: Tensor[(1024, 256, 1, 1), float32], %gpu_0/res4_5_branch2c_bn_s_0: Tensor[(1024,), float32], %gpu_0/res4_5_branch2c_bn_b_0: Tensor[(1024,), float32], %gpu_0/res4_5_branch2c_bn_rm_0: Tensor[(1024,), float32], %gpu_0/res4_5_branch2c_bn_riv_0: Tensor[(1024,), float32], %gpu_0/res5_0_branch2a_w_0: Tensor[(512, 1024, 1, 1), float32], %gpu_0/res5_0_branch2a_bn_s_0: Tensor[(512,), float32], %gpu_0/res5_0_branch2a_bn_b_0: Tensor[(512,), float32], %gpu_0/res5_0_branch2a_bn_rm_0: Tensor[(512,), float32], %gpu_0/res5_0_branch2a_bn_riv_0: Tensor[(512,), float32], %gpu_0/res5_0_branch2b_w_0: Tensor[(512, 512, 3, 3), float32], %gpu_0/res5_0_branch2b_bn_s_0: Tensor[(512,), float32], %gpu_0/res5_0_branch2b_bn_b_0: Tensor[(512,), float32], %gpu_0/res5_0_branch2b_bn_rm_0: Tensor[(512,), float32], %gpu_0/res5_0_branch2b_bn_riv_0: Tensor[(512,), float32], %gpu_0/res5_0_branch2c_w_0: Tensor[(2048, 512, 1, 1), float32], %gpu_0/res5_0_branch2c_bn_s_0: Tensor[(2048,), float32], %gpu_0/res5_0_branch2c_bn_b_0: Tensor[(2048,), float32], %gpu_0/res5_0_branch2c_bn_rm_0: Tensor[(2048,), float32], %gpu_0/res5_0_branch2c_bn_riv_0: Tensor[(2048,), float32], %gpu_0/res5_0_branch1_w_0: Tensor[(2048, 1024, 1, 1), float32], %gpu_0/res5_0_branch1_bn_s_0: Tensor[(2048,), float32], %gpu_0/res5_0_branch1_bn_b_0: Tensor[(2048,), float32], %gpu_0/res5_0_branch1_bn_rm_0: Tensor[(2048,), float32], %gpu_0/res5_0_branch1_bn_riv_0: Tensor[(2048,), float32], %gpu_0/res5_1_branch2a_w_0: Tensor[(512, 2048, 1, 1), float32], %gpu_0/res5_1_branch2a_bn_s_0: Tensor[(512,), float32], %gpu_0/res5_1_branch2a_bn_b_0: Tensor[(512,), float32], %gpu_0/res5_1_branch2a_bn_rm_0: Tensor[(512,), float32], %gpu_0/res5_1_branch2a_bn_riv_0: Tensor[(512,), float32], %gpu_0/res5_1_branch2b_w_0: Tensor[(512, 512, 3, 3), float32], %gpu_0/res5_1_branch2b_bn_s_0: Tensor[(512,), float32], %gpu_0/res5_1_branch2b_bn_b_0: Tensor[(512,), float32], %gpu_0/res5_1_branch2b_bn_rm_0: Tensor[(512,), float32], %gpu_0/res5_1_branch2b_bn_riv_0: Tensor[(512,), float32], %gpu_0/res5_1_branch2c_w_0: Tensor[(2048, 512, 1, 1), float32], %gpu_0/res5_1_branch2c_bn_s_0: Tensor[(2048,), float32], %gpu_0/res5_1_branch2c_bn_b_0: Tensor[(2048,), float32], %gpu_0/res5_1_branch2c_bn_rm_0: Tensor[(2048,), float32], %gpu_0/res5_1_branch2c_bn_riv_0: Tensor[(2048,), float32], %gpu_0/res5_2_branch2a_w_0: Tensor[(512, 2048, 1, 1), float32], %gpu_0/res5_2_branch2a_bn_s_0: Tensor[(512,), float32], %gpu_0/res5_2_branch2a_bn_b_0: Tensor[(512,), float32], %gpu_0/res5_2_branch2a_bn_rm_0: Tensor[(512,), float32], %gpu_0/res5_2_branch2a_bn_riv_0: Tensor[(512,), float32], %gpu_0/res5_2_branch2b_w_0: Tensor[(512, 512, 3, 3), float32], %gpu_0/res5_2_branch2b_bn_s_0: Tensor[(512,), float32], %gpu_0/res5_2_branch2b_bn_b_0: Tensor[(512,), float32], %gpu_0/res5_2_branch2b_bn_rm_0: Tensor[(512,), float32], %gpu_0/res5_2_branch2b_bn_riv_0: Tensor[(512,), float32], %gpu_0/res5_2_branch2c_w_0: Tensor[(2048, 512, 1, 1), float32], %gpu_0/res5_2_branch2c_bn_s_0: Tensor[(2048,), float32], %gpu_0/res5_2_branch2c_bn_b_0: Tensor[(2048,), float32], %gpu_0/res5_2_branch2c_bn_rm_0: Tensor[(2048,), float32], %gpu_0/res5_2_branch2c_bn_riv_0: Tensor[(2048,), float32], %gpu_0/pred_w_0: Tensor[(1000, 2048), float32], %gpu_0/pred_b_0: Tensor[(1000,), float32]) {
  %0 = nn.conv2d(%gpu_0/data_0, %gpu_0/conv1_w_0, strides=[2, 2], padding=[3, 3], kernel_size=[7, 7])an internal invariant was violated while typechecking your program [21:01:01] /Users/ligeng/Workspace/tvm/src/relay/op/nn/convolution.cc:107: Check failed: reporter->AssertEQ(dshape_nchw[1] / param->groups, wshape[1]): 
; 
  %1 = nn.batch_norm(%0, %gpu_0/res_conv1_bn_s_0, %gpu_0/res_conv1_bn_b_0, %gpu_0/res_conv1_bn_rm_0, %gpu_0/res_conv1_bn_riv_0, epsilon=1e-05)
  %2 = %1.0
  %3 = nn.relu(%2)
  %4 = nn.max_pool2d(%3, pool_size=[3, 3], strides=[2, 2], padding=[1, 1])an internal invariant was violated while typechecking your program [21:01:01] /Users/ligeng/Workspace/tvm/src/relay/op/nn/pooling.cc:73: Check failed: data != nullptr: 
...... # > 200 lines
......
......
 %230 = multiply(1f, %gpu_0/pred_b_0)
  %231 = nn.bias_add(%229, %230)
  nn.softmax(%231, axis=1)
}

Even with errors highlighted with color, this is kinda too long to read. I would prefer error message like

An error occurs when executing the following layer 
    %0 = nn.conv2d(%gpu_0/data_0, %gpu_0/conv1_w_0, strides=[2, 2], padding=[3, 3], kernel_size=[7, 7])
INFO: an internal invariant was violated while typechecking your program [21:01:01] /Users/ligeng/Workspace/tvm/src/relay/op/nn/convolution.cc:107: Check failed: reporter->AssertEQ(dshape_nchw[1] / param->groups, wshape[1]): ;

It makes locating the problem easier and avoids potential ambiguity (pooling layer in my case).

#10

This sounds good, can you propose a possible change to enable the new kind of error message?