ONNX model compilation fails with a model that previously worked

Yesterday I pulled the latest code and installed from source, which went without problems. After that, I tried to compile an MNIST onnx model as first test, which failed with the stacktrace below:

The model can be downloaded here: https://we.tl/t-Tghn9o9EQ8 (md5: 9fc8b23aa4f33008360727d2fe1b0823)

  File "/home/martin/Dev/xyz/src/tvm/compile_model.py", line 90, in compile_model
    graph_json, lib, params = relay.build_module.build(func=relay_func, target=target, params=params)
  File "/home/martin/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/relay/build_module.py", line 276, in build
    func = optimize(func, target, params)
  File "/home/martin/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/relay/build_module.py", line 203, in optimize
    func = ir_pass.alter_op_layout(func)
  File "/home/martin/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/relay/ir_pass.py", line 741, in alter_op_layout
    return _ir_pass.AlterOpLayout(expr)
  File "/home/martin/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/_ffi/_ctypes/function.py", line 190, in __call__
    raise get_last_ffi_error()
tvm._ffi.base.TVMError: Traceback (most recent call last):
  [bt] (8) /home/martin/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(+0xd1c4bd) [0x7fec5b8d04bd]
  [bt] (7) /home/martin/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(+0xba5e0d) [0x7fec5b759e0d]
  [bt] (6) /home/martin/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(+0xba8867) [0x7fec5b75c867]
  [bt] (5) /home/martin/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(+0xba7734) [0x7fec5b75b734]
  [bt] (4) /home/martin/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(+0xd1d4f5) [0x7fec5b8d14f5]
  [bt] (3) /home/martin/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(+0xcf8770) [0x7fec5b8ac770]
  [bt] (2) /home/martin/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(+0xcf3d98) [0x7fec5b8a7d98]
  [bt] (1) /home/martin/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(+0xcf3000) [0x7fec5b8a7000]
  [bt] (0) /home/martin/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(+0xf2f5eb) [0x7fec5bae35eb]
  File "/home/martin/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/_ffi/_ctypes/function.py", line 55, in cfun
    rv = local_pyfunc(*pyargs)
  File "/home/martin/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/relay/op/nn/_nn.py", line 134, in alter_op_layout_conv2d
    return topi.nn.conv2d_alter_layout(attrs, inputs, tinfos, op)
  File "</home/martin/.local/lib/python3.6/site-packages/decorator-4.3.2-py3.6.egg/decorator.py:decorator-gen-18>", line 2, in conv2d_alter_layout
  File "/home/martin/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/target.py", line 356, in dispatch_func
    return dispatch_dict[k](*args, **kwargs)
  File "/home/martin/.local/lib/python3.6/site-packages/topi-0.6.dev0-py3.6.egg/topi/x86/conv2d.py", line 297, in _alter_conv2d_layout
    out_channel = attrs.get_int("channels") if F == sym else attrs.get_int("channels").value
AttributeError: 'NoneType' object has no attribute 'value'

This looks like someone introduced a bug or regression into the alter layout pass, could you open an issue against dmlc master so we can CC the appropriate people to work on this. You can try to turn
off the alter-layout optimization if you want to make progress.

Sure, how can I turn off the optimization? I didn’t actively enable it.

It should be disabled by default, it is set at optimization level 2, so I’m not sure why it is executing.

Can you try:

with relay.build_module.build_config(opt_level=2):
   graph_json, lib, params = relay.build_module.build(...)

Okay, I for some reason, there was opt_level=3 set here. I changed it to 2 and now it fails with:

  File "/home/martin/Dev/xyz/src/tvm/compile_model.py", line 112, in compile_model
    lib.export_library(lib_name)
  File "/home/martin/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/module.py", line 128, in export_library
    fcompile(file_name, files, **kwargs)
  File "/home/martin/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/contrib/cc.py", line 33, in create_shared
    _linux_shared(output, objects, options, cc)
  File "/home/martin/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/contrib/cc.py", line 90, in _linux_shared
    raise RuntimeError(msg)
RuntimeError: Compilation error:
/usr/bin/ld: /tmp/tmp_i8odkrm/lib.o: relocation R_X86_64_32S against `.rodata.cst4' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: nonrepresentable section on output
collect2: error: ld returned 1 exit status