[External codegen] DNNL error

Troubleshooting
#1

@comaniac @zhiics

I’m trying new TVM codegen feature for DNNL.

I’m loading my model from MXNet frontend, but adding:

mod['main'] = set_external_func_attr(mod['main'], "dnnl", "dnnl_0")

as inspired by tests/python/relay/test_external_codegen.py, doesn’t seem to do the trick. Indeed I get the following error:

Traceback (most recent call last):

  File "../scripts/tvm_compile_relay.py", line 88, in <module>
    graph, lib, params = relay.build_module.build(mod, target=target, params=param)

  File "/tvm/incubator-tvm/python/tvm/relay/build_module.py", line 244, in build
    graph_json, mod, params = bld_mod.build(func, target, target_host, params)

  File "/tvm/incubator-tvm/python/tvm/relay/build_module.py", line 109, in build
    self._build(func, target, target_host)

  File "/tvm/incubator-tvm/python/tvm/_ffi/_ctypes/function.py", line 207, in __call__
    raise get_last_ffi_error()

tvm._ffi.base.TVMError: Traceback (most recent call last):
  [bt] (8) /tvm/incubator-tvm/build/libtvm.so(tvm::relay::backend::RelayBuildModule::GetFunction(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::{lambda(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)#3}::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const+0x1b5) [0x7fab8ee39365]
  [bt] (7) /tvm/incubator-tvm/build/libtvm.so(tvm::relay::backend::RelayBuildModule::BuildRelay(tvm::relay::Function, std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, tvm::runtime::NDArray, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, tvm::runtime::NDArray> > > const&)+0x60b) [0x7fab8ee3873b]
  [bt] (6) /tvm/incubator-tvm/build/libtvm.so(std::_Function_handler<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*), tvm::relay::backend::GraphRuntimeCodegenModule::GetFunction(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::{lambda(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)#2}>::_M_invoke(std::_Any_data const&, tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&)+0x8c) [0x7fab8ee2d62c]
  [bt] (5) /tvm/incubator-tvm/build/libtvm.so(tvm::relay::backend::GraphRuntimeCodegen::Codegen(tvm::relay::Function)+0x626) [0x7fab8ee2c406]
  [bt] (4) /tvm/incubator-tvm/build/libtvm.so(tvm::relay::backend::GraphRuntimeCodegen::VisitExpr(tvm::relay::Expr const&)+0x575) [0x7fab8ee22d55]
  [bt] (3) /tvm/incubator-tvm/build/libtvm.so(tvm::relay::backend::GraphRuntimeCodegen::VisitExpr_(tvm::relay::TupleGetItemNode const*)+0x29) [0x7fab8ee179e9]
  [bt] (2) /tvm/incubator-tvm/build/libtvm.so(tvm::relay::backend::GraphRuntimeCodegen::VisitExpr(tvm::relay::Expr const&)+0x42d) [0x7fab8ee22c0d]
  [bt] (1) /tvm/incubator-tvm/build/libtvm.so(tvm::relay::backend::GraphRuntimeCodegen::VisitExpr_(tvm::relay::CallNode const*)+0xc2) [0x7fab8ee281f2]
  [bt] (0) /tvm/incubator-tvm/build/libtvm.so(dmlc::LogMessageFatal::~LogMessageFatal()+0x32) [0x7fab8e7b2a22]
  File "/tvm/incubator-tvm/src/relay/backend/graph_runtime_codegen.cc", line 409
TVMError: Operators should be transformed away; try applyingthe fuse_ops transformation to the expression.

Am I missing something else?

#2

As the error message indicates, the program is not properly transformed

#3

This is the code though, I only added

mod['main'] = set_external_func_attr(mod['main'], "dnnl", "dnnl_0")

to the standard script.

import tvm.relay as relay
import mxnet as mx

prefix, epoch = "model", 0
sym, arg_params, aux_params = mx.model.load_checkpoint(prefix, epoch)

img_size = [112, 112]
input_shape = (1, 3, img_size[0], img_size[1])
shape_dict = {'data': input_shape}
mod, param = relay.frontend.from_mxnet(sym, shape_dict, arg_params=arg_params, aux_params=aux_params)

opt_level = 3
postfix = prefix + '_opt'

shape_dict = {'data': input_shape}
target = tvm.target.create("llvm -mcpu=broadwell")


def set_external_func_attr(func, compiler, ext_symbol):
    func = func.set_attribute("Primitive", tvm.expr.IntImm("int32", 1))
    func = func.set_attribute("Compiler", tvm.expr.StringImm(compiler))
    func = func.set_attribute("ExternalSymbol", tvm.expr.StringImm(ext_symbol))
    return func


mod['main'] = set_external_func_attr(mod['main'], "dnnl", "dnnl_0")

print('Compiling...')
with relay.build_config(opt_level=opt_level):
    graph, lib, params = relay.build_module.build(mod, target=target, params=param)

lib.export_library("./" + postfix + ".so")
print('lib export successfully')
with open("./" + postfix + ".json", "w") as fo:
    fo.write(graph)
with open("./" + postfix + ".params", "wb") as fo:
    fo.write(relay.save_param_dict(params))
#4

Which model is it? This feature is still under active development. We tested whole graph annotation in https://github.com/apache/incubator-tvm/pull/4570

#5

Thanks @zhiics

I’ve just pulled https://github.com/apache/incubator-tvm/pull/4570 and used

mod = relay.build_extern_compiler(mod, "dnnl")

now I only get a bunch of warnings related to ops not registered in dnnl, but that’s fine.

Still I get an error when calling relay.build though. The following:

Traceback (most recent call last):

  File "../scripts/tvm_compile_relay.py", line 89, in <module>
    graph, lib, params = relay.build_module.build(mod, target=target, params=param)

  File "/tvm/incubator-tvm/python/tvm/relay/build_module.py", line 245, in build
    graph_json, mod, params = bld_mod.build(func, target, target_host, params)

  File "/tvm/incubator-tvm/python/tvm/relay/build_module.py", line 110, in build
    self._build(func, target, target_host)

  File "/tvm/incubator-tvm/python/tvm/_ffi/_ctypes/function.py", line 207, in __call__
    raise get_last_ffi_error()

tvm._ffi.base.TVMError: Traceback (most recent call last):
  [bt] (7) /tvm/incubator-tvm/build/libtvm.so(TVMFuncCall+0x61) [0x7f60b6d46281]
  [bt] (6) /tvm/incubator-tvm/build/libtvm.so(std::_Function_handler<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*), tvm::relay::backend::RelayBuildModule::GetFunction(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::{lambda(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)#3}>::_M_invoke(std::_Any_data const&, tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&)+0x17) [0x7f60b6ba9037]
  [bt] (5) /tvm/incubator-tvm/build/libtvm.so(tvm::relay::backend::RelayBuildModule::GetFunction(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::{lambda(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)#3}::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const+0x1b5) [0x7f60b6ba8f45]
  [bt] (4) /tvm/incubator-tvm/build/libtvm.so(tvm::relay::backend::RelayBuildModule::BuildRelay(tvm::relay::Function, std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, tvm::runtime::NDArray, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, tvm::runtime::NDArray> > > const&)+0x60b) [0x7f60b6ba831b]
  [bt] (3) /tvm/incubator-tvm/build/libtvm.so(std::_Function_handler<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*), tvm::relay::backend::GraphRuntimeCodegenModule::GetFunction(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::{lambda(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)#2}>::_M_invoke(std::_Any_data const&, tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&)+0x8c) [0x7f60b6b9d20c]
  [bt] (2) /tvm/incubator-tvm/build/libtvm.so(tvm::relay::backend::GraphRuntimeCodegen::Codegen(tvm::relay::Function)+0xf26) [0x7f60b6b9c8e6]
  [bt] (1) /tvm/incubator-tvm/build/libtvm.so(tvm::relay::CompileEngineImpl::LowerExternalFunctions()+0x798) [0x7f60b6bb2b88]
  [bt] (0) /tvm/incubator-tvm/build/libtvm.so(dmlc::LogMessageFatal::~LogMessageFatal()+0x32) [0x7f60b651aaa2]
  File "/tvm/incubator-tvm/src/relay/backend/compile_engine.cc", line 638
TVMError: Check failed: pf: Failed to find the codegen tool for relay.ext.dnnl

P.S. the model I’m testing is here