C++ deploy example: switch to OpenCL

ruslo · December 19, 2018, 5:19pm

Hi,

I’m trying to switch from llvm --system-lib to opencl --system-lib in C++ deploy example:

https://github.com/dmlc/tvm/tree/master/apps/howto_deploy

However I can’t load function, assertion failed here:

  // Get the function from the module.
  tvm::runtime::PackedFunc f = mod.GetFunction(fname);
  CHECK(f != nullptr);

With llvm --system-lib example works fine. With opencl (without --system-lib) this assertion passed but I got another error.

File prepare_test_libs.py: https://gist.github.com/ruslo/95f275de2b8effde94e22b5f02f03a31
File cpp_deploy.cc: https://gist.github.com/ruslo/c7eb1f2afc2acfc3d8297dbe5da3c4b3

I wonder if I’m doing something that is not possible by design. Or is it a error?

Thanks

eqy · December 19, 2018, 10:08pm

Did you build TVM with OpenCL enabled? https://github.com/dmlc/tvm/blob/001ab52509ce1f43fcbdad4d11c1ef2bcad04e10/cmake/config.cmake#L49

ruslo · December 19, 2018, 10:44pm

Yes, -DUSE_OPENCL=ON was used while configuring tvm.

eqy · December 20, 2018, 12:07am

What is the error when you changed the target to “opencl”?

ruslo · December 20, 2018, 12:54am

Error:

[...] /.../cpp_deploy.cc:63: Verify dynamic loading from test_addone.so
terminate called after throwing an instance of 'dmlc::Error'
  what():  [...] /dl/mxnet/3rdparty/tvm/src/runtime/module_util.cc:53: Check failed: ret == 0 (-1 vs. 0)
           [...] /dl/mxnet/3rdparty/tvm/src/runtime/opencl/opencl_module.cc:63: Check failed: e == CL_SUCCESS OpenCL Error, code=-38: CL_INVALID_MEM_OBJECT

Stack trace returned 10 entries:
[bt] (0) /.../_builds/cpp_deploy(dmlc::StackTrace[abi:cxx11]()+0x54) [0x55d5ccb0c274]
[bt] (1) /.../_builds/cpp_deploy(dmlc::LogMessageFatal::~LogMessageFatal()+0x40) [0x55d5ccb0c570]
[bt] (2) /dl/mxnet/3rdparty/tvm/_builds/libtvm_runtime.so(tvm::runtime::OpenCLWrappedFunc::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*, void**) const+0x5b0) [0x7f3c9c6a6a40]
[bt] (3) /dl/mxnet/3rdparty/tvm/_builds/libtvm_runtime.so(std::_Function_handler<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*), tvm::runtime::detail::PackFuncVoidAddr_<4, tvm::runtime::OpenCLWrappedFunc>(tvm::runtime::OpenCLWrappedFunc, std::vector<tvm::runtime::detail::ArgConvertCode, std::allocator<tvm::runtime::detail::ArgConvertCode> > const&)::{lambda(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)#1}>::_M_invoke(std::_Any_data const&, tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&)+0xb6) [0x7f3c9c6a6ca6]
[bt] (4) /dl/mxnet/3rdparty/tvm/_builds/libtvm_runtime.so(TVMFuncCall+0x5e) [0x7f3c9c5fe8ce]
[bt] (5) /.../lib/test_addone.so(+0xa30) [0x7f3c998b2a30]
[bt] (6) /.../lib/test_addone.so(addone+0x19a) [0x7f3c998b289a]
[bt] (7) /dl/mxnet/3rdparty/tvm/_builds/libtvm_runtime.so(+0x51550) [0x7f3c9c625550]
[bt] (8) /.../_builds/cpp_deploy(std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const+0x5a) [0x55d5ccb0cf2e]
[bt] (9) /.../_builds/cpp_deploy(tvm::runtime::TVMRetValue tvm::runtime::PackedFunc::operator()<DLTensor*&, DLTensor*&>(DLTensor*&, DLTensor*&) const+0xc4) [0x55d5ccb0d1aa]

Stack trace returned 8 entries:
[bt] (0) /.../_builds/cpp_deploy(dmlc::StackTrace[abi:cxx11]()+0x54) [0x55d5ccb0c274]
[bt] (1) /dl/mxnet/3rdparty/tvm/_builds/libtvm_runtime.so(+0x51ca7) [0x7f3c9c625ca7]
[bt] (2) /.../_builds/cpp_deploy(std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const+0x5a) [0x55d5ccb0cf2e]
[bt] (3) /.../_builds/cpp_deploy(tvm::runtime::TVMRetValue tvm::runtime::PackedFunc::operator()<DLTensor*&, DLTensor*&>(DLTensor*&, DLTensor*&) const+0xc4) [0x55d5ccb0d1aa]
[bt] (4) /.../_builds/cpp_deploy(Verify(tvm::runtime::Module, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)+0x1f8) [0x55d5ccb0b603]
[bt] (5) /.../_builds/cpp_deploy(main+0x13f) [0x55d5ccb0b93f]
[bt] (6) /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7) [0x7f3c9bc63b97]
[bt] (7) /.../_builds/cpp_deploy(_start+0x2a) [0x55d5ccb0b20a]

Used prepare_test_libs.py: https://gist.github.com/ruslo/4fa15e70910abd371515a5c4890950af
Used cpp_deploy.cc: https://gist.github.com/ruslo/eaee9838b9eeb0af342c1fb3061ffc55

eqy · December 20, 2018, 1:08am

I do not have detailed knowledge of this, but I suspect that there may be an issue directly reading from an array in an OpenCL context. Try copying the data back to a CPU array first before doing the check: https://github.com/dmlc/tvm/blob/838e7181dd5e16aa5c7a87bbfbf11b5b42a80f1a/src/runtime/ndarray.cc#L231

ruslo · December 23, 2018, 2:23pm

Yes, indeed, need to copy data from CPU to GPU and back.

Hint:

Deploy NNVM module using C++ on GPU using OpenCL target

Working example:

So as a summary: target=opencl + dynamic linking is working fine, but target=opencl --system-lib + static linking is not working because we can’t load function from module.