[SOLVED] Allow_missing Device API gpu is not enabled


Hi, @srkreddy1238 @tqchen, I am not able to deploy Non cpu(Cuda, OpenCL) tvm compiled model in c++ statically i am facing below error.

terminate called after throwing an instance of ‘dmlc::Error’
what(): [11:13:16] …/…/src/runtime/c_runtime_api.cc:69: Check failed: allow_missing Device API gpu is not enabled.

even though i enabled Cuda and OpenCL support while building tvm, but i am able to deploy llvm tvm compiled model.


What’s your configuration for USE_CUDA?


@junrushao1994 You mean H/W configuration ?


To use cuda I believe you should turn on this line


Yes i built tvm after enabling Cuda, and OpenCL support only/


I went through your error message and found that the name is “gpu”, which caused this issue. Could you try changing “gpu” to “cuda” or “opencl”?


@srkreddy1238 @junrushao1994 is below DLContext are correct for Cuda?

constexpr int dtype_code = kDLFloat;
constexpr int dtype_bits = 32;
constexpr int dtype_lanes = 1;
constexpr int device_type = kDLGPU;
constexpr int device_id = 0;
constexpr int in_ndim = 4;  

Based on dlpack header file


Sorry i misread. From your error message i see that “device_api.gpu” is not registered, which should have been registered here, so may I confirm with you if the file is compiled and properly linked?


yes same working in python.


It’s weird…Could you print all elements in out_array provided by TVMFuncListGlobalNames?


int dtype_code = kDLFloat;
int dtype_bits = 32;
int dtype_lanes = 1;
int device_type = kDLCPU;
int device_id = 0;
int in_ndim = 4;

DLTensor* input;
TVMArrayAlloc(in_shape, in_ndim, dtype_code, dtype_bits, dtype_lanes, device_type, device_id, &input);
memset(input->data, 0,in_size);
when i do memset it is giving segmentation fault.

How to deploy tvm model for Cuda and OpenCL to create c++ API's

@junrushao1994 Any suggestions on above comments ?


Thank you @junrushao1994 for your response, issue is resoled by using TVMArrayCopyFromBytes().


Glad that you find a way to deal with the issue :slight_smile: