TVMError after Loading Model from File


#1

When I convert my ONNX model to TVM, I can run the TVM model with data successfully. However, once I save the TVM onto file and load it, I cannot successfully run it on data.
Rather, I get the following error:

TVMError: Check failed: in_idx >= 0 (-1 vs. 0) :

input_data = tvm.nd.array(x.astype(dtype))

Here is all my code below:

# Load ONNX model and input data from file
onnx_model = onnx.load('onnx_model.onnx')
x = np.loadtxt('input_data.txt', dtype = 'float32')[0][np.newaxis, :]  # First case

# Declare variables
target = 'llvm'
ctx = tvm.cpu(0)
input_name = 'input_1'
input_shape = {input_name: x.shape}  # x.shape = (1, 48)

# Compile the model with relay
mod, params = relay.frontend.from_onnx(onnx_model, input_shape)
with relay.build_config(opt_level = 1):
    intrp = relay.build_module.create_executor('graph', mod, ctx, target)

# Execute on TVM
dtype = 'float32'
tvm_output = intrp.evaluate()(tvm.nd.array(x.astype(dtype)), **params).asnumpy()
### This above line successfully executes the TVM model ###

# Save the TVM model 
mod, params = relay.frontend.from_onnx(onnx_model, input_shape)
func = mod[mod.entry_func]
with relay.build_config(opt_level = 1):
    graph, lib, params = relay.build_module.build(func, target, params = params)

directory = 'Saved ONNX to TVM/'
if not os.path.exists(directory):
    os.makedirs(directory)

modLib = directory + 'onnx_test.so'  # file names
modJSON = directory + 'onnx_test.json'
modParams = directory + 'onnx_test.params'

lib.export_library(modLib)  # export to files
with open(modJSON, "w") as fo:
    fo.write(graph)
with open(modParams, "wb") as fo:
    fo.write(relay.save_param_dict(params))

# Load TVM model from file
loaded_json = open(modJSON).read()
loaded_lib = tvm.module.load(modLib)
loaded_params = bytearray(open(modParams, "rb").read())

module = graph_runtime.create(loaded_json, loaded_lib, ctx)
module.load_params(loaded_params)

input_data = tvm.nd.array(x.astype(dtype))
### This above line does NOT successfully execute the TVM Model ###
module.run(data=input_data)

Am I formatting it wrong?


#2

I still have not figured out a robust way to load from file. Does anyone have any ideas?


#3

You may want to check what input name your model expects. It might not be “data” but “x” or something else.