TVMError after Loading Model from File


When I convert my ONNX model to TVM, I can run the TVM model with data successfully. However, once I save the TVM onto file and load it, I cannot successfully run it on data.
Rather, I get the following error:

TVMError: Check failed: in_idx >= 0 (-1 vs. 0) :

input_data = tvm.nd.array(x.astype(dtype))

Here is all my code below:

# Load ONNX model and input data from file
onnx_model = onnx.load('onnx_model.onnx')
x = np.loadtxt('input_data.txt', dtype = 'float32')[0][np.newaxis, :]  # First case

# Declare variables
target = 'llvm'
ctx = tvm.cpu(0)
input_name = 'input_1'
input_shape = {input_name: x.shape}  # x.shape = (1, 48)

# Compile the model with relay
mod, params = relay.frontend.from_onnx(onnx_model, input_shape)
with relay.build_config(opt_level = 1):
    intrp = relay.build_module.create_executor('graph', mod, ctx, target)

# Execute on TVM
dtype = 'float32'
tvm_output = intrp.evaluate()(tvm.nd.array(x.astype(dtype)), **params).asnumpy()
### This above line successfully executes the TVM model ###

# Save the TVM model 
mod, params = relay.frontend.from_onnx(onnx_model, input_shape)
func = mod[mod.entry_func]
with relay.build_config(opt_level = 1):
    graph, lib, params =, target, params = params)

directory = 'Saved ONNX to TVM/'
if not os.path.exists(directory):

modLib = directory + ''  # file names
modJSON = directory + 'onnx_test.json'
modParams = directory + 'onnx_test.params'

lib.export_library(modLib)  # export to files
with open(modJSON, "w") as fo:
with open(modParams, "wb") as fo:

# Load TVM model from file
loaded_json = open(modJSON).read()
loaded_lib = tvm.module.load(modLib)
loaded_params = bytearray(open(modParams, "rb").read())

module = graph_runtime.create(loaded_json, loaded_lib, ctx)

input_data = tvm.nd.array(x.astype(dtype))
### This above line does NOT successfully execute the TVM Model ###

Am I formatting it wrong?


I still have not figured out a robust way to load from file. Does anyone have any ideas?


You may want to check what input name your model expects. It might not be “data” but “x” or something else.