"Terminated" while running NNVM network

Questions
#1

Hi, I’m running the following code (modified from tune_nnvm_x86.py) and getting a “Terminated” error when the num_trials is set to 100 or more. It works well when num_trials is small, I’m assuming some default timeout is set for running module.time_evaluator() but I can’t seem to find where the error is raised.

net, params, data_shape, out_shape = get_network(model_name, batch_size)

with nnvm.compiler.build_config(opt_level=3):
    graph, lib, params = nnvm.compiler.build(net, target=target, shape={'data': data_shape}, params=params, dtype=dtype)

    ctx = tvm.cpu()
    data_tvm = tvm.nd.array((np.random.uniform(size=data_shape)).astype(dtype))
    module = runtime.create(graph, lib, ctx)
    module.set_input('data', data_tvm)
    module.set_input(**params)

    num_trials = 100
    ftimer = module.module.time_evaluator("run", ctx, number=1, repeat=num_trials)
    prof_res = np.array(ftimer().results)
    avg_time = np.mean(prof_res)
    print("time                 = {0:.3f} ms".format(avg_time*1000))
#2

Can you provide some trace stack message?

#3

Here’s all the output I get and there’s no stack trace as this is not raising an exception. I think the first few lines of the output are a result of one of the imports I’m using (the warning is defined in python/tvm/autotvm/task/dispatcher.py and does not relate to “Terminate”).

Cannot find config for target=llvm, workload=('conv2d', (64, 3, 224, 224, 'float32'), (64, 3, 3, 3, 'float32'), (1, 1), (1, 1), (1, 1), 'NCHW', 'float32'). A fallback configuration is used, which may bring great performance regression.
Cannot find config for target=llvm, workload=('conv2d', (64, 64, 112, 112, 'float32'), (128, 64, 3, 3, 'float32'), (1, 1), (1, 1), (1, 1), 'NCHW', 'float32'). A fallback configuration is used, which may bring great performance regression.
Cannot find config for target=llvm, workload=('conv2d', (64, 128, 56, 56, 'float32'), (256, 128, 3, 3, 'float32'), (1, 1), (1, 1), (1, 1), 'NCHW', 'float32'). A fallback configuration is used, which may bring great performance regression.
Cannot find config for target=llvm, workload=('conv2d', (64, 256, 56, 56, 'float32'), (256, 256, 3, 3, 'float32'), (1, 1), (1, 1), (1, 1), 'NCHW', 'float32'). A fallback configuration is used, which may bring great performance regression.
Cannot find config for target=llvm, workload=('conv2d', (64, 256, 28, 28, 'float32'), (512, 256, 3, 3, 'float32'), (1, 1), (1, 1), (1, 1), 'NCHW', 'float32'). A fallback configuration is used, which may bring great performance regression.
Cannot find config for target=llvm, workload=('conv2d', (64, 512, 28, 28, 'float32'), (512, 512, 3, 3, 'float32'), (1, 1), (1, 1), (1, 1), 'NCHW', 'float32'). A fallback configuration is used, which may bring great performance regression.
Cannot find config for target=llvm, workload=('conv2d', (64, 512, 14, 14, 'float32'), (512, 512, 3, 3, 'float32'), (1, 1), (1, 1), (1, 1), 'NCHW', 'float32'). A fallback configuration is used, which may bring great performance regression.
Terminated

grep -r "Terminated" ./tvm-home would show Binary file ./build/libtvm.so matches but I cannot find a corresponding source file.

Thanks,