Stuck at how to use the nnvm model [ deploy, params, graph ] files once they have been saved in the temp folder as library files using python code. How to use them successfully using the C++ runtime code ?
For now, need to study how the make file functions for the C development of executing a file
The python code snippet [from s2s_lstm example of TVM/nnvm/tutorials/nlp directory] :
# Compile both encoder and decoder model on NNVM
# ----------------------------------------------
# Creates NNVM graph definition from keras model file.
from tvm.contrib import graph_runtime
target = 'llvm'
ctx = tvm.cpu(0)
# Parse Encoder model
sym, params = nnvm.frontend.from_keras(encoder_model)
inp_enc_shape = (1, max_encoder_seq_length, num_encoder_tokens)
shape_dict = {'input_1': inp_enc_shape}
# Build Encoder model
with nnvm.compiler.build_config(opt_level=2):
enc_graph, enc_lib, enc_params = nnvm.compiler.build(sym, target, shape_dict, params=params)
print("Encoder build ok.")
# Create graph runtime for encoder model
tvm_enc = graph_runtime.create(enc_graph, enc_lib, ctx)
tvm_enc.set_input(**enc_params)
# Parse Decoder model
inp_dec_shape = (1, 1, num_decoder_tokens)
shape_dict = {'input_2': inp_dec_shape,
'input_3': (1, latent_dim),
'input_4': (1, latent_dim)}
# Build Decoder model
sym, params = nnvm.frontend.from_keras(decoder_model)
with nnvm.compiler.build_config(opt_level=2):
dec_graph, dec_lib, dec_params = nnvm.compiler.build(sym, target, shape_dict, params=params)
print("Decoder build ok.")
# Create graph runtime for decoder model
tvm_dec = graph_runtime.create(dec_graph, dec_lib, ctx)
tvm_dec.set_input(**dec_params)
# Decodes an input sequence.
def decode_sequence(input_seq):
# Set the input for encoder model.
tvm_enc.set_input('input_1', input_seq)
# Run encoder model
tvm_enc.run()
This is basically loading a nnvm.frontend model and then building it and running the model.
I added code for saving the graphs in a folder to be used later for the C++ runtime that is given in this TVM example : https://github.com/dmlc/tvm/tree/master/apps/howto_deployhttps://github.com/dmlc/tvm/tree/master/apps/howto_deploy
But it does not seem to be compatible with the C++ runtime libraries as it produces a library file error. This only happens with graphs generated using nnvm.compiler.build and not for those which are generated using relay.build
How can I run the nnvm generated graphs with c++ runtime ?
Is there is a way to convert the nnvm graphs into relay.build models , that would just solve these issues once and for all.
Thanks for patience!