Compatibility of NNVM graph with Relay for quantization of any graph


Is there a way I can quantize nnvm graphs using relay.quantize fuction ?

Seems the relay quanitzation function does not take the nnvm graphs as compatible input to be quantized.
This is the code snippet from seq2seq using LSTM example of TVM :

# Creates NNVM graph definition from keras model file.

from tvm.contrib import graph_runtime
target = 'llvm'
ctx = tvm.cpu(0)

# Parse Encoder model
sym, params = nnvm.frontend.from_keras(encoder_model)
inp_enc_shape = (1, max_encoder_seq_length, num_encoder_tokens)
shape_dict = {'input_1': inp_enc_shape}

# Quantize graph code
with relay.quantize.qconfig(skip_k_conv=0, round_for_shift=True):
    sym = relay.quantize.quantize(sym, params=params)

This gives the error :

File “/home/raj/tvm/tutorials/quantized graphs/”, line 174, in
sym, params = relay.quantize.quantize(sym, params)
File “/home/raj/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/relay/quantize/”, line 372, in quantize
graph = optimize(graph, params)
File “/home/raj/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/relay/quantize/”, line 308, in optimize
for arg in func.params:
AttributeError: ‘Symbol’ object has no attribute ‘params’

The code is using an nnvm.frontend graph and then trying to quantize it using relay.quantize
But it seems ‘params’ is not a compatible/passed as argument in ‘Symbol’.

Is it possible to quantize nnvm graph through some other function ?
I searched the nnvm library , but could not find any such definition.

Or is it possible to convert the nnvm graph into a relay graph and then convert back to nnvm ?

Thanks for patience !


NNVM and Relay graph are not compatible. You can convert nnvm graph to relay with to_relay but it’s not possible to convert it back


Thanks, to_relay is useful and I am trying to deploy various mdoels with this , seems not all models may be possible to convert (some are producing ’ string / dataframe not matching’ errors when converting nnvm graph to relay format)