Relay quantization not working as previously

Hi My code was consuming models from frontends mxnet, onnx, coreML and the quantizing the model, using relay.quantize function. But it seems to not work now and all the codes are giving a consistent error at the same line where relay.quantize.quantize function is called and seems ‘parameters’ are not being able to be read by this function :

  **File "from_coreml_deploy_graph.py", line 49, in <module>**
**    func = relay.quantize.quantize(func, params)**

  File "/home/raj/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/relay/quantize/quantize.py", line 432, in quantize
    graph = prerequisite_optimize(graph, params)

  File "/home/raj/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/relay/quantize/quantize.py", line 401, in prerequisite_optimize
    graph = _bind_params(graph, params)

  File "/home/raj/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/relay/quantize/quantize.py", line 373, in _bind_params
    for arg in func.params:

  File "tvm/_ffi/_cython/./node.pxi", line 81, in tvm._ffi._cy3.core.NodeBase.__getattr__

**AttributeError: '<class 'tvm.relay.module.Module'>' object has no attribute 'params'**

Previously, I had successfully got results and without errors with the code. This is just an edited from_mxnet.py file , including the quantization part. I have attached the code below :

import mxnet as mx
import tvm
import tvm.relay as relay
import numpy as np

######################################################################
# Download Resnet18 model from Gluon Model Zoo
# ---------------------------------------------
# In this section, we download a pretrained imagenet model and classify an image.
from tvm.contrib.download import download_testdata
from mxnet.gluon.model_zoo.vision import get_model
from PIL import Image
from matplotlib import pyplot as plt
block = get_model('resnet18_v1', pretrained=True)
img_url = 'https://github.com/dmlc/mxnet.js/blob/master/data/cat.png?raw=true'
img_name = 'cat.png'
synset_url = ''.join(['https://gist.githubusercontent.com/zhreshold/',
                      '4d0b62f3d01426887599d4f7ede23ee5/raw/',
                      '596b27d23537e5a1b5751d2b0481ef172f58b539/',
                      'imagenet1000_clsid_to_human.txt'])
synset_name = 'imagenet1000_clsid_to_human.txt'
img_path = download_testdata(img_url, 'cat.png', module='data')
synset_path = download_testdata(synset_url, synset_name, module='data')
with open(synset_path) as f:
    synset = eval(f.read())
image = Image.open(img_path).resize((224, 224))
plt.imshow(image)
plt.show()

def transform_image(image):
    image = np.array(image) - np.array([123., 117., 104.])
    image /= np.array([58.395, 57.12, 57.375])
    image = image.transpose((2, 0, 1))
    image = image[np.newaxis, :]
    return image

x = transform_image(image)
print('x', x.shape)

######################################################################
# Compile the Graph
# -----------------
# Now we would like to port the Gluon model to a portable computational graph.
# It's as easy as several lines.
# We support MXNet static graph(symbol) and HybridBlock in mxnet.gluon
shape_dict = {'data': x.shape}
func, params = relay.frontend.from_mxnet(block, shape_dict)


## we want a probability so add a softmax operator
func = relay.Function(func.params, relay.nn.softmax(func.body), None, func.type_params, func.attrs)

# Quantize the final graph
with relay.quantize.qconfig(skip_k_conv=0, round_for_shift=True):
    func = relay.quantize.quantize(func, params)

######################################################################
# now compile the graph
target = 'llvm'
with relay.build_config(opt_level=3):
    graph, lib, params = relay.build(func, target, params=params)

######################################################################
# Execute the portable graph on TVM
# ---------------------------------
# Now, we would like to reproduce the same forward computation using TVM.
from tvm.contrib import graph_runtime
ctx = tvm.cpu(0)
dtype = 'float32'
m = graph_runtime.create(graph, lib, ctx)
# set inputs
m.set_input('data', tvm.nd.array(x.astype(dtype)))
m.set_input(**params)
# execute
m.run()
# get outputs
tvm_output = m.get_output(0)
top1 = np.argmax(tvm_output.asnumpy()[0])
print('TVM prediction top-1:', top1, synset[top1])

######################################################################
# Use MXNet symbol with pretrained weights
# ----------------------------------------
# MXNet often use `arg_params` and `aux_params` to store network parameters
# separately, here we show how to use these weights with existing API
def block2symbol(block):
    data = mx.sym.Variable('data')
    sym = block(data)
    args = {}
    auxs = {}
    for k, v in block.collect_params().items():
        args[k] = mx.nd.array(v.data().asnumpy())
    return sym, args, auxs
mx_sym, args, auxs = block2symbol(block)
# usually we would save/load it as checkpoint
mx.model.save_checkpoint('resnet18_v1', 0, mx_sym, args, auxs)
# there are 'resnet18_v1-0000.params' and 'resnet18_v1-symbol.json' on disk

######################################################################
# for a normal mxnet model, we start from here
mx_sym, args, auxs = mx.model.load_checkpoint('resnet18_v1', 0)
# now we use the same API to get Relay computation graph
relay_func, relay_params = relay.frontend.from_mxnet(mx_sym, shape_dict,
                                                     arg_params=args, aux_params=auxs)
# repeat the same steps to run this model using TVM

Could anyone please let me know what should I do to fix this ?

@vinx13 Maybe you know this already?

We have moved to module to module style api, so you need to pass a module to relay.quantize.quantize.

Thanks for the help @vinx13 and @janimesh !

I solved it by using :

##Quantize model code

mod['main'] = relay.quantize.quantize(mod['main'], params)

##build graph

graph, lib, params = relay.build(mod, target=target) 

for all the graphs , and it worked! Also took some hint from this post: https://discuss.tvm.ai/t/error-quantizing-a-tensorflow-model-module-vs-function-changes/3295?u=rajshekharm

1 Like