[Solved] How to quantize the weights in parameters?

yongsun · July 10, 2019, 5:33pm

It looks like relay.quantize.quantize only return a quantized graph, but leave the parameter untouched?

net = relay.quantize.quantize(net, params=params)

How could we get the quantized weights? Thanks!

vinx13 · June 28, 2019, 3:29am

The quantized params will be bind to the quantized model. They will become constants in net

yongsun · July 2, 2019, 4:23pm

Thanks for replying. How could I persist the quantized params and load it later? Thanks!

vinx13 · July 3, 2019, 3:23am

github.com

dmlc/tvm/blob/master/python/tvm/relay/param_dict.py#L24


# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.
# pylint: disable=invalid-name
"""Helper utility to save parameter dicts."""
import tvm


_save_param_dict = tvm.get_global_func("tvm.relay._save_param_dict")
_load_param_dict = tvm.get_global_func("tvm.relay._load_param_dict")


def save_param_dict(params):
"""Save parameter dictionary to binary bytes.


The result binary bytes can be loaded by the
GraphModule with API "load_params".


Parameters
----------
params : dict of str to NDArray
    The parameter dictionary.

yongsun · July 3, 2019, 6:37pm

net = relay.quantize.quantize(net, params=params)

The problem is params was not touched, and still remained in float32 type.

vinx13 · July 4, 2019, 5:59am

graph, lib, params = relay.build(net)

Then you can obtain updated params

yongsun · July 5, 2019, 4:10pm

@vinx13 Oh, I see, I should not pass params in relay.build for a quantized model. Thanks so much

henry099 · April 13, 2020, 3:02am

@vinx13 Hi,How to get the quantized params before build?