Constant params should be constants

I’ve noticed that relay.Var is always used for model weights, even when those weights are declared as constants in the framework that they were imported from.

This seems problematic to me because:

  1. We aren’t able to apply constant folding to the model weights. Any normalization, scale, reshape or transpose on weights has to be recomputed for each inference. These operations on weights are not uncommon and can also be introduced by relay passes such as ConvertLayout.
  2. For external codegen, many 3rd party inference libraries require weights for conv, BN, dense, etc to be constant C arrays.
  3. It is confusing. If an input is a variable, it should be a Var. If it is constant, it should be a Const.

I’m also curious what exactly happens when a Var is designated as a “param”? Is it possible to apply constant folding to params?

Is it simply a matter of updating the frontend parsers to use relay.Constant? Are there some other reasons why Var was chosen to be used?

Thank you!

4 Likes

@lhutton1 Is this related to your issue? [External Codegen] Constant tensors in c-codegen

Thanks, I think this question is subtly different, this is about why constant tensors aren’t represented as Const in relay. Whereas, my question relates to how to represent a Const value in codegen.

On a side note @trevor-m, you can achieve this with the BindParamsByName pass if you haven’t figured it out already. For example:

f = relay.build_module.bind_params_by_name(mod["main"], params)
mod = tvm.IRModule()
mod["main"] = f
3 Likes