I’m trying to load InceptionV3 via Keras and using relay to convert the model, I receive a TypeError: batch_norm() missing 1 required positional argument: 'gamma' error.
After looking through the keras layer a bit, it seems that scale is not set in the layer, and is being handled in the layer.
Could gamma be an optional param in underlying cpp function?
Anyone got any ideas on this? I’m really hoping to get inception working from Keras.
Would that be working if we provide a default gamma value to batch_norm?
Potentially. Seems like it’s just multiplying by gamma if scale is True. Probably can default gamma to 1 if scale is False?
What I have in mind is if we can set the default to the gamma argument so that you don’t need to specify its value when importing from Keras. You could make that change and file a PR if so, since it won’t affect any of existing workloads.
I see your point.
Would it make sense to put this under the op.nn function or specifically for keras’ _convert_batchnorm function?
The functionality is the same, with the default gamma being 1 (since it then becomes an unused constant). Happy to submit a PR for either.
I prefer to do it in relay.op.nn since the similar issue may happen in other frontends in the future.
Ok :). I’ll make a PR. Thanks!
Actually. Having a look at the conversion function, the expression table is being used to generate new free variables from constants. The values as vars attached to params are used instead of the explicit values.
Unsure how to pass the value along without the expression being generated.
An alternative way is putting None as the default value and generates an expression in the function when gamma=None.
gamma is not specified directly as a parameter when converting it to a relay operator. It is stored in the weights in the keras layer. I believe it’s similar with other frameworks, as well.
Since the internal batch_norm function just passes the values to the cpp function, it would not follow the rest of relay’s paradigm to generate a gamma=None variable. It would also screw up the ordering, as the variables after it also have to have default values, or gamma has to be placed with the other default variables.
Since the function is called internally, it might not cause an issue, but I’m unsure this would be the case.
I got your point. Then the solution might be putting gamma=None and generating a default expression here, it seems no other ops doing the similar things tho.
Otherwise, if Keras is the only frontend that handles scale in the layer, generating a default gamma when converting Keras batch_norm to Relay would be a straightforward solution.
cc @Huyuwei