Rewrite simplify rules limited to index type vars

Development
#1

Hi all,

We’re working with a number of tensor expressions and trying to simplify them but rewrite simplify has a condition to apply simplifications only on index type Expressions. Our case is very simple and worked with the previous simplifier, it looks something like (x + y) - (x + y), where x and y are both floats.

The mentioned condition is here (and also in other operations’ mutators): https://github.com/dmlc/tvm/blob/master/src/arithmetic/rewrite_simplify.cc#L144

Shouldn’t we be able to simplify expressions of other types (not just integers)?

@tqchen Maybe I’m not seeing the full context, but what would be some sample cases where is it unsafe to simplify non-integer Expressions? Do you think it’d be safe if we extend that condition to floats?

Thanks!
Giancarlo

#2

The main goal of simplifier is integer set analysis, so we restrict most of them integers to avoid certain conditions like floating pt communicative errors.

Also given that most expression are already compact, we kept them a it is. If you feel that there is a compelling reason to support other types. I think we could lift the restriction, but still vet each cases carefully when we add rules for non-integer cases.

1 Like
#3

Hi @tqchen, thanks a lot for your reply. I see your point, there could certainly be some cases where simplifying floats is not safe.

We certainly have a need to simplify some float expressions (which were being simplified with the old simplifier). So, considering your point and the fact that in rewrite_simplify the rules refer to variables (not const values), do you think we could safely apply the simplification rules to float expressions? Maybe not completely lifting the condition but adding float as a supported type?

Thinking of something like (x - x), where x is a var float, x will always be equal to x without any risk of losing information due to a precision issue. Isn’t it?

It’d be really helpful to have your insights here.

Thanks again,
Giancarlo

#4

Yes, I agree that we could lift some of the simplification rules for float values as well, we just need to vet these rules more carefully. One way to add such vetting is to add a limited set of vetted rules into an else branch and explicitly comment that these rules works for floats, contributions are more than welcomed

1 Like
#5

Awesome! Thank you very much for your insights here.

Cheers,