Problem in ANF and DCE pass


First, if you apply ToANormalForm pass twice to a relay program, it will double the program size. For example, running the following test case, anf2 is different from anf.

x = relay.const(1)
y = relay.const(2)
val = x + y
anf = run_opt_pass(val, [transform.ToANormalForm(), transform.InferType()])
anf2 = run_opt_pass(anf, [transform.ToANormalForm(), transform.InferType()])

Second, deadcode elimination pass will go into an infinite loop after the FusedOps pass. The following example can never finish.

import tvm.relay.testing
g = tvm.relay.testing.inception_v3.get_net(1, 1000, (3, 299, 299), 'float32')
g = run_opt_pass(g, transform.Sequential([transform.SimplifyInference(), transform.FuseOps()]))
# the next pass never finish
g = run_opt_pass(g, transform.DeadCodeElimination())

cc @MarisaKirisame @jroesch


for the former, dead code elimination remove all naive binding.
for the latter, a case on Primitive for DCE is needed.

Since I am pushing on training very hard, can you try fix DCE yourself? It shouldnt be hard.