[Graph IR] Comparison between Relay, NNVM and others


When we write a pass on graph IR, what’s the benefits of Relay IR compared to nnvm graph IR?

replies from @jroesch

jroesch [4:55 AM]
@xqdan the type information is part of the AST, we support more flexible dtype and shape inference, we have explicit variable bindings which is good for code generation and scoping issues, we support abstraction (i.e subgraphs/functions) we have support for control operators baked into the IR, we support recursion, can encode loops in the IR, and we support shape polymorphism/dynamic shapes in the IR. We also have a new AD algorithm (not yet merged) which can compute Nth order gradients over things like map, fold, etc

jroesch [4:55 AM]
We also have support for data types coming up allowing us to define networks over lists, trees, etc
We also unified the attributes/parameters system from NNVM into TVM, we have well defined semantics for the the entire IR in contrast to NNVM which had generic IR, and then semantics that were given to NNVM graphs by things like NNVM compiler/executor.
The IR also supports inline constants which can be arbitrary tvm.nd.NDArray
Meaning we don’t need specialized operators for scalar/non-scalar/generic case
For example we can just use Relay interpreter to do constant evaluation


I have some questions about relay,

  1. Can relay support dynamic graph like pytorch?
  2. How does relay support flexible shape, do we have examples?
  3. We notice that TF supports SSA on dataflow graph, SSA equals with ANF in general, so what’s the difference between these solutions? can we say they are comparable?
  4. What’s the advantage of relay for runtime scheduling? what’s meaning by “The tight coupling between runtime representation and compile-time representation has limited flexibility and frustrated developers; Relay will decouple the representations.”



Especially, what’s the solution of relay for batching the deep-learning models that operate over data of varying size and structure.


I think this is the one you want: https://github.com/dmlc/tvm/issues/2122