Graph level exploration with Relay: where to get started?

Development
#1

Hello, I’m trying to use TVM and Relay to do some graph level optimization exploration, e.g. given a compute graph as an input, search the graph and manipulate nodes (split, fuse, etc) based on some rules or cost model and output a new graph that might have better performance.

Should I get started by writing a new pass for Relay? Or there’re some existing works that I can leverage? I see codes like combine_parallel_conv2d, fuse_ops under src/relay/pass. Can anyone give me a pointer here?

#2

Hi @moderato

Thanks for trying TVM. In my opinion, I think the tutorials might be a help to get start.

https://docs.tvm.ai/tutorials/

#3

Thank you @cchung100m for the reply! So I end up with the c code of Relay and I’m looking into op_attr_type.h and trying to understand these op types.

I wonder how ops are categorized into these types. For example, it’s quite obvious max, mean, sum, etc are reduce ops. However, I don’t quite understand why ops like cos, floor, sqrt, etc are broadcast instead of elementwise. Also why broadcast ops and elementwise ops are both scheduled as injective ops? Are the types listed in op_attr_type.h not exclusive to each other? Thx!

#4
  1. Ops you mentioned ( cos, floor, sqrt …) are registered with RELAY_REGISTER_UNARY_OP whose attr_type is kElemWise. Source code is in src/relay/op/op_common.h. They are not kBoardcast.
  2. kElemwise and kBoardcast Ops’s behavior are different in Pass::fuse_ops.
  3. In my opinion, enum OpPatternKind is not classification on all Ops, it describes the op’s pattern attribute, and helps fuse_ops to decide whether op and parent op can fuse together.