Any materials of Relay for beginners?

Questions
#1

Hi community,

Great to see the Relay IR is more and more mature now and I notice several deep learning frameworks are migrating their IR to Relay, eg. MXNet and Myia. I’m wondering if there are any public materials (documents, slides, video records) for beginners about the software architecture, usage, data structures and advantages compared with NNVM v1, MLIR and Glow. That will help us to better leverage Relay in frameworks and also help us to understand the differences between Relay and the existing IR of frameworks. I know there is a Relay paper published on arXiv recently, maybe my understand is wrong, but seems the paper is more from PL perspective.

3 Likes
#2

Some related materials, specific questions are also more than welcomed.

1 Like
#3

I want to followup to add that relay should be viewed in the context of the TVM stack, which offers an intelligent and principled way for deep learning compilation.

Note that IR is only one part of the story. The main thing is what TVM stack infra offers, in terms of compilation, optimizations and runtime. Some of the specific compilation infrastructure relay offers now includes(not an exhaustive list and it is growing):

  • Generic fusion algorithm for effective operator fusion
  • Generic pass infra and optimisations for automatic quantizations
  • Principled handling of subgraph via function and module
  • Support of recursion and dynamic workloads(via VM)
  • Interpolation with frontends such as pytorch, tf, mxnet
  • Automatic code generation and integration with AutoTVM
  • End to end support for specialized accelerators

This is the reason why I am super excited about the general technical direction relay provides.

#4

Thanks for the information and links, Tianqi. There’re two things I try to understand for relay:

  1. what are the main differences and advantages of it compared with previous NNVM v1 and recent MLIR or Glow?
  2. regarding mxnet, how big the change will be if we migrate from NNVM v1 to relay? I understand that mxnet has it’s own operators and doesn’t rely on the compiler part of TVM.

Sorry no specific technical question yet. I’m reading the slides of relay from TVM conf. Hope not many things are changed since then~

#5

The improvement from NNVM v1 is quite elaborated in https://github.com/dmlc/tvm/issues/1673
Specifically here are a some items that I in particular like

  • Generic pass infra for subgraph via function and module
  • Support for dynamic workloads
  • Generic operator fusion and other optimizations

MLIR related discussion can be found in Google lasted work: MLIR Primer

The cost of migration of MXNet is mainly the topic that I think belongs to the mxnet community. I cannot speculate the general cost, but I do think that given the benefits bought by the general relay and TVM infra, it would be helpful to do so eventually

#6

As Tianqi has already answered a lot of questions on the side of TVM compiler stack, I would love to answer why a deep learning framework (like MXNet, PyTorch, etc) needs NNVM v2.

Personally, compared with MXNet’s current solution, I feel the most intriguing part is that we are able to treat every backend equally.

More specifically, MXNet currently uses subgraph, which crops part of the graph out and then treats it as an operator, which only hides stuff and prevents us from optimizing things globally.

I am not trying to say subgraph is bad - this has been one of fine choices - because of the expressiveness issue in NNVM v1. And now, in NNVM v2, with real functions, closures, we would expect that we benefit almost all backend developers for almost all hardwares with equal opportunity, not to narrow you guys down in a single tiny operator.

I am not confident about the general cost migrating from NNVM v1 to v2, but as an open source community, we are very open to any discussion, open to address concerns from any party : -)

#7

Thank you @tqchen and @junrushao for the inputs. I would say yes the subgraph feature is the key concern for us as backend developers. Besides that, I’m also interested in the design and management of optimization passes. Seems I should go through the RFC discussion first~