The Relay paper describes a plan for automatic differentiation. How far along is this? In particular, is it specced out enough for casual contributors? Training support in NNVM is sparse, at best, and I’d be interested in moving Relay training forward more quickly.
We have a new version of it that has a more flexible method for ad over Relay programs, but the branch needs to be polished enough to be upstreamed to the TVM repository. We have been focused on inference but could prioritize upstreaming it. Let me sync with the other people working on Relay and get back to you.
2 Likes