Pointers / Advise from the Community

Hi,

I have been watching TVM from a far for quite some time.

I am quite interested in Relay as a more general purpose IR. I am involved in the Apache Arrow project, particularly, the Rust implementation and I am wondering how feasible it might be to integrate with TVM.

I am not working in deep learning but I still think that I could leverage TVM. Arrow has a tensor type that I would use to integrate with TVM in the same way as the current Rust frontend does with ndarray.

I would like to build an abstraction in Rust that internally builds the Relay IR and then uses TVM for optimizations and deployment (I am also very interested in leveraging TVM to compute derivatives for my computations).

I am certainly no C++ expert and TVM is quite a large and complex project so I would just like to ask if anyone in the community sees any major issues with what I am proposing or has any advise for where I should start?

Thank you

3 Likes

Sounds very promising. @tqchen if you can ping some rust experts

cc @jroesch@nhynes who might have more thoughts

Hi @paddyhoran thanks for typing up these thoughts.

Just to understand a bit better, it sounds like you’re proposing work on two things:

  1. Enable the rust TVM runtime to support Arrow tensor type data as an efficient input format.
  2. Write a proc_macro or API (or both) that enables you to construct Relay IR from Rust (and then invoke compilation etc on it).

Does that capture it right? Both sound reasonable to me from a high level perspective. Point 1 seems like pretty low hanging fruit and I know @jroesch has ideas and thoughts on point 2.

Also, do you have any specific use cases in mind to enable with this kind of support? (just curious)

Hey @jknight,

Spot on with what I am proposing, I agree that supporting the Arrow Tensor type will be trivial enough.

I actually work on Actuarial modeling, I’m exploring how I can take innovations from TVM and integrate them.

Great! I think these both align well with what we’d like to see in Rust+TVM. If you put an RFC or pre-RFC together for 2 either in this thread or a new thread and ping me, then I’ll make sure at least @jroesch and I take a look at it.

Any machine learning system, as long as it has a data pipeline, would largely benefit from Apache Arrow. I am not an expert in Rust, but really would love to see how it goes :slight_smile:

Yep, I think Arrow integration could be really powerful. Thanks @jknight, I think I need to research more, etc. before opening up an RFC. Thanks all for the discussion.