Rust frontend support

Application
#1

Hi everyone

I’m a Rust fan and I’ve decided to add Rust bindings for TVM frontend API. This is a WIP dedicating my free time: https://github.com/ehsanmok/tvm-rust

Cheers
Ehsan

#2

as far as i know there is a concurrent effort by @nhynes, maybe you can coordinate your effort

#3

Interesting! I’m open to it :slight_smile: checking his crate now.

#4

Hey @ehsanmok, right now I’m porting the TVM runtime in tvm-rs with a focus on ensuring build compatibility with rust-sgx-sdk via xargo.

As of right now, I have

  • basic runtime w/ threading [with tests]
  • graph runtime (de)serialization [with tests]
  • untested graph runtime
    I just need to add some integration tests for the graph runtime and SGX threading and it’ll be good to go for my purposes. Please feel encouraged to submit a PR! Might I ask what your particular desiderata are for the Rust runtime?
#5

Hi @nhynes, that’s great to hear! I think we are heading to different directions though. My motivation is building a general idiomatic Rust support for runtime now.

Rust as a serious, modern contender of C, lacks a DL framework. There have been various efforts in creating DL ones such as autumn/leaf but that’s not being maintained anymore. There has been various other efforts going into DL frameworks, but in my opinion they have all realized how heavy the task is. Instead I’m in favor of leveraging the existing ones instead of reinventing the wheel. With huge demand of deploying models on embedded devices, I think TVM lies in a sweet spot and I can foresee it in relation to embedded-wg (maybe more to the future).

I’m definitely open to more discussions and ideas.

1 Like
#6

I think that we’re not actually heading in different directions at all :slight_smile: My goal is to create a generic TVM runtime that happens to support SGX (as the current one does).

If you want to collaborate, here’s what I had envisioned:

  • first-class support for the ndarray crate to allow both TVM and non-TVM math ops
  • automatic conversions between ndarray, tvm::Tensor, and DLTensor
  • support for model graphs using the NNVM IR

All of those things are basically already implemented. What exists so far should mostly support your use case. What kinds of things would you want to see added? Perhaps replacing the ndarray ops with TVM ops? It would be interesting to see a TVM syslib module which has the core tensor ops but where TVM really shines is fusing many small operators. This requires ahead-of-time definition of the desired computation; otherwise you start getting into the territory of languages like Julia. (Note that even the MXNet imperative API is making calls to OpenBLAS primitives via mshadow; ndarray does support the OpenBLAS backend)

#7

I see! so I propose to make it our collaborative effort. I’ve made some initial suggestions in separate issues in your project and we can start from those.