Relay model zoo?

Development
#1

In my time using TVM to do various research projects, I found the thing I wanted most of all was a model zoo – pretrained models already pulled in to Relay/TVM. tvm.relay.testing.* was a great starting place for me, but I’d love to see (1) pretrained weights included, (2) a greater variety of state-of-the-art models, and (3) perhaps some models which are imported from different frontends, to serve as importing examples.

I understand that this may contradict what I see as a key feature of TVM – that it should be easy to get models from any frontend into TVM, and so a model zoo should be theoretically unnecessary. In my limited experience, though, it was not necessarily straightforward to do this importing. For what it’s worth, I may not have given the importers a fair try – but in general I found the ones I used to be hard to use at best. If I cared about just one or two models (as some users of TVM might), then the cost to import the model would be worth it. In my use cases, though, I was trying to get as many models up and running as I could, and found that the overhead of importing was my main barrier.

I’m wondering if others have opinions on this idea. I’m no longer using TVM for research as directly, but I wanted, at the very least, to start the discussion on the idea of building a Relay model zoo.

#2

Hi,

I find your post very interesting, I would expect that the frontend conversion into Relay is quite mature and stable. Therefore I would want to gather more info on your POV.

Would you mind elaborating on these points?

  1. What models where you trying to convert?
  2. From what framework were they coming from?
  3. What exactly was the “least” straightforward aspect of importing a model?
  4. What was your workflow? (i.e. what were you trying to do research?)

Thanks in advance :slight_smile:

#3

As a worker of hardware accelerator company, I find almost each company need to create a NN model parser tool to do the same work of Relay frontend, and the most important part of it is keep a model zoo.

From my work experiences from two hardware accelerator companies, maintain a IR and support NN framwork(e.g., Tensorflow, caffe, tflit) need lots of time and work, and Relay make a better job than these company internal tools, but we still want Relay have a well maintained model zoo and useful utilities(e.g, tools to extract one single operator’s Relay IR and its constants for hardware to do unit test).

If Relay have a well maintained model zoo and these utilities I think lots of companies who hasn’t NN parser tool will use Relay as their IR, if them think Relay is too complex(e.g., they needn’t control flow), then I think they only need make a tool convert Relay IR to their IR instead of make a whole NN parser tool.

Actually this thought is from a real work task, when I import ResNet-50 Tensorflow model to TVM, quantize it and build it, I found it will report error in quantize phase, it is strange to me, because this ResNet-50 model is used almost anywhere as a benchmark model, I know Relay must be have supported it, but I meet the error, and after some debugging, I found after Relay Tensorflow frontend the Relay IR have lots of useless small functions, then I remember I have saw “RemoveUnusedFunctions” pass in discuss forum, so apply it resolve the quantize error. Even through it doesn’t cause too much time to solve this problem, but I think it shouldn’t be appeared, so if there is a model zoo, I would like to push this to it and other people will not meet the error.

3 Likes