[USABILITY IDEA] Create a command line compiler that would perform common build tasks out of the box

The command line app for example could compile a NN model to data in these common situations:

  1. classify an image
  2. classify a video, frame by frame
  3. etc. etc.

It’s much easier to run an executable, than to write programs using APIs that TVM provides.

In my opinion, although this idea is worthwhile to consider, it’s hard to define a general command line application for all users. For instance, here is an example command line for image classification:

python -m tvm compile --model="GluonCV:resnetv2_50" \
                      --pretraied=ture \
                      --shape="1,3,224,224" \
                      --dtype="float32" \
                      --tuning=true \
                      --tuning-options={...} \
                      --opt-level=3 \
                      --output="deploy.tar"

As can be seen, users have to specify lots of details in order to let TVM figure out their intentions, and the requirements are vary from users to users. A more practical solution from my perspective is writing a tutorial to teach users how to make command line apps based on their needs. The command line app can be built on the top of TVM and can be fully customized.

1 Like

No need to cover all uses, just cover some basic uses that are most common. More advanced use should still require scripting/programming.

BTW Your command line misses these two important options:

  • input normalization, some models require 0…255, -128…+128, etc.
  • input color scheme, some models require BGR and some RGB.
  1. That’s my concern actually. It is hard to define what’s “the most common uses”. It seems to me that you can only define it by collecting lots of user feedback. Taking the example I made, I have no idea how many of the 8 options I listed are considered as “advanced use”.

  2. This is exactly what I want to raise. Even I have listed 8 options, which I already feel too many, you can easily find more important options.

I would like to clarify that I am not against to this idea. I just point out the concern I have when defining the command options. Having a command-line app could definitely help end-users to get started, but it could also be a bad experience if a new user finds the command line options are helpless to her use cases.

1 Like

It’s a good idea to have a command line compiler. Here are some ideas:

  • “most common uses”: The pre-tuned models in the tophub should be considered.
  • clang have around 400 command line arguments, but it’s still considered to be a successful compiler. The important thing is it’s difficult to define a set of reasonable default options, in order to make the command line compiler easy to use.

Basic uses are quite obvious.

  • Image classification
  • SSD/region detection
  • Pose estimation
  • same with pictures and videos

Just compile an app that would “annotate” a video based on the NN computation and people would think that this is awesome. -)

I would rather I think the input should be an arbitrary network defined with supported frontends, while the output should be compiled tuple (graph, params, lib).

I agree with @liangfu, because TVM itself is more like an engine instead of an application. Based on @yurivict’s basic uses, it would be better to have a separate project on the top of TVM to provide such applications. Similar to the relationship between Gluon CV/NLP and MXNet.

2 Likes

We’ve been looking into creating a command line interface and have a few initial scripts. These primarily diverge from what’s been discussed here in that the auto-tuning, compilation and execution steps are all separated into their own subcommands with the intention of not coupling the interface too closely to any particular ML use case. I’d be happy to put up an RFC/discuss post if there’s interest in making them available and allowing others to contribute.

Sounds good. Looking forward to seeing the RFC :slight_smile: