Dose TVM support training models?


#1

all tutorials in nnvm use pre-trained models from mxnet or onnx or coreml
Does NNVM support training models?


Is TVM applicable for training?
#2

No, TVM purpose is to use existing models to compile for different hardware and backends. See tvm.ai


#3

ngrpha supports training models。 So I think NNVM should support training too


#4

Actually optimizer is added into nnvm one month ago, so nnvm does support training. Since parameter will be updated during training, some optimization techniques need to be modified.


Here you can see in nnvm how we can update parameters.


#5

thanks very much!
This is more like a hand-writing training process.
If I wrote a model with mxnet, I can train it easily. Can I generate a compute-graph of training, and use tvm to run it, so that I can train a mxnet model on a special hardware that mxnet doesnot support?


#6

I have never tried training but looks like some of the operators do not have backward definition.


Train a model with the TVM
#7

@xiaoshiyi Yes. As Yizhi mentioned, currently one major issue is that not all operators have FGradient operator. Another concern is that when opt_level=0, which is the SimplifyInference, batchnorm and dropout just have inference behavior. To train networks with these two operators, you may need to somehow disable this optimizing pass. Some interface modifications might need here.


#8

Interesting discussion!
Recently, I’ve been working on writing my own CNN trainer using TVM but due to my lack of familiarity with the API, I wasn’t sure how to express certain steps of the computation.

I’ve asked the question in a separate thread [1].

[1] Expressing strided output tensors