How can i do AutoTVM once for a operator but apply the best config for other input/output shape with best performance?


#1

How can i do AutoTVM tuning once for an operator but apply the best config for other input/output shape with best performance?

As you know, AutoTVM get best config only with certain input/output shape of compute. If shape changed, the operator need do AutoTVM Tuning again. This feature limits AutoTVM to be used for common algrithm optimization.

For example, using resize as cnn preprocess, the input image shape is always mutable, so we cann’t optimize resize operator using AutoTVM.


#2

This is a current limitation of AutoTVM - the issue here is that schedule settings are highly dependent on input shape: changing the shape from resizing input images may lead schedules to be suboptimal.

I suggest two solutions: (1) to pre-tune to all a whitelist of all shapes that your optimization algorithm might explore, or (2) perform learning in the loop when we encounter a new shape and use transfer learning to accelerate the convergence.

@merrymercy and @eqy might have more to say


#5

Same problem can be encountered in many new CNN networks, like SPP-Net, using Spatial pyramid pooling layer insert before FC layers so that input data of CNN could have mutable shape, and don’t need crop/wrap for input anymore. FCN(Fully Convolution Networks) also have no limits for input shape.

Up to now, we cann’t using AutoTVM optimization for these mutable networks, unless using AutoTVM as JIT, but the tunning time is also unacceptable.


#6

Regarding “any” shape support, this does indeed test the limits of our current approach with AutoTVM’s static shape assumptions. @jroesch and @haichen are working on supporting dynamic shapes in relay. Perhaps we can start an RFC regarding AutoTVM support and how we can apply knowledge on those shapes (e.g. size along one dimension is a multiple of N) to improve performance at runtime.