Batch_size and dtype as autotuning parameters

Hi,

I am testing “tune_relay_cuda.py” and “tune_relay_x86.py”. In get_network of the files, there are two configurable parameters. batch_size and dtype. By default, batch_size = 1 and dtype = float32.

What else does TVM support for batch_size (2,3,… N) and dtype (e.g., int8, int16, etc)? Also, whenever I use different parameters, I need to re-run tuning. Is it right?

TVM supports arbitrary batch sizes, but usually your only need to tune batch size 1 and the result can be applied to others. For quantized data types, TVM now supports int8 well, and yes you need to tune for different data types separately.