How to do auto-tuning for a list of specific ops？

Questions
#1

I have read tutorials about auto-tuning, seems there are two kinds for auto tuning.

  1. write a schedule for a specific op and create a task for it.
  2. extract tasks from a graph, which is to auto tune the whole model.

Here I have a lot of models to tune, and their convolution configurations have a lot in common. For example, I have 100 models with convolutions with 1x3x224x224 input and 512x3x1x1 kernel.

So if I tune each model, I will tune this kind of convolution 100 times.

I want to get the whole set of all ops, and tune each op for just once.

Is there any api that supports this situation ?

#2

Hi,

The implementation of function prune_old_tasks in this discussion [solved]Can we resume an autotuning session? may helps you.

1 Like
#3

Thank you！Seems it will work to me.

#4

The official solution to this question is leveraging extract_from_multiple_programs. You can give that API multiple models at once, and it will return a list of unique tasks. By tuning the list of tasks, your log file can be used for all models you provided.

#5

Thank you ! I have two questions for this solution.

  1. To use this API, I have to load all models at the same time, which seems to have a huge demand for memory.

  2. I didn’t find a way to save tasks to disk, that means if the tuning is interrupted , I have to do the whole thing from beginning?

#6

Good questions in general.

  1. This would be a problem if you have limited memory in the machine.

  2. Currently AutoTVM doesn’t have an official API to export tasks. At this moment, one solution I can think of is writing a script to extract necessary task information and save it to a JSON file. When importing the JSON file, your script uses task.create to recover all tasks.

Actually, the soltution to question 2 is also applicable to question 1. Once you have a mechanism to export tasks, you can extract tasks from models sequentially and export them. Then you can de-deuplicate redundant tasks in the exported JSON file.