[AutoTVM] How to tune dense layers?

tico · August 15, 2019, 9:04am

Hi,

I was wondering if is possible to use AutoTVM to tune dense layers and if so could you please share or pointing me to an example on how to achieve this. My target system is x86.

Thanks

gasgallo · August 23, 2019, 9:53am

Interested too. It would be nice to suppress the warning about the dense layer missing optimization and maybe also get a little improvement in performance

gasgallo · November 13, 2019, 6:28am

Is this still not possible?

jackwish · November 13, 2019, 8:36am

Add more ops to your tuning scripts like:

from

github.com

apache/incubator-tvm/blob/master/tutorials/autotvm/tune_relay_x86.py#L192






########################################################################
# Finally, we launch tuning jobs and evaluate the end-to-end performance.


def tune_and_evaluate(tuning_opt):
    # extract workloads from relay program
    print("Extract tasks...")
    mod, params, data_shape, out_shape = get_network(model_name, batch_size)
    tasks = autotvm.task.extract_from_program(mod["main"], target=target,
                                              params=params, ops=(relay.op.nn.conv2d,))


    # run tuning tasks
    print("Tuning...")
    tune_kernels(tasks, **tuning_opt)
    tune_graph(mod["main"], data_shape, log_file, graph_opt_sch_file)


    # compile kernels with graph-level best records
    with autotvm.apply_graph_best(graph_opt_sch_file):
        print("Compile...")
        with relay.build_config(opt_level=3):

to

            params=params, ops=(relay.op.nn.conv2d, relay.op.nn.dense)

gasgallo · November 13, 2019, 9:15am

@jackwish Is topi dense layer registered in autotvm for x86 target?

comaniac · November 13, 2019, 5:40pm

It is: https://github.com/apache/incubator-tvm/blob/master/topi/python/topi/x86/dense.py

jackwish · November 14, 2019, 1:41am

Sure it is.

PS. why post must be at least 20 characters…

gasgallo · November 14, 2019, 2:30am

Thanks everyone, I’ll try it asap.

gasgallo · November 14, 2019, 3:19am

@jackwish @comaniac it does work! But tuned model performance are the same if not a bit worse than untuned one.

jackwish · November 14, 2019, 7:40am

Sorry but I didn’t get it…

jackwish · November 14, 2019, 7:41am

Really? What’s your workload?

gasgallo · November 14, 2019, 7:49am

workload=('dense', (1, 25088, 'float32'), (512, 25088, 'float32'), 0, 'float32')

gasgallo · November 15, 2019, 2:43am

@jackwish any clue or anything else I can try?