Performance regression with relay (opt_level=3)


#1

I’m having some problems when deploying this model on x86 architecture using relay.
When setting opt_level=3 performance get worse by almost 3 times compared to using opt_level=1.
Any clue why does this happen?

I’ve also tuned convolution on x86 using autotvm, but inference is still quite slow. Is there any other way I can further speed up the inference?


#2

Recently there are several similar issues on x86. Did you try debugger runtime to see the performance difference between opt_level=1/3?


#3

Hi @kevinthesun,

I have been also working on deploying model on x86 and getting bad performance results. For example, the situation in explained in the following post:

You mentioned that recently there are issues on x86. Could you please elaborate on the current situation for x86?

Thanks


#4

quantization is a different topic. @janimesh


#5

Can you share your tuning_option? That might be the reason why autotuning can’t get great performance.