Hello, i’m not sure what am I missing here but I am trying to compare the inference performance between tvm and simple mxnet resnet-18. I have Autotuned the model and it has some performance increase but it is still slower than the simple Mxnet model. I am getting 76 ms for the tvm auto-tuned model and 9 ms for the simple mxnet model. Can some one please help me?
This is the code for loading auto-tuned model
This is the mxnet code:
I have attached the graph debug output:
Preformatted text
Change target to “llvm -mcpu=skylake-avx512”?
Yeah, I did but there is no improvement. I noticed that when I load the model I don’t do rt.set_input(**params) because I am loading a bytes and I am not sure how to turn the bytes into dict. I’m not sure if that can effect it.
How many CPU cores are you using? Latency number looks like single core inference.
Yes, I am using single core. But i don’t think it would change prediction time for tvm. I have tried with multiple cores but not much improvement.
Multi-core should significantly improve performance. Can you use htop to watch the cpu usage?
I see what you are saying but should it still perform better than mxnet? like it is a lot slower.
I think the problem is in Auto-tuning, I am following the tutorial on tvm site : https://tvm.apache.org/docs/tutorials/autotvm/tune_relay_x86.html
But they are getting 3.16 ms and I am getting 160 ms. Do you know how to fix this? I’m not sure what to change in the tuning configuration