autotvm fails on gemm tutorial

I was trying to use code from https://zhuanlan.zhihu.com/p/75203171 (it’s a Chinese post, but it’s essential the same as Writing tunable template and Using auto-tuner ) with modified size 4096^3. However, in my AWS instance, it keeps reporting tons of similar error like this

DEBUG:autotvm:No: 199 GFLOPS: 0.00/0.00 result: MeasureResult(costs=(TVMError(‘Traceback (most recent call last):\n [bt] (3) /home/ubuntu/.local/lib/python3.6/site-packages/tvm-0.6.0-py3.6-linux-x86_64.egg/tvm/libtvm.so(TVMFuncCall+0x61) [0x7f51e3d495e1]\n [bt] (2) /home/ubuntu/.local/lib/python3.6/site-packages/tvm-0.6.0-py3.6-linux-x86_64.egg/tvm/libtvm.so( ZNSt17_Function_handlerIFvN3tvm7runtime7TVMArgsEPNS1_11TVMRetValueEEZNS1_15TypedPackedFuncIFRNS0_5StageES7_NS0_7IterVarEEE17AssignTypedLambdaIZNS1_8Registry15set_body_methodIS7_S8_IS9_EEERSD_MT_FT0_DpT1_EEUlS7_S9_E_EEvSG_EUlRKS2_S4_E_E9_M_invokeERKSt9_Any_dataOS2_OS4 +0x98) [0x7f51e3570c48]\n [bt] (1) /home/ubuntu/.local/lib/python3.6/site-packages/tvm-0.6.0-py3.6-linux-x86_64.egg/tvm/libtvm.so(tvm::Stage::vectorize(tvm::IterVar)+0xb0) [0x7f51e38e7c70]\n [bt] (0) /home/ubuntu/.local/lib/python3.6/site-packages/tvm-0.6.0-py3.6-linux-x86_64.egg/tvm/libtvm.so(dmlc::LogMessageFatal::~LogMessageFatal()+0x32) [0x7f51e35375e2]\n File “/home/ubuntu/tvm/src/schedule/schedule_lang.cc”, line 377\nTVMError: Check failed: var->iter_type == kDataPar || var->iter_type == kOpaque || var->iter_type == kUnrolled || var->iter_type == kVectorized || var->iter_type == kTensorized || var->iter_type == kParallelized: Cannot vectorize on CommReduce’,),), error_no=2, all_cost=0.006806135177612305, timestamp=1579123976.0153828) [(‘tile_x’, [-1, 8, 32]), (‘tile_y’, [-1, 16, 32]), (‘tile_k’, [-1, 2048]), (‘reorder’, (2, 0, 1)), (‘ann’, [‘unroll’, ‘vec’, ‘unroll’])],None,10215900

I tried with code in Writing tunable template and Using auto-tuner and it also reports errors like

No: 9 GFLOPS: 0.00/0.00 result: MeasureResult(costs=(RuntimeError(‘Traceback (most recent call last):\n [bt] (3) /home/ubuntu/.local/lib/python3.6/site-packages/tvm-0.6.0-py3.6-linux-x86_64.egg/tvm/libtvm.so(TVMFuncCall+0x61) [0x7f6bdfd335e1]\n [bt] (2) /home/ubuntu/.local/lib/python3.6/site-packages/tvm-0.6.0-py3.6-linux-x86_64.egg/tvm/libtvm.so(std::_Function_handler<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*), tvm::runtime::RPCModuleNode::WrapRemote(void*)::{lambda(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) #1 }>::_M_invoke(std::_Any_data const&, tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&)+0x41) [0x7f6bdfd914f1]\n [bt] (1) /home/ubuntu/.local/lib/python3.6/site-packages/tvm-0.6.0-py3.6-linux-x86_64.egg/tvm/libtvm.so(tvm::runtime::RPCSession::CallFunc(void*, tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*, void* ( )(int, tvm::runtime::TVMArgValue const&), tvm::runtime::PackedFunc const )+0x167) [0x7f6bdfd9afb7]\n [bt] (0) /home/ubuntu/.local/lib/python3.6/site-packages/tvm-0.6.0-py3.6-linux-x86_64.egg/tvm/libtvm.so(dmlc::LogMessageFatal::~Lo’,),), error_no=4, all_cost=10.207964420318604, timestamp=1579122195.5362496) [(‘tile_y’, [-1, 128]), (‘tile_x’, [-1, 8])],None,46

What’s wrong here? Thank you for your help!

1 Like

I found the problem is associated with vectorize on k.inner dimension, but still don’t know what wrong with it. Why TVM can vectorize the outer loop?

I meet the same problem, but i don’t have good solution for it