[Error] runner=autotvm.LocalRunner not working

#1

Hi There

I built tvm from source today and my autotvm.LocalRunner is throwing an error which was not the case with TVM that I built last time ( quite a few days ago).


[Task  1/ 4]  Current/Best:    0.00/   0.00 GFLOPS | Progress: (0/2000) | 0.00 sTraceback (most recent call last):
  File "latest_autotune_script.py", line 134, in <module>
    tune_tasks(tasks, **tuning_option)
  File "/workspace/work/tvm/tensorflow/utils/utils.py", line 220, in tune_tasks
    autotvm.callback.log_to_file(tmp_log_file)])
  File "/tvm/python/tvm/autotvm/tuner/xgboost_tuner.py", line 86, in tune
    super(XGBTuner, self).tune(*args, **kwargs)
  File "/tvm/python/tvm/autotvm/tuner/tuner.py", line 108, in tune
    measure_batch = create_measure_batch(self.task, measure_option)
  File "/tvm/python/tvm/autotvm/measure/measure.py", line 252, in create_measure_batch
    attach_objects = runner.set_task(task)
  File "/tvm/python/tvm/autotvm/measure/measure_methods.py", line 342, in set_task
    super(LocalRunner, self).set_task(task)
  File "/tvm/python/tvm/autotvm/measure/measure_methods.py", line 212, in set_task
    raise RuntimeError("Cannot get remote devices from the tracker. "
RuntimeError: Cannot get remote devices from the tracker. Please check the status of tracker by 'python -m tvm.exec.query_rpc_tracker --port [THE PORT YOU USE]' and make sure you have free devices on the queue status.
Exception in thread Thread-5:
Traceback (most recent call last):
  File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.5/threading.py", line 862, in run
    self._target(*self._args, **self._kwargs)
  File "/tvm/python/tvm/autotvm/measure/measure_methods.py", line 563, in _check
    while not ctx.exist:  # wait until we get an available device
  File "/tvm/python/tvm/_ffi/runtime_ctypes.py", line 176, in exist
    self.device_type, self.device_id, 0) != 0
  File "/tvm/python/tvm/_ffi/_ctypes/function.py", line 209, in __call__
    raise get_last_ffi_error()
tvm._ffi.base.TVMError: Traceback (most recent call last):
  [bt] (6) /tvm/build/libtvm.so(TVMFuncCall+0x61) [0x7f3c978e9371]
  [bt] (5) /tvm/build/libtvm.so(+0x131c376) [0x7f3c978e7376]
  [bt] (4) /tvm/build/libtvm.so(tvm::runtime::RPCDeviceAPI::GetAttr(DLContext, tvm::runtime::DeviceAttrKind, tvm::runtime::TVMRetValue*)+0x264) [0x7f3c97911d14]
  [bt] (3) /tvm/build/libtvm.so(+0x134def6) [0x7f3c97918ef6]
  [bt] (2) /tvm/build/libtvm.so(tvm::runtime::RPCSession::HandleUntilReturnEvent(tvm::runtime::TVMRetValue*, bool, tvm::runtime::PackedFunc const*)+0x12e) [0x7f3c97918d8e]
  [bt] (1) /tvm/build/libtvm.so(+0x135c5bc) [0x7f3c979275bc]
  [bt] (0) /tvm/build/libtvm.so(tvm::common::Socket::Error(char const*)+0x326) [0x7f3c979273f6]
  File "/tvm/src/runtime/rpc/../../common/socket.h", line 313
TVMError: Socket SockChannel::Recv Error:Connection reset by peer

Is this method deprecated ? or is there any change while using localrunner ?

1 Like