HI,
I’ve been hitting This event loop is already running when trying to run autotuning from Jupyter notebooks. Is this something that is easily avoided? (I do realize that Jupyter and long-running things are … special.)
For now I have worked around by moving the invocation of the autotuner in a second thread (thread.Thread(…) ; t.start(); t.join() where in the thread I create but don’t start a tornado IOLoop and then run the autotuner):
# we use threading and tornado here to work around TVM and Jupyter colliding over IOLoops
# In a regular python command line, you should be able to just call the tuner...
import threading
import tornado
# create tuner
tuner = tvm.autotvm.tuner.XGBTuner(tsk, loss_type='rank')
if os.path.isfile(tmp_log_file):
tuner.load_history(tvm.autotvm.record.load_from_file(tmp_log_file))
# do tuning
tsk_trial = min(n_trial, len(tsk.config_space))
def tune_task_fn():
iol = tornado.ioloop.IOLoop()
tuner.tune(
n_trial=2000,
early_stopping=600,
measure_option=tvm.autotvm.measure_option(
builder=tvm.autotvm.LocalBuilder(timeout=10),
runner=tvm.autotvm.LocalRunner(number=20, repeat=3, timeout=4, min_repeat_ms=150)),
callbacks=[
tvm.autotvm.callback.progress_bar(tsk_trial, prefix=prefix),
tvm.autotvm.callback.log_to_file(tmp_log_file)
])
t = threading.Thread(target=tune_task_fn)
t.start()
t.join()
This looks a bit clumsy but seems to work (maybe it can be helpful to others). Of course, one easy remedy would be to do the thread + ioloop dance in the autotuner itself if there is an ioloop already running in the current thread.
Best regards
Thomas