Topi convolution

giuseros · July 19, 2019, 10:56am

Hi again,
I am still through the process of learning tvm, and what I want to do now is use a topi convolution (for Mali) and optimize it through the tuner.

This is the code that I am using:

@autotvm.template
def topi_conv(Wi, Hi, C, Ww, Hw, Nw):

    X = tvm.placeholder((1, Hi, Wi, C), name='X')
    W = tvm.placeholder((Hw, Ww, C, Nw), name='W')
    cfg = autotvm.get_config()
    print(cfg)
    out = topi.mali.conv2d(X, W, (1,1), 'SAME', (1,1), layout='NCHW')
    ts = topi.generic.schedule_conv2d_nchw(cfg, [out])

    return ts, [X, W, C]


task = autotvm.task.create(topi_conv, args=(N, M, C, n, m, num_filters), target=target, target_host=target_host)

Unfortunately, I am seeing the following error:

Traceback (most recent call last):
  File "/local/eclipse/plugins/org.python.pydev.core_6.5.0.201809011628/pysrc/pydevd.py", line 1767, in <module>
    main()
  File "/local/eclipse/plugins/org.python.pydev.core_6.5.0.201809011628/pysrc/pydevd.py", line 1752, in main
    debugger.connect(host, port)
  File "/local/eclipse/plugins/org.python.pydev.core_6.5.0.201809011628/pysrc/pydevd.py", line 1108, in run
    return self._exec(is_module, entry_point_fn, module_name, file, globals, locals)
  File "/local/eclipse/plugins/org.python.pydev.core_6.5.0.201809011628/pysrc/pydevd.py", line 1115, in _exec
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/local/eclipse/plugins/org.python.pydev.core_6.5.0.201809011628/pysrc/_pydev_imps/_pydev_execfile.py", line 25, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/home/giuros01/tvm_investigation/optimize_convolution_topi.py", line 34, in <module>
    task = autotvm.task.create(topi_conv, args=(N, M, C, n, m, num_filters), target=target, target_host=target_host)
  File "/home/giuros01/tvm/python/tvm/autotvm/task/task.py", line 191, in create
    sch, _ = func(*args)
  File "<decorator-gen-293>", line 2, in config_dispatcher
  File "/home/giuros01/tvm/python/tvm/autotvm/task/dispatcher.py", line 220, in dispatch_func
    return dispatch_dict[cfg.template_key](cfg, *args, **kwargs)
  File "/home/giuros01/tvm/python/tvm/autotvm/task/task.py", line 296, in template_call
    return func(*args, **kwargs)
  File "/home/giuros01/tvm_investigation/optimize_convolution_topi.py", line 28, in topi_conv
    out = topi.mali.conv2d(X, W, (1,1), 'SAME', (1,1), layout='NCHW')
  File "<decorator-gen-134>", line 2, in conv2d
  File "/home/giuros01/tvm/python/tvm/target.py", line 372, in dispatch_func
    return dispatch_dict[k](*args, **kwargs)
  File "<decorator-gen-281>", line 2, in config_dispatcher
  File "/home/giuros01/tvm/python/tvm/autotvm/task/dispatcher.py", line 220, in dispatch_func
    return dispatch_dict[cfg.template_key](cfg, *args, **kwargs)
KeyError: ''

I am not sure what I am doing wrong. It looks like the configuration does not get initialized, but I thought it was done automatically. If not, how do I initialize it?

Thank you in advance for any help,
Giuseppe

giuseros · July 19, 2019, 2:36pm

After some suggestions from github, I am now running the following:

@autotvm.template
def topi_conv(Wi, Hi, C, Ww, Hw, Nw):
    X = tvm.placeholder((1, Hi, Wi, C), name='X')
    W = tvm.placeholder((Hw, Ww, C, Nw), name='W')
    out = topi.mali.conv2d(X, W, (1,1), 'SAME', (1,1), layout='NCHW')
    ts = topi.generic.schedule_conv2d_nchw([out])

    return ts, [X, W, out]

X = tvm.placeholder((1, N, M, C), name='X')
W = tvm.placeholder((n, m, C, num_filters), name='W')
args = [X, W, (1,1), 'SAME', (1,1), 'NCHW', 'float32']
env = autotvm.task.topi_integration.TaskExtractEnv.get()
sargs = autotvm.task.topi_integration.serialize_args(args)
task = autotvm.task.create('topi_nn_conv2d', sargs, target=target, target_host=target_host,template_key='direct')

measure_option = autotvm.measure_option(
    builder=autotvm.LocalBuilder(),
    runner=autotvm.RPCRunner(device_key, host=opencl_device_tracker, port=9190, number=5, timeout=5))
tuner = autotvm.tuner.XGBTuner(task)

tuner.tune(n_trial=10,
           measure_option=measure_option,
           callbacks=[ autotvm.callback.progress_bar(10, prefix='convolution'),
                       autotvm.callback.log_to_file('convolution.log')])
remote = autotvm.measure.request_remote(device_key, opencl_device_tracker, opencl_device_port, timeout=10000)

with autotvm.apply_history_best('convolution.log'):
    ctx = remote.context(target, 0)
    with tvm.target.create(target):
        s, arg_bufs = topi_conv(N, M, C, n, m, num_filters)
        func = tvm.build(s, arg_bufs, target=target, target_host=target_host)

The error is gone, but I receive a warning:
Cannot find config for target=opencl -device=mali, workload=('topi_conv', 1024, 1024, 3, 3, 3, 4). A fallback configuration is used, which may bring great performance regression.

There is still something strange here, like I am not able to link against the right configuration.

Thanks,
Giuseppe

vinx13 · July 19, 2019, 2:39pm

Remove @autotvm.template decorator in topi_conv and it should work (the warning said that it cannot find a log for this tunning task)

giuseros · July 19, 2019, 2:47pm

If I remove it, I get:
Cannot find config for target=opencl -device=mali, workload=('conv2d', (1, 1024, 1024, 3, 'float32'), (3, 3, 3, 4, 'float32'), (1, 1), 'SAME', (1, 1), 'NCHW', 0). A fallback configuration is used, which may bring great performance regression.

Thanks again,
Giuseppe

vinx13 · July 19, 2019, 2:48pm

Not sure what happened, but if you look at you tuning log, you should be able find the workload in this warning

giuseros · July 19, 2019, 2:52pm

This is one line from the log file:
{"i": ["opencl -device=mali", "topi_nn_conv2d", [["TENSOR", [1, 1024, 1024, 3], "float32"], ["TENSOR", [3, 3, 3, 4], "float32"], [1, 1], "SAME", [1, 1], "NCHW", "float32"], {}, ["conv2d", [1, 1024, 1024, 3, "float32"], [3, 3, 3, 4, "float32"], [1, 1], "SAME", [1, 1], "NCHW", "float32"], {"i": 30521, "e": [["tile_co", "sp", [1, 1, 3]], ["tile_oh", "sp", [2, 512, 1]], ["tile_ow", "sp", [1, 3, 1]], ["reorder_0", "re", [0, 1, 2, 3, 4, 5, 6, 9, 7, 8]], ["ann_reduce", "an", ["none", "unroll"]], ["ann_spatial", "an", ["none", "vec", "none"]]], "t": "direct", "c": null}], "r": [[1000000000.0], 1, 0.03088688850402832, 1563547226.6584065], "v": 0.1}

I don’t understand how to make the auto-tuner look for topi_nn_conv2d

giuseros · July 19, 2019, 2:57pm

Got IT!

After a bit of trial and error, this is the code I came up with:
X = tvm.placeholder((1, N, M, C), name=‘X’)
W = tvm.placeholder((n, m, C, num_filters), name=‘W’)
args = [X, W, (1,1), ‘SAME’, (1,1), ‘NCHW’, ‘float32’]
env = autotvm.task.topi_integration.TaskExtractEnv.get()
sargs = autotvm.task.topi_integration.serialize_args(args)
task = autotvm.task.create(‘topi_nn_conv2d’, sargs, target=target, target_host=target_host,template_key=‘direct’)

measure_option = autotvm.measure_option(
    builder=autotvm.LocalBuilder(),
    runner=autotvm.RPCRunner(device_key, host=opencl_device_tracker, port=9190, number=5, timeout=5))
tuner = autotvm.tuner.XGBTuner(task)
  
tuner.tune(n_trial=10,
           measure_option=measure_option,
           callbacks=[ autotvm.callback.progress_bar(10, prefix='convolution'),
                       autotvm.callback.log_to_file('convolution.log')])
  
remote = autotvm.measure.request_remote(device_key, opencl_device_tracker, opencl_device_port, timeout=10000)
 
with autotvm.apply_history_best('convolution.log'):
    ctx = remote.context(target, 0)
    with tvm.target.create(target):
        out = topi.nn.conv2d(X, W, (1,1), 'SAME', (1,1), layout='NCHW', out_dtype='float32')
        ts = topi.generic.schedule_conv2d_nchw([out])
        func = tvm.build(ts, (X, W, out), target=target, target_host=target_host)

I guess the issue was that I was not passing the output (“out”) to the tvm.build function

giuseros · July 19, 2019, 3:04pm

Soory Wuwei, I spoke too early.

I don’t know why, with the exact same code, I still have the same error:
Cannot find config for target=opencl -device=mali, workload=('topi_conv', 1024, 1024, 3, 3, 3, 4). A fallback configuration is used, which may bring great performance regression.

giuseros · July 19, 2019, 3:43pm

It is all very strange.

Just to recap. This is the code I am running:

X = tvm.placeholder((1, N, M, C), name='X')
W = tvm.placeholder((n, m, C, num_filters), name='W')
args = [X, W, (1,1), 'SAME', (1,1), 'NCHW', 'float32']
env = autotvm.task.topi_integration.TaskExtractEnv.get()
sargs = autotvm.task.topi_integration.serialize_args(args)
task = autotvm.task.create('topi_nn_conv2d', sargs, target=target, target_host=target_host,template_key='direct')
measure_option = autotvm.measure_option(
    builder=autotvm.LocalBuilder(),
    runner=autotvm.RPCRunner(device_key, host=opencl_device_tracker, port=9190, number=5, timeout=5))
tuner = autotvm.tuner.XGBTuner(task)
    
tuner.tune(n_trial=10,
           measure_option=measure_option,
           callbacks=[ autotvm.callback.progress_bar(10, prefix='convolution'),
                       autotvm.callback.log_to_file('convolution.log')])
   
remote = autotvm.measure.request_remote(device_key, opencl_device_tracker, opencl_device_port, timeout=10000)
 
with autotvm.apply_history_best('convolution.log'):
    ctx = remote.context(target, 0)
    with tvm.target.create(target):
        out = topi.nn.conv2d(X, W, (1,1), 'SAME', (1,1), 'NCHW','float32')
        ts = topi.generic.schedule_conv2d_nchw([out])
        func = tvm.build(ts, (X, W, out), target=target, target_host=target_host)

This is a sample from convolution.log:
{"v": 0.1, "r": [[1000000000.0], 4, 7.2558629512786865, 1563549693.7079325], "i": ["opencl -device=mali", "topi_nn_conv2d", [["TENSOR", [1, 1024, 1024, 3], "float32"], ["TENSOR", [3, 3, 3, 4], "float32"], [1, 1], "SAME", [1, 1], "NCHW", "float32"], {}, ["conv2d", [1, 1024, 1024, 3, "float32"], [3, 3, 3, 4, "float32"], [1, 1], "SAME", [1, 1], "NCHW", "float32"], {"e": [["tile_co", "sp", [1, 1, 3]], ["tile_oh", "sp", [32, 16, 2]], ["tile_ow", "sp", [1, 3, 1]], ["reorder_0", "re", [0, 1, 2, 3, 4, 5, 6, 9, 7, 8]], ["ann_reduce", "an", ["none", "unroll"]], ["ann_spatial", "an", ["vec", "unroll", "none"]]], "c": null, "t": "direct", "i": 87563}]}

And this is the warning I am getting:
Cannot find config for target=opencl -device=mali, workload=('conv2d', (1, 1024, 1024, 3, 'float32'), (3, 3, 3, 4, 'float32'), (1, 1), 'SAME', (1, 1), 'NCHW', 'float32'). A fallback configuration is used, which may bring great performance regression.

As you can see, there is a conv_2d entry in the log file. I don’t understand how I can make it read it.

giuseros · July 19, 2019, 4:08pm

Ok, I am making small progress.

If you see in the log file, there is a column with all 4s and 1s. Those are errors, here is the list:

class MeasureErrorNo(object):
    """Error type for MeasureResult"""
    NO_ERROR = 0              # no error
    INSTANTIATION_ERROR = 1   # actively detected error in instantiating a template with a config
    COMPILE_HOST = 2          # error when compiling code on host (e.g. tvm.build)
    COMPILE_DEVICE = 3        # error when compiling code on device (e.g. OpenCL JIT on the device)
    RUNTIME_DEVICE = 4        # error when run program on device
    WRONG_ANSWER = 5          # answer is wrong when compared to a golden output
    BUILD_TIMEOUT = 6         # timeout during compilation
    RUN_TIMEOUT = 7           # timeout during run
    UNKNOWN_ERROR = 8         # unknown error

I guess something is wrong on the device (it is either RUNTIME_DEVICE or INSTANTIATION_ERROR), but other examples ran fine. It is still all very strange.

vinx13 · July 19, 2019, 4:19pm

The workload in your log is okay. I guess this is related to tuning error, did you get a reasonable GFLOPS after tuning? If so there should be at least one valid log entry

giuseros · July 19, 2019, 4:23pm

Apparently I was using too much memory.

Although it is strange: I am using 1024x1024x3 float32 inputs which amount to 16 MB. Even 10 times this is still way under the maximum memory on the board.

Anyway, as soon as I moved 1024 to 128, it gave valid entries, although some of them are still invalid (with an INSTANTIATION_ERROR). Is it possible that there are transformations for which too much memory is used?

Thanks,
Giuseppe

roastduck · August 5, 2019, 7:59am

Hi giuseros,

Have you figured out why your most original code not working? I’m encountering the same error.

Here’s my code:

import logging
import sys

import numpy as np
import tvm
import topi
from tvm import autotvm

@autotvm.template
def conv(N, Cin, Cout, H, W, K, strides, padding, dilation):
    data = tvm.placeholder((N, Cin, H, W))
    kernel = tvm.placeholder((Cout, Cin, K, K))

    with tvm.target.create("cuda"):
        out = topi.nn.conv2d(data, kernel, strides=strides, padding=padding, dilation=dilation)
        schedule = topi.generic.nn.schedulre_conv2d_nchw([out])

    return schedule, [data, kernel, out]

N, Cin, Cout, H, W, K = 1, 3, 10, 224, 224, 5
strides = 1
padding = 2
dilation = 1
task = autotvm.task.create(conv, args=(N, Cin, Cout, H, W, K, strides, padding, dilation), target='cuda')
print(task.config_space)

And the error is:

Traceback (most recent call last):
  File "tune_conv.py", line 24, in <module>
    task = autotvm.task.create(conv, args=(N, Cin, Cout, H, W, K, strides, padding, dilation), target='cuda')
  File "/home/v-shizta/experiments/tvm/tvm/python/tvm/autotvm/task/task.py", line 191, in create
    sch, _ = func(*args)
  File "</home/v-shizta/.local/lib/python3.5/site-packages/decorator.py:decorator-gen-185>", line 2, in config_dispatcher
  File "/home/v-shizta/experiments/tvm/tvm/python/tvm/autotvm/task/dispatcher.py", line 220, in dispatch_func
    return dispatch_dict[cfg.template_key](cfg, *args, **kwargs)
  File "/home/v-shizta/experiments/tvm/tvm/python/tvm/autotvm/task/task.py", line 296, in template_call
    return func(*args, **kwargs)
  File "tune_conv.py", line 15, in conv
    out = topi.nn.conv2d(data, kernel, strides=strides, padding=padding, dilation=dilation)
  File "</home/v-shizta/.local/lib/python3.5/site-packages/decorator.py:decorator-gen-23>", line 2, in conv2d
  File "/home/v-shizta/experiments/tvm/tvm/python/tvm/target.py", line 372, in dispatch_func
    return dispatch_dict[k](*args, **kwargs)
  File "</home/v-shizta/.local/lib/python3.5/site-packages/decorator.py:decorator-gen-135>", line 2, in config_dispatcher
  File "/home/v-shizta/experiments/tvm/tvm/python/tvm/autotvm/task/dispatcher.py", line 220, in dispatch_func
    return dispatch_dict[cfg.template_key](cfg, *args, **kwargs)
KeyError: ''

What does the template_key argument of autotvm.task.create mean? It seems undocumented.

Plus, why were you calling topi.generic.schedule_conv2d_nchw(cfg, [out])? I thought this function only accept one argument [out].

Thanks.