[Solved][Hybrid script] Compilation error when passing list argument

#1

In hybrid version of multi box_prior: https://github.com/kevinthesun/tvm/blob/RefactorSSDOperator/topi/python/topi/vision/ssd/multibox.py#L13-L104

It runs fine with numpy input, but got compilation error while building:

Traceback (most recent call last):
  File "test_topi_vision.py", line 262, in <module>
    test_multibox_prior()
  File "test_topi_vision.py", line 163, in test_multibox_prior
    verify_multibox_prior((1, 3, 50, 50))
  File "test_topi_vision.py", line 159, in verify_multibox_prior
    check_device(device)
  File "test_topi_vision.py", line 154, in check_device
    f = tvm.build(s, [data, out], device)
  File "/Users/wayao/anaconda/lib/python2.7/site-packages/tvm-0.5.dev0-py2.7-macosx-10.6-x86_64.egg/tvm/build_module.py", line 536, in build
    binds=binds)
  File "/Users/wayao/anaconda/lib/python2.7/site-packages/tvm-0.5.dev0-py2.7-macosx-10.6-x86_64.egg/tvm/build_module.py", line 348, in lower
    stmt = ir_pass.StorageFlatten(stmt, binds, 64, cfg.instrument_bound_checkers)
  File "tvm/_ffi/_cython/function.pxi", line 286, in core.FunctionBase.__call__ (tvm/_ffi/_cython/core.cpp:6689)
  File "tvm/_ffi/_cython/function.pxi", line 231, in core.FuncCall (tvm/_ffi/_cython/core.cpp:5961)
  File "tvm/_ffi/_cython/base.pxi", line 143, in core.CALL (tvm/_ffi/_cython/core.cpp:1670)
tvm._ffi.base.TVMError: [18:24:45] /Users/wayao/Documents/tvm/src/pass/storage_flatten.cc:111: Check failed: it != buf_map_.end() Cannot find allocated buffer for placeholder(w, 0x10fa767f0)

It looks like variable w is falsely identified as a tensor buffer?
Here is the test: https://github.com/kevinthesun/tvm/blob/RefactorSSDOperator/topi/tests/python/test_topi_vision.py#L103-L165

@were

#2

dai wo yi yue, then i can debug.

#3

wo de guo, const_range cannot allocate buffer defined inside properly.
i have opened a pr 2 fix this.

#4

Getting another issue. In test case https://github.com/kevinthesun/tvm/blob/RefactorSSDOperator/topi/tests/python/test_topi_vision.py#L164, the index of const_range will unexpectedly goes out of range in if statement. In debugging mode, I feed in numpy and it can generate correct result.

  File "test_topi_vision.py", line 262, in <module>
    test_multibox_prior()
  File "test_topi_vision.py", line 164, in test_multibox_prior
    verify_multibox_prior((1, 3, 224, 224), sizes=(0.5, 0.25, 0.1), ratios=(1, 2, 0.5))
  File "test_topi_vision.py", line 159, in verify_multibox_prior
    check_device(device)
  File "test_topi_vision.py", line 147, in check_device
    out = ssd.multibox_prior(data, sizes, ratios, steps, offsets, clip)
  File "<decorator-gen-107>", line 2, in multibox_prior
  File "/Users/wayao/anaconda/lib/python2.7/site-packages/tvm-0.5.dev0-py2.7-macosx-10.6-x86_64.egg/tvm/target.py", line 357, in dispatch_func
    return func(*args, **kwargs)
  File "/Users/wayao/anaconda/lib/python2.7/site-packages/topi-0.5.dev0-py2.7.egg/topi/vision/ssd/multibox.py", line 107, in multibox_prior
    tvm.convert(steps), tvm.convert(offsets))
  File "<decorator-gen-106>", line 2, in hybrid_multibox_prior
  File "/Users/wayao/anaconda/lib/python2.7/site-packages/tvm-0.5.dev0-py2.7-macosx-10.6-x86_64.egg/tvm/hybrid/api.py", line 27, in wrapped_func
    parser = parse_python(src, func.__globals__, args)
  File "/Users/wayao/anaconda/lib/python2.7/site-packages/tvm-0.5.dev0-py2.7-macosx-10.6-x86_64.egg/tvm/hybrid/parser.py", line 514, in parse_python
    parser.parsed_body = parser.visit(root)
  File "/Users/wayao/anaconda/lib/python2.7/ast.py", line 241, in visit
    return visitor(node)
  File "/Users/wayao/anaconda/lib/python2.7/site-packages/tvm-0.5.dev0-py2.7-macosx-10.6-x86_64.egg/tvm/hybrid/parser.py", line 157, in visit_Module
    return self.visit(node.body[0])
  File "/Users/wayao/anaconda/lib/python2.7/ast.py", line 241, in visit
    return visitor(node)
  File "/Users/wayao/anaconda/lib/python2.7/site-packages/tvm-0.5.dev0-py2.7-macosx-10.6-x86_64.egg/tvm/hybrid/parser.py", line 169, in visit_FunctionDef
    res = visit_list_to_block(self.visit, node.body)
  File "/Users/wayao/anaconda/lib/python2.7/site-packages/tvm-0.5.dev0-py2.7-macosx-10.6-x86_64.egg/tvm/hybrid/parser.py", line 37, in visit_list_to_block
    lst = [visit(stmt) for stmt in lst if not util.is_docstring(stmt)]
  File "/Users/wayao/anaconda/lib/python2.7/ast.py", line 241, in visit
    return visitor(node)
  File "/Users/wayao/anaconda/lib/python2.7/site-packages/tvm-0.5.dev0-py2.7-macosx-10.6-x86_64.egg/tvm/hybrid/parser.py", line 443, in visit_For
    _body = visit_list_to_block(self.visit, node.body)
  File "/Users/wayao/anaconda/lib/python2.7/site-packages/tvm-0.5.dev0-py2.7-macosx-10.6-x86_64.egg/tvm/hybrid/parser.py", line 37, in visit_list_to_block
    lst = [visit(stmt) for stmt in lst if not util.is_docstring(stmt)]
  File "/Users/wayao/anaconda/lib/python2.7/ast.py", line 241, in visit
    return visitor(node)
  File "/Users/wayao/anaconda/lib/python2.7/site-packages/tvm-0.5.dev0-py2.7-macosx-10.6-x86_64.egg/tvm/hybrid/parser.py", line 443, in visit_For
    _body = visit_list_to_block(self.visit, node.body)
  File "/Users/wayao/anaconda/lib/python2.7/site-packages/tvm-0.5.dev0-py2.7-macosx-10.6-x86_64.egg/tvm/hybrid/parser.py", line 37, in visit_list_to_block
    lst = [visit(stmt) for stmt in lst if not util.is_docstring(stmt)]
  File "/Users/wayao/anaconda/lib/python2.7/ast.py", line 241, in visit
    return visitor(node)
  File "/Users/wayao/anaconda/lib/python2.7/site-packages/tvm-0.5.dev0-py2.7-macosx-10.6-x86_64.egg/tvm/hybrid/parser.py", line 434, in visit_For
    bodies.append(visit_list_to_block(self.visit, node.body))
  File "/Users/wayao/anaconda/lib/python2.7/site-packages/tvm-0.5.dev0-py2.7-macosx-10.6-x86_64.egg/tvm/hybrid/parser.py", line 37, in visit_list_to_block
    lst = [visit(stmt) for stmt in lst if not util.is_docstring(stmt)]
  File "/Users/wayao/anaconda/lib/python2.7/ast.py", line 241, in visit
    return visitor(node)
  File "/Users/wayao/anaconda/lib/python2.7/site-packages/tvm-0.5.dev0-py2.7-macosx-10.6-x86_64.egg/tvm/hybrid/parser.py", line 337, in visit_If
    if_body = visit_list_to_block(self.visit, node.body)
  File "/Users/wayao/anaconda/lib/python2.7/site-packages/tvm-0.5.dev0-py2.7-macosx-10.6-x86_64.egg/tvm/hybrid/parser.py", line 37, in visit_list_to_block
    lst = [visit(stmt) for stmt in lst if not util.is_docstring(stmt)]
  File "/Users/wayao/anaconda/lib/python2.7/ast.py", line 241, in visit
    return visitor(node)
  File "/Users/wayao/anaconda/lib/python2.7/site-packages/tvm-0.5.dev0-py2.7-macosx-10.6-x86_64.egg/tvm/hybrid/parser.py", line 224, in visit_Assign
    rhs = self.visit(node.value)
  File "/Users/wayao/anaconda/lib/python2.7/ast.py", line 241, in visit
    return visitor(node)
  File "/Users/wayao/anaconda/lib/python2.7/site-packages/tvm-0.5.dev0-py2.7-macosx-10.6-x86_64.egg/tvm/hybrid/parser.py", line 386, in visit_BinOp
    lhs = self.visit(node.left)
  File "/Users/wayao/anaconda/lib/python2.7/ast.py", line 241, in visit
    return visitor(node)
  File "/Users/wayao/anaconda/lib/python2.7/site-packages/tvm-0.5.dev0-py2.7-macosx-10.6-x86_64.egg/tvm/hybrid/parser.py", line 386, in visit_BinOp
    lhs = self.visit(node.left)
  File "/Users/wayao/anaconda/lib/python2.7/ast.py", line 241, in visit
    return visitor(node)
  File "/Users/wayao/anaconda/lib/python2.7/site-packages/tvm-0.5.dev0-py2.7-macosx-10.6-x86_64.egg/tvm/hybrid/parser.py", line 386, in visit_BinOp
    lhs = self.visit(node.left)
  File "/Users/wayao/anaconda/lib/python2.7/ast.py", line 241, in visit
    return visitor(node)
  File "/Users/wayao/anaconda/lib/python2.7/site-packages/tvm-0.5.dev0-py2.7-macosx-10.6-x86_64.egg/tvm/hybrid/parser.py", line 298, in visit_Subscript
    buf = buf[i]
  File "/Users/wayao/anaconda/lib/python2.7/site-packages/tvm-0.5.dev0-py2.7-macosx-10.6-x86_64.egg/tvm/container.py", line 28, in __getitem__
    .format(len(self), i))
IndexError: Array index out of range. Array size: 3, got index 3
#5

I am 95% confident that zhe ge shi ni xie cuo le, at least bu shi wo de cuo.
You do const_range(num_size + num_ratios - 1), i.e. 3+3-1=5.
Then these 0, 1, 2, 3, 4 access size and ratio, both these 2 values are 3-sized arrays.