[SOLVED] Failed to compile models for ARMv7 Mali GPU with opt_level 3

apivovarov · October 6, 2018, 5:04am

I’ve got ASUS Tinker Board S
Quad-core Cortex-A17 @1.8 GHz (ARMv7)
Quad-core ARM Mali-T760 MP4 GPU @600MHz (OpenCL 1.1)
2GB RAM

I tried to compile resnet18_v1 and squeezenet1.1 for Mali GPU (ARMv7) using the latest tvm code from github.
The compilation and running of the compiled models work fine for opt_level 1 and 2!
But if I switch to opt_level 3 I start getting error - Target shape size is different to source

Compilation code:

gist.github.com

https://gist.github.com/apivovarov/72a0584f0a47133a96619c69c8a34e77

compile-cl.py

#!/usr/bin/env python3

import nnvm
import tvm
import numpy as np

from mxnet.gluon.model_zoo.vision import get_model

batch_size = 1
image_shape = (3, 224, 224)

This file has been truncated. show original

resnet18_v1 compilation output:
(147456 vs. 51380224) Target shape size is different to source. Target: [6,6,16,4,64]

gist.github.com

https://gist.github.com/apivovarov/084bb2fec28d90c561be3ed25f4e850b

resnet18_v1.out

target: opencl -device=mali -model=unknown , opt_level: 3 , data_shape: (1, 3, 224, 224)
Compiling...
Traceback (most recent call last):
  File "./compile-cl.py", line 32, in <module>
    graph, lib, params = nnvm.compiler.build(sym, target, shape={"data": data_shape}, params=params)
  File "/usr/local/lib/python3.5/dist-packages/nnvm-0.8.0-py3.5.egg/nnvm/compiler/build_module.py", line 281, in build
    graph = optimize(graph, shape, dtype, layout)
  File "/usr/local/lib/python3.5/dist-packages/nnvm-0.8.0-py3.5.egg/nnvm/compiler/build_module.py", line 176, in optimize
    graph = graph.apply(["InferShape", "SimplifyInference"])
  File "/usr/local/lib/python3.5/dist-packages/nnvm-0.8.0-py3.5.egg/nnvm/graph.py", line 234, in apply

This file has been truncated. show original

squeezenet1.1 compilation output
(262144 vs. 205520896) Target shape size is different to source. Target: [4,4,64,4,64]

gist.github.com

https://gist.github.com/apivovarov/cff3565a9d2d741d26b8b69629d25b7a

squeezenet1.1-output

target: opencl -device=mali -model=unknown , opt_level: 3 , data_shape: (1, 3, 224, 224)
Compiling...
Cannot find config for target=opencl -device=mali -model=unknown, workload=('conv2d', (1, 3, 224, 224, 'float32'), (64, 3, 3, 3, 'float32'), (2, 2), (0, 0), 'NCHW', 'float32'). A fallback configuration is used, which may bring great performance regression.
Traceback (most recent call last):
  File "./compile-cl.py", line 30, in <module>
    graph, lib, params = nnvm.compiler.build(sym, target, shape={"data": data_shape}, params=params)
  File "/usr/local/lib/python3.5/dist-packages/nnvm-0.8.0-py3.5.egg/nnvm/compiler/build_module.py", line 281, in build
    graph = optimize(graph, shape, dtype, layout)
  File "/usr/local/lib/python3.5/dist-packages/nnvm-0.8.0-py3.5.egg/nnvm/compiler/build_module.py", line 176, in optimize
    graph = graph.apply(["InferShape", "SimplifyInference"])

This file has been truncated. show original

The host platform is 32 bit ARMv7 (maybe int precision on 32 bit platform is the root cause of the issue)

apivovarov · October 4, 2018, 12:08am

The solution here is to compile the models for ASUS Tinker Board S on regular x86_64 Ubuntu box

target = tvm.target.mali()
target_host = 'llvm -device=arm_cpu -model=rk3288 -target=armv7l-linux-gnueabihf'