The issue exist only for NHWC models!!!
Affected models:
- All
tflitemodels - tensorflow models if you do not add parameter
layout='NCHW'torelay.frontend.from_tensorflow()
I tried to compile non-quantized tflite mobilenet model for android i686
The compilation works fine, but shared library export hangs (_SaveToFile call hangs).
I tried it on Ubuntu 16, 18 and Mac OS.
llvm 6.0.0 and 7.0.1
To reproduce the issue:
non-quantized tflite mobilenet model can be downloaded from https://github.com/tensorflow/models/blob/master/research/slim/nets/mobilenet_v1.md
install tflite package
wget https://raw.githubusercontent.com/dmlc/web-data/master/tensorflow/tflite/whl/tflite-0.0.1-py3-none-any.whl
pip3 install tflite-0.0.1-py3-none-any.whl --user
install android-ndk-r18
# for Mac OS
wget https://dl.google.com/android/repository/android-ndk-r18b-darwin-x86_64.zip
unzip android-ndk-r18b-darwin-x86_64.zip
sudo mv android-ndk-r18b /opt
export TVM_NDK_CC=/opt/android-ndk-r18b/toolchains/x86-4.9/prebuilt/darwin-x86_64/bin/i686-linux-android-g++
# for Linux
wget https://dl.google.com/android/repository/android-ndk-r18b-linux-x86_64.zip
unzip android-ndk-r18b-linux-x86_64.zip
sudo mv android-ndk-r18b /opt
export TVM_NDK_CC=/opt/android-ndk-r18b/toolchains/x86-4.9/prebuilt/linux-x86_64/bin/i686-linux-android-g++
code to compile and export - lib.export_library hangs
#!/usr/bin/env python3
import tvm
from tvm import relay
from tvm.contrib import ndk
import tflite.Model
input_tensor = "input"
input_shape = (1, 224, 224, 3)
input_dtype = "float32"
m_file = "mobilenet_v1_1.0_224.tflite"
print(m_file)
tflite_model_buf = open(m_file, "rb").read()
# get TFLite model from buffer
tflite_model = tflite.Model.Model.GetRootAsModel(tflite_model_buf, 0)
mod, params = relay.frontend.from_tflite(tflite_model,
shape_dict={input_tensor: input_shape},
dtype_dict={input_tensor: input_dtype})
target = "llvm -target=i686-linux-android"
print("target: {}".format(target))
target_host = None
print("target_host: {}".format(target_host))
print("Compiling...")
with relay.build_config(opt_level=3):
graph, lib, params = relay.build(mod[mod.entry_func], target, target_host=target_host, params=params)
print("Compilation done")
print("Saving files")
# save the graph, lib and params into separate files
path_lib = "model.so"
#arch-x86 (i686-linux-android)
sysroot="/opt/android-ndk-r18b/platforms/android-21/arch-x86"
toolchain="/opt/android-ndk-r18b/toolchains/x86-4.9/prebuilt/linux-x86_64"
options=["-shared", "-fPIC", "--sysroot", sysroot, "--gcc-toolchain="+toolchain]
lib.export_library(path_lib, ndk.create_shared, options=options)
# the line above hangs ^^^ (in particular `_SaveToFile` call hangs)
print("export_library done")
with open("model.json", "w") as fo:
fo.write(graph)
with open("model.params", "wb") as fo:
fo.write(relay.save_param_dict(params))
print("Files saved")
I also tried to compile and export the model for other platforms - results are below:
-
aarch64-linux-android- works! -
x86_64-linux-android- works! -
i686-linux-android- hangs!
# aarch64-linux-android - works!
export TVM_NDK_CC=/opt/android-ndk-r18b/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/aarch64-linux-android-g++
target = "llvm -target=aarch64-linux-android"
sysroot="/opt/android-ndk-r18b/platforms/android-21/arch-arm64"
toolchain="/opt/android-ndk-r18b/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64"
# x86_64-linux-android - works!
export TVM_NDK_CC=/opt/android-ndk-r18b/toolchains/x86_64-4.9/prebuilt/linux-x86_64/bin/x86_64-linux-android-g++
target = "llvm -target=x86_64-linux-android"
sysroot="/opt/android-ndk-r18b/platforms/android-21/arch-x86_64"
toolchain="/opt/android-ndk-r18b/toolchains/x86_64-4.9/prebuilt/linux-x86_64"
# i686-linux-android - hangs!
export TVM_NDK_CC=/opt/android-ndk-r18b/toolchains/x86-4.9/prebuilt/linux-x86_64/bin/i686-linux-android-g++
target = "llvm -target=i686-linux-android"
sysroot="/opt/android-ndk-r18b/platforms/android-21/arch-x86"
toolchain="/opt/android-ndk-r18b/toolchains/x86-4.9/prebuilt/linux-x86_64"