Auto-tuning tflite format models on mobile gpu

Hi, I will be appreciate it if I get any hints on this issue. :slight_smile:
I’d like to tune pre-trained tflite models such as mobilenet-v1 from google (https://www.tensorflow.org/lite/guide/hosted_models) on an android mobile device.
But there’s a format issue. (NHWC)

I followed your guide.

  1. I can register and connect the android device.
  2. rpc test is done. (including opencl)
  3. “Auto-tuning a convolutional network for Mobile GPU” in tutorials, the default model is resnet.
    There’s a compile issue while running tune_relay_mobile_gpu.py.
  4. When I change the model as “mobilenet”, no errors but the device is disconnected then try to connect during auto-tuning. It seems working.

So, I modified “tune_relay_mobile_gpu.py” a little bit to load tflite model.

def get_network( batch_size):
output_shape = (batch_size, 1001)
tflite_model_file = os.path.join("./", “mobilenet_v1_1.0_224.tflite”)
tflite_model_buf = open(tflite_model_file, “rb”).read()
import tflite.Model
tflite_model = tflite.Model.Model.GetRootAsModel(tflite_model_buf, 0)
input_tensor = “input”
input_shape = (batch_size, 224, 224, 3)
from tvm import relay
mod, params = relay.frontend.from_tflite(tflite_model,
shape_dict={input_tensor: input_shape}
dtype_dict={input_tensor: dtype})
return mod, params, input_shape, output_shape

Loading is okay but I got an assertion error after that.

File “/home/dongjin/Desktop/fall/tvm/python/tvm/autotvm/task/topi_integration.py”, line 179, in _topi_nn_conv2d
assert layout == ‘NCHW’, “only support NCHW currently”

Because all data formats are NHWC in tflite file… so I’m wondering if there is a way to transpose this NHWC format to the NCHW format before calling autotvm.task.extract_from_program method.

Regards,
nadongguri.

I think your requirement is related to this issue: Layout conversion pass

@comaniac, thank you for the comment.

I guess Layout conversion pass is working in process, right?
I could find “AlterOpLayout” but I’m not sure how to apply it. :sweat_smile:

Yes it’s WIP.
The current AlterOpLayout cannot be easily applied, and that’s why we have that RFC filed.

@tqchen do you have any suggestion about a quick workaround?