Missing tflite operators

We’ve been trying to run some internal pre-quantized models with the tflite frontend and ran into the following missing operators in the tflite frontend. We’d like to add support for these and see if there are others in the community who are interested in this activity to prevent any duplication of effort.

SPLIT
FAKE_QUANT
PACK
BATCH_TO_SPACE_ND
SPACE_TO_BATCH_ND
RELU
TANH
UNPACK
TRANSPOSE
GREATER
CAST
L2_NORMALIZATION
ZEROS_LIKE
TILE
SHAPE
EXPAND_DIMS
SUM
TRANSPOSE_CONV
SLICE
REDUCE_MAX

Reaching out to others who I’ve seen ask here for other such operators and to figure out if there is common interest. @sreeharsha @FrozenGene @apivovarov , please comment.

Hi Ramana
REDUCE_MAX op was recently added to tflite frontend.

Are your models open-sourced? Can we download the models?

Hi Alexander,

Thanks for your response. Ah just saw the support for REDUCE_MAX. Let me investigate why this is failing for us with operator unsupported again.

Sorry no our models aren’t open sourced. Would you know of any tools like creduce to create smaller models that could be used as testcases ?

Ramana

Sorry about the noise on REDUCE_MAX, looks like I was looking at an older set of results. Yes I now see REDUCE_MAX seems to work.

Hi @apivovarov
I am a newbie to TVM and interested in learning how to add operators for the frontend. I’d like to know where can I refer to the implementation of the missing/unsupported operators, for example, REDUCE_MAX for the tflite frontend. I would appreciate if you can point me some directions, thanks.

you can search in git log history and look at the corresponding PR

1 Like

Thanks for the suggestions :slight_smile:

Anyone interested in adding PRELU (id 54) to tflite frontend?

I have custom tflite model which uses it.

https://raw.githubusercontent.com/tensorflow/tensorflow/r1.13/tensorflow/lite/schema/schema.fbs

PRELU Description https://github.com/keras-team/keras/blob/master/keras/layers/advanced_activations.py#L59

@inadob @FrozenGene @cchung100m @ramana-arm

PRelu PR: https://github.com/apache/incubator-tvm/pull/4298

Thank you Zhao!

Another popular operator which is missing in TFLite frontend is Conv2DTranspose. https://www.tensorflow.org/api_docs/python/tf/keras/layers/Conv2DTranspose

I think it should map to TRANSPOSE_CONV = 67

table TransposeConvOptions {
  padding:Padding;
  stride_w:int;
  stride_h:int;
}

seems that conv2d transpose only support nchw. I will have a look

Actually, the model which I can not compile now is palm_detection.tflite

https://github.com/google/mediapipe/blob/master/mediapipe/models/palm_detection.tflite

It uses custom OP Convolution2DTransposeBias from mediapipe

Can we support it by adding 2 ops to Relay graph - Convolution2DTranspose + Bias?

I’m not sure why mediapipe uses custom op Convolution2DTransposeBias. Conv2DTranspose was added to TFLite in version 1.9

I tried to create Keras model with Conv2DTranspose layer and I set use_bias=True and bias_initializer='random_uniform'.

https://keras.io/layers/convolutional/#conv2dtranspose

data = keras.layers.Input(shape=(32, 32, 3))
x = keras.layers.Conv2D(filters=10, kernel_size=(3, 3), strides=(2, 2), padding='same')(data)
x2 = keras.layers.Conv2DTranspose(filters=10, kernel_size=(3, 3), strides=(2, 2), padding='same', 
        use_bias=True,  bias_initializer='random_uniform')(x)

That model can be converted to TFLite model. Keras Conv2DTranspose was converted to two TFLite operators TRANSPOSE_CONV and ADD

TFLite model visualization: (download html and open it): https://www.dropbox.com/s/au39pvmw3fquhm1/conv2d_transpose_bias.tflite.htm?dl=0

Currently we don’t support custom op. But I think if we need it, one possible way is to simulate TFLite logic in our fe parser, but I haven’t investigated whether custom op has some special attrs not covered in our parser framework.

case kTfLiteBuiltinCustom:
    if (custom_name == "Convolution2DTransposeBias") {
        return make_unique<Convolution2DTransposeBiasParser>();
    }
class Convolution2DTransposeBiasParser : public TFLiteOperationParser {
 public:
  Status IsSupported(const TfLiteContext* context,
                     const TfLiteNode* tflite_node,
                     const TfLiteRegistration* registration) final {
    RETURN_IF_ERROR(CheckTensorIsAvailable(context, tflite_node, 1));
    TfLiteTransposeConvParams* tf_options = nullptr;
    RETURN_IF_ERROR(RetrieveCustomInitialData(tflite_node, &tf_options));
    RETURN_IF_ERROR(
        CheckStrides(tf_options->stride_height, tf_options->stride_width));
    return OkStatus();
  }
  Status Parse(const TfLiteNode* tflite_node,
               const TfLiteRegistration* registration, GraphFloat32* graph,
               ObjectReader* reader) final {
    auto* node = graph->NewNode();
    node->operation.type = ToString(OperationType::CONVOLUTION_TRANSPOSED);
    RETURN_IF_ERROR(reader->AddInput(node, 0));
    RETURN_IF_ERROR(reader->AddOutputs(node));

    const auto* params = reinterpret_cast<const TfLiteTransposeConvParams*>(
        tflite_node->custom_initial_data);
    ConvolutionTransposedAttributes attr;
    attr.stride =
        params ? HW(params->stride_height, params->stride_width) : HW(1, 1);

    RETURN_IF_ERROR(reader->ReadTensor(1, &attr.weights));
    reader->ReadTensor(2, &attr.bias).IgnoreError();  // bias is optional

    UpdatePadding(params->padding, graph->FindInputs(node->id)[0]->tensor.shape,
                  &attr);

    node->operation.attributes = std::move(attr);
    return OkStatus();
  }
};

transpose_conv_bias.cc uses

Input Tensors:
  DataInputTensor
  WeightsTensor
  BiasTensor
TransposeConvBias
  input_shape
  filter_shape
  bias_shape

  ConvParams
    stride_width
    stride_height
    padding_type {SAME, VALID}
    padding_values.width
    padding_values.height

palm_detection.tflite visualization is here

Convolution2DTransposeBias op does not have any Options

It has 3 input tensors:

327    conv2d_transpose/Kernel    FLOAT32    [256, 2, 2, 256]    162    None
328    conv2d_transpose/Bias      FLOAT32    [256]               163    None
329    conv2d_transpose           FLOAT32    [1, 16, 16, 256]      0    None

I think we also need to think about moving up to a newer version than tf 1.13 especially if we need newer operators.

Ramana

TFLite not has style_transfer models. When I tried to compile style_transfer_quantized_dynamic.tflite I got the following missing operators error:

tvm.error.OpNotImplemented: 
The following operators are not supported in frontend TFLite: 
'SQUARED_DIFFERENCE', 
'STRIDED_SLICE', 
'RSQRT', 
'SHAPE', 
'MIRROR_PAD'

Anyone wants to pick particular operator to implement?