[SOLVED] C++ example for OpenCL float16 calculation on Mali-T860

howave · October 26, 2018, 2:17am

I convert the model and params to float16 format,

and set some parameters for float16 calculation.

int dtype_code = kDLFloat;
int dtype_bits = 16;
int dtype_lanes = 1;
int device_type = kDLOpenCL;
int device_id = 0;

then uses float2half and half2float for data conversion,

TVMArrayCopyFromBytes(x, data_x, total_inputsizeof(uint16_t));
TVMArrayCopyToBytes(y, data_y, total_outputsizeof(uint16_t));

float32 results are verified according to the example but the float16 results by above settings are wrong.

Could you have some guides？Thanks!

merrymercy · October 21, 2018, 7:34am

The correctness of fp16 in python is verified here https://github.com/merrymercy/tvm-mali/issues/3#issuecomment-371849500.

@qhou also verified fp16 on mali (in C++ API?). Maybe he can share some thoughts.

howave · October 21, 2018, 7:51am

Yes, I also verified the correctness of our fp16 models in python.
We also need C++ API for deployment

howave · October 22, 2018, 8:06am

It works now, there is something wrong with my float/half conversion functions.

merrymercy · October 24, 2018, 4:32am

Nice, you can also use these functions for float/half conversion.

github.com

dmlc/tvm/blob/master/src/runtime/builtin_fp16.cc

/*!
 * Copyright (c) 2018 by Contributors
 * \file builtin_fp16.cc
 * \brief Functions for conversion between fp32 and fp16
*/
#include <builtin_fp16.h>
#include <tvm/runtime/c_runtime_api.h>

extern "C" {

// disable under msvc
#ifndef _MSC_VER

TVM_WEAK uint16_t __gnu_f2h_ieee(float a) {
  return __truncXfYf2__<float, uint32_t, 23, uint16_t, uint16_t, 10>(a);
}

TVM_WEAK float __gnu_h2f_ieee(uint16_t a) {
  return __extendXfYf2__<uint16_t, uint16_t, 10, float, uint32_t, 23>(a);
}

This file has been truncated. show original

qhou · October 26, 2018, 3:57am

We implemented float32 to float16 in assembly for aarch64. It works well.

jinfagang · March 28, 2022, 2:14am

@merrymercy Hi, can you elaborate how to call this function in C++? I have a float* a, and I assigned values to it. How should I copy these float32 data to a float16 DLTensor?

many thanks if there any help

lizhengwei1992 · August 18, 2022, 11:27am

you can use the function like:

        out[indexOut] = (int)(__gnu_h2f_ieee(*(outputData + indexOutputData)));