Good to hear about Hexagon with open source DL runtime apart from Qualcomm’s in house NPE SDK.
Hexagon runtime with quantization and heterogeneous execution TVM can be a great value for Qualcomm platforms.
Thanks for the great work!
I think we could use macro to control alignment. For example Tianqi’s response ‘USE_HEXAGON’. If defined this macro, we make them be 128, if not be 64 as original.
I agree with Tianqi’s suggestions. The kHexagonDeviceAPI is import and isolated. Then we could add codegen part, then we could run. Finally we could have topi schedule.
The most important thing is show how we run end-to-end model. For example mobilenet.
I have one quick question: Why we restrict Android? Hexagon SDK has Linux toolchain too. Because our dsp will run on Linux.
1 Like
Thanks for the RFC! Just for my self-education purpose, what is the theoretical peak FLOPS of Hexagon and what is your observed FLOPS when running CONV on it?
2 Likes
For consistency reasons, I think it is fine to increase the alignment requirement by default. Note that this will only cause a bit fragmentation for small allocas, but usually most temp allocation is big.
However, in GetTempAllocaAlignment function, it will reduce alignment / 2 in while loop. I am worried about this will affect DSP’s rule, it requires 128 bits alignment.
OK that makes sense and we might need a special target aware function for that
GetTempAllocaAlignment function maybe should be modified for special DSP target and return 128(kTempAllocAlignment) directly.
1 Like
I’m working on some changes and will have patches for review soon.
2 Likes
Patch #1 is ready. This is the runtime support for Hexagon: https://github.com/dmlc/tvm/pull/3163
1 Like
It seems the support for Hexagon CDSP is not ready now. Is there any plan, e.g. to add the support for Hexagon CDSP in the last release of TVM.
Another question, besides CDSP, there is also a dedicated NPU in SDM8150, will Qualcomm add the TVM support for NPU in the future?
2 Likes
There was a delay caused by the need for another legal review (since there were comments on the license headers in the patch).
I will post an updated patch within a few days.
hello ,I briefly looked at the code.I have a question.you use LLVM to generate Hexagon code from TVM IR.Why add a Hexagon target directly instead of adding it under LLVM like X86 or ARM?
1 Like
This patch only contains runtime support for Hexagon. There is codegen also, but it’s not included here (and it is in src/codegen/llvm)
My current problem is that I don’t see special X86 or ARM support in the runtime’s code.
The special runtimes are mostly for offloading to extra devices from host. ARM and X86 are usually the hosts, so the runtime for them is fairly simple. It’s still there, look at cpu_device_api.cc for example.
2 Likes
I think if adding copyonright within hexagon code or not will not be a problem cause that these code will only used on Qualcomm soc.
Maybe a little bit later. But I think there is one slide is very good: http://pages.cs.wisc.edu/~danav/pubs/qcom/hexagon_hotchips2013.pdf
And Hexagon DSP has good support for fixed-int computation, but not be suitable for float arithmetic. Like hexagon has some instructions can be very useful for convolution, for example vrmpy, which could be used for dot.
And according to my test, SNPE on 8155 Hexagon could execute mobilenet v2 quant model in less than 5ms.
One more thing, thanks for the help and discussion of @kparzysz, based on his contributed runtime pr, I have completed TVM support of hexagon (LLVM hexagon CodeGen, schedule, hexagon parallel support and so on), but I must admit that we have big gap compared with SNPE, I am working on it. When time is suitable and follow up @kparzysz, I could contribute back to upstream.
4 Likes
Can you show me the code implementation of Hexgon’s Codegen?
I have a question about Licensing. PR 3163 (Initial implementation of Hexagon runtime support) proposes to add new build option USE_HEXAGON_SDK which is required for Hexagon DSP support
tvm_option(USE_HEXAGON_SDK "Path to the Hexagon SDK root (required for Hexagon support)" /path/to/sdk)
Hexagon SDK is not open-source software. It has particular License
Question is: Can a company use TVM which is built with HEXAGON_SDK in a commercial publicly available web service?