Does TVM currently supported for RISC-V？

#1

Hi, does TVM currently supported for RISC-V?
I have saw the presentation Enabling TVM on RISC-V Architectures with SIMD Instructions.
Is it any tutorials for me to learn and practice about it?

#2

Hi,

Is there any update on this? I have seen an example of micro TVM with RISC-V, however, I was wondering if this is supported as well in full-fledged TVM?

#3

AFAIK, TVM upstream doesn’t support RISC-V. The presentation was given by Prof. Lee at National Tsing Hua University, but he doesn’t neither release the implementation nor file a RFC. Not sure about the status of uTVM tho.

#4

@tico I believe @comaniac is right that host-side TVM does not support RISC-V. However, µTVM does support bare-metal RISC-V (see here) and recently became AutoTVM-enabled (see here). The AutoTVM PR name is a bit misleading, because the AutoTVM infrastructure works for all µTVM targets (including RISC-V), but we were focused on getting it working on ARM Cortex-M processors.

We have currently only tested RISC-V support on the Spike simulator, but given that µTVM works on physical ARM boards, it should be straightforward to get it working on a physical RISC-V device.

Our biggest pushes right now are:

  • Enabling vectorization for ARM processors, so we can become competitive with hand-optimized libraries. Once support is there for ARM’s SIMD intrinsics, I’ll start looking at SIMD for RISC-V.
  • Creating a fully self-hosted runtime. Although µTVM supports end-to-end models without control flow, it still relies on the host for allocating tensors in the network.

I’m in the process of writing a blogpost that will have a more detailed description of the µTVM infrastructure, and there will be up-to-date examples of using µTVM for inference and autotuning. Along with the blogpost, I’ll be submitting a tuneup PR filled with bug fixes and quality-of-life improvements. In the meantime, you can always look at the tests for example inference usage.

Let me know if you have any more questions! :slightly_smiling_face:

4 Likes
#5

Hi @weberlo,

Thanks the nice and clear explanation of the current status. I actually saw your uTVM tutorial at FCRC (https://www.youtube.com/watch?v=VUQ9IAK0SR0), which was very helpful :slight_smile:.

I fully agree that the two points that you mentioned as next directions are key for uTVM both for achieving a good performance by means of vectorization and also for making uTVM more practical by allowing it to be fully deployed in production environments without depending on any host.

I am glad to hear that you are writing a blogpost since I have been looking to understand in more detail uTVM. If possible it would be nice if you could document the differences between the full TVM runtime vs uTVM. Also any limitation or part of the existing TVM infrastructure that do no apply or still not supported in uTVM. Finally,

I have a couple of further questions:

  • What targets are already supported in uTVM besides RISC-V?
  • What is the average size of the uTVM runtime + operators?
  • Does uTVM supports all the operators already supported by the Relay frontends?

Thanks!

#6

If possible it would be nice if you could document the differences between the full TVM runtime vs uTVM.

Will do!

What targets are already supported in uTVM besides RISC-V?

ARM, RISC-V, and x86 are the only targets we’ve tested so far.

What is the average size of the uTVM runtime + operators?

The runtime itself is about 1kb, and the operator size can vary quite a bit. As an example, the default schedule for conv2d is about 500 bytes, but tuned schedules can get as large as 10kb, if you allow heavy loop unrolling. Though if you’re resource-constrained, it’s easy to just use a schedule with no unrolling.

Does uTVM supports all the operators already supported by the Relay frontends?

We haven’t done a thorough sweep over operators to see what our coverage is, but I believe our main deficit is in the support of transcendental functions (e.g., tanh, sigmoid, and exp). To include support for those, we’ll need to look into open-source C math libraries, or for lower-precision data types, we could generate tables for each required transcendental.

Hope this helps!

1 Like
#7

Thanks for the answers!

I just missed one important question: Is quantization supported in uTVM?

Looking forward to your blogpost! Keep us posted! :slight_smile:

#8

I haven’t tried importing pre-quantized models or using the Relay quantization tool, but I don’t suspect there are any fundamental limitations.

#9

Looking forward to the self-hosted uTVM example/blog posts.

@weberlo, is it possible to “slice out” runtime related source code from TVM, together with the dumped out device-side c code, throwing them into cross-compiler to generate bare-metal binary?

1 Like
#10

You can easily grab C code for operators, and they will work fine in a bare-metal setting. The runtime part is a bit trickier. Luckily, there’s some work being done right now on a MISRA-C bare-metal runtime (RFC here) that might interest you.

#11

Hi @weberlo,

I know this is an old topic, but can you explain the steps to run source code from TVM on the Spike simulator? Do I set the target to riscv32/riscv64? Thanks.

#12

hi @aywkwok,

Unfortunately the old way we were doing this is deprecated and doesn’t work right now. You can currently run against QEMU using the Zephyr RTOS:

  1. docker/bash.sh ci_qemu tests/scripts/task_config_build_qemu.sh
  2. docker/bash.sh ci_qemu tests/scripts/task_build.sh build -j2
  3. docker/bash.sh ci_qemu python tests/micro/zephyr/test_zephyr.py --microtvm-platforms=host_riscv64

We’d love to add back SPIKE support for RISC-V, but may need to wait a little longer for the project API refactor to land.

Andrew