Could I use TVM to deploy a nerual networks to a hardware that TVM has not supported

Could we find and change some codes or api in TVM?Then we can deploy our nerual network on a hardware based on FPGA.I can’t understand clearly what’s the edge fpga means inTVM Website.

Edge FPGA just means the FPGA used in edge devices such as IoTs and mobile phones. You can check if you can modify the VTA module to support your architecture on the FPGAs, or you can check BYOC if you prefer to treat it as an ASIC accelerator.

Thank you for replying me.You mean that I can modify some source codes in VTA?Could you tell me roughly how and where to change it?

I’m not familiar with VTA but just know its flow. @thierry @liangfu may provide more info about it.

1 Like

There are 3 options: (1) Library level integration (think CuDNN) with Bring your Own Codegen. See this post for an excellent introduction: https://tvm.apache.org/2020/07/15/how-to-bring-your-own-codegen-to-tvm (2) Compiler-level integration (think gcc or LLVM), is the standard way TVM interfaces with hardware backends, GPU or CPU. There are many examples on how this is achieved, TVM also has a C code generation backend which can be very handy (e.g. micro-tvm) (3) ISA-level integration: this is used when there is no compiler stack or library to use. In which case you need to generate from the TVM IR an instruction stream to your hardware. We did this for VTA as @comaniac pointed, which is an open source hardware design. The idea here is that we rely on TVM IR lowering to a set of runtime API calls that can JIT low-level instructions (e.g. DMA, synchronization, tensor intrinsic call) to our hardware. It’s one solution among many. In this case, there are many possible options, all of which vary, and depend on how your hardware is designed.

6 Likes

Thank you so much for replying me!

The BYOC introduction is great, but I want to generate OpenCL kernels for conv2d that use a custom library. I am confused because if I follow the introduction I cannot override opencl codegen as it is in tvm/src/target/source and not in tvm/src/relay.

What are the differences between these two codegen locations and how can I reach my goals for opencl codegen?