Relay and TOPI in code execution walkthrough

Questions
#1

Hi,

reading code walkthrough I have problem in placing and understanding where exactly Relay and TOPI are. After a lot of reading and watching talks from conference my best guess is that those are extra paths to the mentioned description. Is this correct understanding?

Is TOPI and Relay just different ways of expressing same operations and this would be adding extra stages to get to the level described?

I’d appreciate any comment on that.

#2

I am exploring Relay and TVM compiler stack. My understanding is Relay is functional programming meant for computation graph optimizations. Each operator of Relay will have corresponding backed operator in TOPI. Later this TOPI operator description will generate low-level tensor compute expression (TVM IR). Thus low level target specific optimizations like scheduling, loop unroll, vectorization will happen in TVM IR. Finally this TVM IR will be converted into target code using codegen module. I think, ideally we have some example showing life of an computation graph using TVM stack for beginners to understand. I am trying to derive such an flow.

1 Like
#3

Thank you, and I agree 100% with you that we need some kind of life of an computation graph using TVM stack. It has been 2 months I am trying to figure out, and I did some progress with the help of this forum. In the future, I’d hope to get more help from TVM experts in this forum.

#4

That document is outdated by now. We removed NNVM recently and we are focusing on Relay. But the codepath of the vector add example is still the same, so for lower level details that document is still useful.

what @IshwaraK described is correct. I agree that we need under-the-hood doc for graph level (or in Relay term, functional IR level) layer. Generally we are not great at adding and maintaining documentation. We rely on “Just dive into the code” mindset :slight_smile:

#5

Thank you masahi for confirming what ishwaraK said is correct, and that is also what I thought.

I have been diving into code for couple of weeks, and I have not really figured out one thing so far. Where does the TVM calls LLVM compiler to generate back end code ?

#6

It’s here https://github.com/apache/incubator-tvm/blob/master/src/codegen/codegen.cc#L39

When the target is llvm, it calls codegen.build_llvm function (at L53). You can grep “codegen.build_llvm” to see where it is defined.

Build() function itself is called from here https://github.com/apache/incubator-tvm/blob/master/src/api/api_codegen.cc#L36 This is the entry point for python. you can grep _Build inside python directory to see how it is called.

1 Like
#7

Thank you masahi, this is great! Thank you so much. Now, I should have told you what I was trying to accomplish. As far as I know we can call LLVM compiler with different optimization levels and flags such as mentioned here to force auto vectorize of the loops: https://llvm.org/docs/Vectorizers.html

I am wondering if I can (or how I can) pass those kinds of optimization flags within TVM to LLVM? How can i find out what kind of optimization flags are passed to LLVM when I generate code using TVM for ARM?

Thank you again, and that was my original intent for the LLVM. But you really pointed me to right direction.

#8

we can pass some flags like -mcpu=core-avx2, but I don’t think we can affect LLVM optimization.

We use -O3 for optimizations (see below). You can google what optimizations are turned on when OptLevel = 3.

1 Like