What did Codegen do in tvm?


#1

For instance, when I viewing the code in the tvm/src/codegen,I am confused. I want to know what codegen does to make it match the corresponding backend.


#2

It translates the TVM IR into another IR that can be compiled into executable code. For example, code in src/codegen/llvm generates LLVM IR, which is then handed over to the LLVM’s optimizer/code generator to get the object file or assembly. codegen_c.cc will translate TVM IR into C code which then can be compiled using a C compiler, etc.


How to get C code after GEMM optimization python code, on TVM
#3

Thank you for your answer,My current question is how to read the codegen code. because I can not reach the level of fast reading code.For example, I want to read the implementation of X86, but there are too many codegen files, such as codegen.c, codegen_c.cc and llvm folder corresponding codegen_cpu.cc, codegen_x86_64.cc.At the same time, there are more codes in the file. How can I read the key points?


#4

X86-64 uses LLVM, so you could start with codegen_llvm.cc. codegen_cpu.cc is derived from codegen_llvm, and codegen_x86_64.cc is further derived from codegen_cpu. Most of the codegen happens in the first two files, whatever is not handled by codegen_cpu is dealt with in codegen_llvm. The last one, codegen_x86_64 is only there to generate some vector intrinsics.
They all work in a similar way: they have visitor functions that are called for each element (statement or expression) of the TVM/Halide IR, and generate the corresponding LLVM IR. For example,

llvm::Value* CodeGenLLVM::VisitExpr_(const And* op) {
  return builder_->CreateAnd(MakeValue(op->a), MakeValue(op->b));
}

This is one of the simplest functions in there, but illustrates the concept: MakeValue takes an expression (in TVM/Halide IR) and returns the corresponding llvm::Value. Then the call to CreateAnd creates an And operator in LLVM IR, using the llvm::Value values returned by MakeValue.
Many of the translations are not so straightforward, so the “Visit” functions are more complicated, but the concept remains more or less the same.


#5

Thank you for your answer and I have helped a lot. I have been reading the code in these two days.I suddenly have a question about whether Halide IR was converted to LLVM IR in CodeGen. Is this step already using LLVM?


#6

Yes, codegen converts Halide IR into LLVM IR.


#7

Thank you very much for your help, but I still have some questions. If you have WeChat or other communication software, can you give me a convenient way to communicate?


#8

No, sorry. Feel free to ask more questions on this forum, this way other people with the same questions can benefit from it.


#9

OK,Here is my current problem: my general purpose is to add a specific backend supported by LLVM in tvm, so I tried to use existing riscv, my idea is that riscv is a CPU architecture, and I don’t need to support intrin instructions now, what do I need to do?


#10

It may be the case that you don’t need to do anything, although it’s somewhat unlikely. Are you using a RISC-V simulator? Does it simulate the whole system, or just the environment for running applications?

When you create an LLVM target, pass “-target riscv32” (or riscv64) to it, i.e. “llvm -target riscv32” and see what happens. You will need LLVM with RISC-V backend enabled, and one that supports JIT compilation (I’m not sure if that’s available for RISC-V, but you can try and see).

In general, the easiest approach is to pretend that RISC-V is just another CPU, and rely on the CPU code. When something fails, fix it and try again. You’ll get a better understanding of what pieces are missing, and that’s something that’s often difficult to fully predict ahead of time.

I think someone has already tried using TVM on RISC-V, but it didn’t work because LLVM didn’t support PIC for RISC-V at the time. The RISC-V backend is actively developed in LLVM, and so this may no longer be a problem now.


#11

Thank you very much for your help. I can ask you about the progress of the hexagon about the codegen part, because I also want to add a new dsp backend, but I am confused about what to do.


#12

codegen_c.cc will translate TVM IR into C code which then can be compiled using a C compiler, etc

in this post

I wanted to ask, can one get to access the C code which is being compiled, for any graph being un on TVM ? I wanted to see the C code output that codegen generates from TVM IR.


#13

Hello! I would like to ask a favor:

In TVM generate code module, such as generating cuda code (/src/codegen/codegen_cuda. cc: 66):

void CodeGenCUDA::VisitStmt_(const ir::For* op) {
CHECK(is_const_int(op->min, 0));
if (op->for_type == ir::ForType::Unrolled) {
PrintIndent();
stream << “#pragma unroll\n”;
}
CodeGenC::VisitStmt_(op);
}

My understanding of this code is that the op->for_type type value was obtained in(/src/op/op_util.cc: 251):

ir::ForType IterVarTypeToForType(IterVarType iter_type) {
switch (iter_type) {
case kDataPar:
return ForType::Serial;
case kParallelized:
return ForType::Parallel;
case kVectorized:
return ForType::Vectorized;
case kUnrolled:
return ForType::Unrolled;
default:
return ForType::Serial;
}
}

But I use pdb and gdb joint tracking, and insert breakpoints here, I found that program did not perform to it, it makes me confused. Could this op->for_type value have been obtained elsewhere?
Thank you for your kind help!


#14

For::make takes ForType as an argument, so For could be created directly with a given type. Set a breakpoint at For::make, and see if the type has already been set there.


#15

Thanks for your answer, I made it!


#16

Sorry to bother you again!:sweat_smile:

I want to generate gpu-like code for DL operators with TVM and add guidance statements to the generated code. such as: #pragma unroll 4 . So I simulated the generation process of cuda code and found that cuda supports adding guidance statements (tvm/src/codegen/codegen_cuda.cc:66).

void CodeGenCUDA::VisitStmt_(const ir::For* op) {
     CHECK(is_const_int(op->min, 0));
     if (op->for_type == ir::ForType::Unrolled) {
     PrintIndent();
     stream << "#pragma unroll\n";
   }
  CodeGenC::VisitStmt_(op);
}

It is shown here that whether or not to add the guidance statement depends on the for_loop type in the operator: for_type . Just like yesterday I asked you, the value of for_type comes from For ::make , while thread_tag in For ::make comes from IterVarNode::make , as shown in the source code below(tvm/src/lang/expr.cc:129):

IterVar IterVarNode::make(Range dom,
                      Var var,
                      IterVarType t,
                      std::string thread_tag) {
       NodePtr<IterVarNode> n = make_node<IterVarNode>();
       n->dom = dom;
       n->var = var;
       n->iter_type = t;
       n->thread_tag = thread_tag;
       return IterVar(n);
}

tvm/src/api/api_lang.cc:315:

TVM_REGISTER_API("_IterVar")
.set_body_typed<IterVar(Range, Var, int, std::string)>([](
Range dom, Var var, int iter_type, std::string thread_tag ) {
return IterVarNode::make(
  dom, var,
  static_cast<IterVarType>(iter_type),
  thread_tag);
});

I used gdb to trace the thread_tag here and found that tracing up to the C++ / python interface, but I still couldn’t find where the thread_tag value came from, which confused me.

Should the guidance statement be manually specified in the input operator? It’s not determined by TVM by the type of for_loop in the input operator, right?


#17

The thread_tag comes from the function thread_axis (see python/tvm/api.py).

These guidance statements (like unrolling) come from the schedule for a given operator. It seems like TVM is doing some automatic unrolling, but in general scheduling directives should be added explicitly.


#18

Thank you!
The guidance language really should be specified by the user in the input operator
I am currently trying to generate C code. Do you have any research on this area? I saw a similar problem on github, but failed to generate C code, please see the following link picture:Please see the last question in this post