Running relay_quick_start.py crash at tvm c++ code when tvm is compiled with c++ debug mode(-O0 -g)

Questions
#1

cd tvm/build
cmake -DCMAKE_BUILD_TYPE=Debug …
make -j8

cd tvm/tutorials
gdb --args python3 relay_quick_start.py

crash log:

*** stack smashing detected ***: /usr/bin/python3 terminated

Thread 1 “python3” received signal SIGABRT, Aborted.
0x00007ffff7825428 in __GI_raise (sig=sig@entry=6) at …/sysdeps/unix/sysv/linux/raise.c:54
54 …/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0 0x00007ffff7825428 in __GI_raise (sig=sig@entry=6) at …/sysdeps/unix/sysv/linux/raise.c:54
#1 0x00007ffff782702a in __GI_abort () at abort.c:89
#2 0x00007ffff78677ea in __libc_message (do_abort=do_abort@entry=1, fmt=fmt@entry=0x7ffff797f49f “*** %s ***: %s terminated\n”)
at …/sysdeps/posix/libc_fatal.c:175
#3 0x00007ffff790915c in __GI___fortify_fail (msg=, msg@entry=0x7ffff797f481 “stack smashing detected”) at fortify_fail.c:37
#4 0x00007ffff7909100 in __stack_chk_fail () at stack_chk_fail.c:28
#5 0x00007fffdd190701 in tvm::relay::Stack::Stack (this=0x16a1608) at /home/carl/gerrit/tvm/src/relay/backend/interpreter.cc:145
#6 0x00007fffdd190da3 in tvm::relay::Interpreter::Interpreter (this=<error reading variable: Cannot access memory at address 0xffffffffffffffe8>,
mod=<error reading variable: Cannot access memory at address 0xffffffffffffffe0>,
context=<error reading variable: Cannot access memory at address 0xffffffffffffffd0>,
target=<error reading variable: Cannot access memory at address 0xffffffffffffffd8>)
at /home/carl/gerrit/tvm/src/relay/backend/interpreter.cc:223
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
(gdb)

#2

Does this problem go away at higher optimization levels? could you also provide details of your platforms and versions?

#4

The problem will disappear when opt_level <= 1. My notebook is Ubuntu16.04.06 64bit, GPU is Geforce 840M with driver 418.39 and cuda 10.1.105. gcc is 5.4 and llvm is 5.02.

#5

I met this issue too, and it only happened when CMAKE_BUILD_TYPE=Debug

I’ve found a hack which can work around this issue, but I haven’t figured out the root cause. It seems related to how Interpreter is initialized.

#if 0
Expr PartialEval(const Expr& e) {
  return TransformF([&](const Expr& e) {
      return LetList::With([&](LetList* ll) {
          PartialEvaluator pe(FreeVars(e));
          return Remap(DeDup(pe.VisitExpr(e, ll)->dynamic));
        });
    }, e);
}

TVM_REGISTER_API("relay._ir_pass.partial_evaluate")
.set_body([](TVMArgs args, TVMRetValue* ret) {
    *ret = PartialEval(args[0]);
  });
#endif
#6

I try your method, it works,thanks!