Latest TVM failed to build on Mac OSX with LLVM10

Lyken17 · June 16, 2020, 9:14pm

When I tried to build the latest tvm, I meet following erors

➜  build git:(master) make
[  9%] Built target tvm_runtime
[ 10%] Linking CXX shared library libtvm.dylib
Undefined symbols for architecture x86_64:
  "_LLVMInitializeAArch64AsmParser", referenced from:
      tvm::codegen::InitializeLLVM() in llvm_common.cc.o
  "_LLVMInitializeAArch64AsmPrinter", referenced from:
      tvm::codegen::InitializeLLVM() in llvm_common.cc.o
  "_LLVMInitializeAArch64Target", referenced from:
      llvm::InitializeAllTargets() in llvm_common.cc.o
  "_LLVMInitializeAArch64TargetInfo", referenced from:
      tvm::codegen::InitializeLLVM() in llvm_common.cc.o
      llvm::InitializeAllTargets() in llvm_common.cc.o
  "_LLVMInitializeAArch64TargetMC", referenced from:
      tvm::codegen::InitializeLLVM() in llvm_common.cc.o
  "_LLVMInitializeAMDGPUAsmParser", referenced from:
      tvm::codegen::InitializeLLVM() in llvm_common.cc.o
...

The environment that I am using is

Clang version

➜  build git:(master) clang -v
clang version 10.0.0
Target: x86_64-apple-darwin19.5.0
Thread model: posix
InstalledDir: /usr/local/opt/llvm/bin

Cmake config

➜  build git:(master) cmake --version
cmake version 3.14.0

CMake suite maintained and supported by Kitware (kitware.com/cmake).

➜  build git:(master) cmake ..
-- Build with RPC support...
-- Build with Graph runtime support...
-- VTA build with VTA_HW_PATH=/Users/ligengzh/Workspace/tvm/3rdparty/vta-hw
-- Build VTA runtime with target: sim
-- Link with static LLVM libraries
-- Found LLVM_INCLUDE_DIRS=/usr/local/opt/llvm/include
-- Found LLVM_DEFINITIONS=-D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS
-- Found TVM_LLVM_VERSION=100
-- Build with LLVM 10.0.0
-- Set TVM_LLVM_VERSION=100
-- Build with contrib.sort
-- Build with contrib.hybriddump
-- Build with thread support...
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/ligengzh/Workspace/tvm/build

Any suggestions?

junrushao · June 17, 2020, 12:05am

Can you try using LLVM from conda-forge?

conda install clangdev=8.0.1 -c conda-forge

Lyken17 · June 17, 2020, 1:55pm

I didn’t use conda on my mac, so I download llvm 8.0 from the official site. Now the problem is solved via switching from llvm10.0 -> 8.0.1. It seems that tvm does not support llvm 10 yet?

alopez_13 · June 17, 2020, 4:08pm

Not sure what could be wrong in your setup I have LLVM 10.0.0 and have built TVM several times. I assume you changed config.cmake to reflect your system capabilities?

set(USE_LLVM /usr/local/opt/llvm/bin/llvm-config)

I also used brew to install llvm, but I do remember installing from source at some point. Finally, you may need to install the CLI options for xcode and/or use xcode-select.

Hope it helps

junrushao · June 17, 2020, 5:16pm

Just had a quick try and I am able to confirm the build failure on macOS with LLVM 10.0.0

junrushao · June 17, 2020, 6:23pm

This is a linking issue which means those symbols are not found from LLVM…In the meantime, it looks like LLVM 6/7/8/9 works fine. It typically won’t happen if LLVM is released with correct binary.

It is unclear to me right now what happens. I will investigate further maybe this weekend.

Lyken17 · June 18, 2020, 6:04am

The thing really matter is not the location of llvm, but its version. Can you type /usr/local/opt/llvm/bin/llvm-config --version and share the output? I confirmed on two macs that tvm cannot build with llvm 10.0.

alopez_13 · June 18, 2020, 2:13pm

Here you go:

alopez@ravenloft>/usr/local/opt/llvm/bin/llvm-config --version

10.0.0

So I am running with version 10.0.0 If it helps this is the output of my cmake build:

alopez@ravenloft>cmake …

– Build with RPC support…

– Build with Graph runtime support…

– Build with Graph runtime debug support…

– VTA build with VTA_HW_PATH=/Users/alopez/Documents/Code/latest-tvm/3rdparty/vta-hw

– Build VTA runtime with target: sim

– Build with Metal support

– Use llvm-config=/usr/local/opt/llvm/bin/llvm-config

– /usr/local/Cellar/llvm/10.0.0_3/include

– Found LLVM_INCLUDE_DIRS=/usr/local/Cellar/llvm/10.0.0_3/include

– Found LLVM_DEFINITIONS= -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS

– Found TVM_LLVM_VERSION=100

– Build with LLVM

– Set TVM_LLVM_VERSION=100

– JAVA_HOME is not defined. Set it to ensure proper use

– Using Java at /usr/bin/java

– ANTLR4=/usr/bin/java;-jar;/usr/local/Cellar/antlr/4.8_1/antlr-4.8-complete.jar

– Use BLAS library /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/Accelerate.framework

– Build with contrib.sort

– Build with contrib.hybriddump

– Building Relay in debug mode…

– Build with thread support…

– Configuring done

– Generating done

– Build files have been written to: /Users/alopez/Documents/Code/latest-tvm/build

Lyken17 · June 18, 2020, 3:46pm

ah, that’s interesting. TVM successfully builds with LLVM 10 on some machines, while failed on others. @junrushao do you have any thoughts?

Between, can you also share the ouput of git log, wonder which commit your build is based on.

alopez_13 · June 18, 2020, 4:02pm

Here is my git log:

commit 3e72be58f362a21dbcc1de36f9dbed216e854baf ( HEAD -> master , origin/master , origin/HEAD )

Should be from June 17th (yesterday)

kparzysz · June 18, 2020, 4:50pm

Your clang may not have support for the targets that TVM wants to use. Try running

echo 'int main(){}' | clang -S -target aarch64 -x c - -o -

and see if you get some output, or if you get an error message.

Lyken17 · June 19, 2020, 12:00am

Here is the output on my mac

➜  ~ echo 'int main(){}' | clang -S -target aarch64 -x c - -o -

	.text
	.file	"-"
	.globl	main                    // -- Begin function main
	.p2align	2
	.type	main,@function
main:                                   // @main
// %bb.0:
	mov	w8, wzr
	mov	w0, w8
	ret
.Lfunc_end0:
	.size	main, .Lfunc_end0-main
                                        // -- End function
	.ident	"clang version 10.0.0 "
	.section	".note.GNU-stack","",@progbits
	.addrsig

junrushao · June 19, 2020, 6:57am

@Lyken17

I am able to get llvm-10 statically linked, by setting USE_LLVM to be “llvm-config --ignore-libllvm”.

It is weird to me why linking the shared libLLVM-10.dylib doesn’t work. In my case, I looked into its symbols using the command below, and did see those like “_LLVMInitializeAArch64AsmParser” existing

nm -gU /path/to/libLLVM-10.dylib

junrushao · June 19, 2020, 7:23am

I see. I found that the problem is due to CMake’s build in command “find_package(LLVM)” failing with LLVM-10.

@Lyken17 could you add this line somewhere under find_package?

message(STATUS "LLVM_LIBS = ${LLVM_LIBS}")

If the output is blank, congratulations, your cmake doesn’t works either

A workaround is to specify the full path to llvm-config in USE_LLVM.

Lyken17 · June 22, 2020, 2:00am

Solved via PR https://github.com/apache/incubator-tvm/pull/5859 Thanks Junru!