Can I build the TVM runtime locally and then put it inside the target device? I am going to use iMX 6 DL and it has a total of only 32 MB flash.
Yes, you can cross-compile it.
you can compile tvm runtime on any armv7 device. e.g. on RaspberryPi and then copy whl, egg or install tvm runtime to folder , zip the folder and copy zip file to iMX 6 DL.