It is very nice that at deployment time of python applications, all you need is the libtvm_runtime.so + tvm/python. This is a small thing, but it will be even nicer if the python code also has a small runtime.py file that only loads the compiled library for serving.
1 Like
Thanks for the suggestion. Right now, the additional boiler plate is tvm.module.load("compiled.lib"). So the same code deployment should work for runtime only env.
The philosophy on the python side is to use the same set of APIs, and that is why we did not introduce a special runtime.py. Can you be a bit more specific about what is needed? Is the concern mainly about reducing code size? or is there something that we miss?
The use case I have in mind is running gpu-compiled tvm code with regular pytorch. The user still needs to clone all the tvm code (or at least the python part) to run it. So the concern is more about the convince of running the compiled code.
Now that I am thinking about it, a cleaner solution than runtime.py is to have the python package of tvm installable with pip (either from pypi or a github path). This way, it can be installed automatically with requirements.txt