Exported Model from PyTorch to TVM from tutorial doesn't load

#1

Hi, TVM newbie here: I was following the tutorial here: https://docs.tvm.ai/tutorials/frontend/from_pytorch.html Specifically, I am interested in making repeated inference requests from an exported PyTorch model.

So I followed the recommendation here: How could I save the "graph", "lib", and "params" and exported the trained artifacts.

# save the graph, lib and params into separate files
from tvm.contrib import util

temp = util.tempdir()
path_lib = temp.relpath("deploy_lib.tar")
lib.export_library(path_lib)
with open(temp.relpath("deploy_graph.json"), "w") as fo:
    fo.write(graph)
with open(temp.relpath("deploy_param.params"), "wb") as fo:
    fo.write(relay.save_param_dict(params))

Now I try to create a TVMModule so I can load from model file using this API:

    tvm::runtime::Module module = tvm::runtime::Module::LoadFromFile("/path/to/exported/lib/above");

This tries to dlopen() the library at some point and fails because the above exported file is only an ELF-binary (*.o) not a shared object, I get the following error from TVM’s runtime/module:

"only ET_DYN and ET_EXEC can be loaded"

Any thoughts on what could be wrong?

#2

I browsed the code around a bit more and figured that this is addressed in the python APIs only.

Using using C++ APIs has this downside. I have a patch to make a PR but don’t have write permissions. So I am attaching my patch for anyone who might want to pick it up (not a fix but a clear error message) since it requires to build the share object file from lib.

From 68cd2717c5003a12ef147059054f79aa6d646854 Mon Sep 17 00:00:00 2001
From: Manoj Rao <mail@mycpu.org>
Date: Sun, 17 May 2020 19:02:30 -0700
Subject: [PATCH] Create a helpful error message while loading model libraries

If the user creates a module from exported model as described
in discussion here:
https://discuss.tvm.ai/t/exported-model-from-pytorch-to-tvm-from-tutorial-doesnt-load/6704?u=mycpuorg
For users who may choose to use the runtime directly the error messages can be
strange.

Provide a tip to the user in error message after a load failure.


 src/runtime/dso_library.cc | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/src/runtime/dso_library.cc b/src/runtime/dso_library.cc
index 4df5c0552a3d..6bc32b54169e 100644
--- a/src/runtime/dso_library.cc
+++ b/src/runtime/dso_library.cc
@@ -83,6 +83,9 @@ class DSOLibrary final : public Library {
     lib_handle_ = dlopen(name.c_str(), RTLD_LAZY | RTLD_LOCAL);
     CHECK(lib_handle_ != nullptr)
         << "Failed to load dynamic shared library " << name
+        << "If you passed an ELF file in .o in " << name << " then you should "
+        << "consider creating shared lib. Ex: for llvm simply "
+        << "g++ -shared -fPIC " << name << " -o libcompiledmodel.so"
         << " " << dlerror();
   }
 
--