This is the stack abstraction I was talking about:
native void tvmFuncPushArgLong(long arg);
native void tvmFuncPushArgDouble(double arg);
native void tvmFuncPushArgString(String arg);
native void tvmFuncPushArgBytes(byte arg);
native void tvmFuncPushArgHandle(long arg, int argType);
That stack abstraction. Looking carefully at the jni bindings, I can see that it is threadsafe; sort of. It has no unwind mechanism and I really don’t see why the function interface isn’t just a generic object array that you can then do rtti on and convert to the actual tvm value objects. It is storing hidden state and so depending on how the upper layers are coded this state always has a chance to interact in an undefined fashion with code running later. As I said, I feel it is unnecessary.
JNA is more minimal in the sense that you do not have to write a jni.c file nor do you have to compile and bundle an extra shared library. With JNA, I can publish a jar to maven that contains no further binary artifacts that will work with either tvm or tvm-runtime with no further changes and will continue to work as the shared library changes. JNA is less work, overall, that writing bespoke JNI.
If I were to go the bespoke JNI route, I would use javacpp which does your JNI bindings for you automatically as I did in the first version. I believe that javacpp was designed to work on both the main JVM and Dalvik. Javacpp is also less work, overall, than bespoke JNI.
So, I don’t understand the claim to minimalism. I do understand the need for small APK sizes and this had absolutely not occured to me. While at NVIDIA, I developed custom JNI bindings for a graphics subsystem to use in their automotive platforms so I chose the same route that TVM has currently chosen but I was not as educated about the java ecosystem as I am now and I didn’t have a well designed C interface to our system already in place.
It may be worth it to keep the current bindings but there isn’t a good reason for me to use them as they don’t give raw access to the C api, they don’t link to the compiler or dsl api’s, their array access is quite incomplete in the sense that it doesn’t expose the fact that arrays are DL-tensors (you can’t get/set the stride, or the root data), etc etc etc. They appear to be specialized to precisely their use case in terms of running tvm code in a specific context. The clojure bindings are exactly equivalent to the python bindings so if you can imagine trying to put the python bindings on top of the current jvm bindings in tvm you see how this is just absolutely not even a remote possibility. So in order for us to share code the root jvm bindings need to be:
- greatly expanded in scope.
- The shared library has to have a more dynamic lookup mechanism. One thing I want to do is use tvm with mxnet in which case I will be dynamically binding to the mxnet shared library instead of tvm.
- Raw access to the C api. No wrappers; I don’t want to figure out layers of abstractions on top of the base abstraction.
- On maven central.
As an aside, for edge devices I have been considering node bindings. I looked into node’s ffi system and it is a mess but nodeffi would get you both android and ios devices, potentially. And potentially access to at least 4 million developers.