Memory leak in JNI Call of "tvmArrayCopyFromTo"

sol401430 · May 8, 2019, 7:00pm

We observed that our App(which simply run TVM inference in a loop, using static input array to test latency), will always been killed(or crush) over night after long running. some investigation show the memory used by the App continuously increase, and we find a lot of unreachable memory allocations using dumpsys, and the number is increasing very quickly;
see dumpsys below:

dumpsys -t 600 meminfo --unreachable 17181                                                                             
Applications Memory Usage (in Kilobytes):
Uptime: 502760754 Realtime: 502760754

** MEMINFO in pid 17181 [ml.dmlc.tvm.android.demo] **
                   Pss  Private  Private  SwapPss     Heap     Heap     Heap
                 Total    Dirty    Clean    Dirty     Size    Alloc     Free
                ------   ------   ------   ------   ------   ------   ------
  Native Heap    33421    33328        4       54    59392    51006     8385
  Dalvik Heap     8622     8600        4       53     8758     4411     4347
 Dalvik Other      410      408        0        0                           
        Stack       60       60        0        0                           
       Ashmem        2        0        0        0                           
      Gfx dev      236      236        0        0                           
    Other dev       14        0       12        0                           
     .so mmap     8479      256     6532       45                           
    .jar mmap        4        0        4        0                           
    .apk mmap     3454     2980       12        0                           
    .ttf mmap       52        0        0        0                           
    .dex mmap     2550        4     1232        0                           
    .oat mmap       47        0       20        0                           
    .art mmap     6737     6060       96        5                           
   Other mmap       22        4        8        1                           
   EGL mtrack     7720     7720        0        0                           
    GL mtrack     3392     3392        0        0                           
      Unknown      930      832       88        3                           
        TOTAL    76313    63880     8012      161    68150    55417    12732
 
 App Summary
                       Pss(KB)
                        ------
           Java Heap:    14756
         Native Heap:    33328
                Code:    11040
               Stack:       60
            Graphics:    11348
       Private Other:     1360
              System:     4421
 
               TOTAL:    76313       TOTAL SWAP PSS:      161
 
 Objects
               Views:        8         ViewRootImpl:        1
         AppContexts:        5           Activities:        1
              Assets:        2        AssetManagers:        0
       Local Binders:        8        Proxy Binders:       27
       Parcel memory:        5         Parcel count:       20
    Death Recipients:        1      OpenSSL Sockets:        0
            WebViews:        0
 
 SQL
         MEMORY_USED:        0
  PAGECACHE_OVERFLOW:        0          MALLOC_SIZE:        0
 
 
 Unreachable memory
  27643912 bytes in 398 unreachable allocations
  ABI: 'arm64'

  69632 bytes unreachable at 738601e000
   first 20 bytes of contents:
   738601e000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
   738601e010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................

  69632 bytes unreachable at 738602f000
   first 20 bytes of contents:
   738602f000: 00 00 80 3f 00 00 80 3f 00 00 80 3f 00 00 80 3f ...?...?...?...?
   738602f010: 00 00 80 3f 00 00 80 3f 00 00 80 3f 00 00 80 3f ...?...?...?...?

  69632 bytes unreachable at 7386040000
   first 20 bytes of contents:
   7386040000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
   7386040010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
....

I think the memory leak is due to this pointer assignment below:

github.com

dmlc/tvm/blob/master/jvm/native/src/main/native/ml_dmlc_tvm_native_c_api.cc#L428


JNIEnv *env, jobject obj, jlong jfrom, jlong jto) {
return TVMArrayCopyFromTo(reinterpret_cast<TVMArrayHandle>(jfrom),
                          reinterpret_cast<TVMArrayHandle>(jto), NULL);
}


JNIEXPORT jint JNICALL Java_ml_dmlc_tvm_LibInfo_tvmArrayCopyFromJArray(
JNIEnv *env, jobject obj, jbyteArray jarr, jlong jfrom, jlong jto) {
jbyte *data = env->GetByteArrayElements(jarr, NULL);


TVMArray *from = reinterpret_cast<TVMArray *>(jfrom);
from->data = static_cast<void *>(data);


int ret = TVMArrayCopyFromTo(static_cast<TVMArrayHandle>(from),
                             reinterpret_cast<TVMArrayHandle>(jto), NULL);


from->data = NULL;
env->ReleaseByteArrayElements(jarr, data, 0);


return ret;
}

here from->data may already pointed to a pre-allocated memory space; Actually, any call for NDArray’s copyFrom() method(see below), an temp NDArray tmpArr will be created with pre-allocated memory, and the current implementation of tvmArrayCopyFromJArray will cause this pre-allocated memory unreachable, due to the pointer assignment mentioned above;

github.com

dmlc/tvm/blob/master/jvm/core/src/main/java/ml/dmlc/tvm/NDArray.java#L75


 */
public void copyFrom(float[] sourceArray) {
  checkCopySize(sourceArray.length);
  if (dtype.typeCode != TVMType.FLOAT || dtype.bits != 32) {
    throw new IllegalArgumentException("Cannot set float[] for " + dtype.toString() + " array");
  }
  byte[] nativeArr = new byte[sourceArray.length * dtype.numOfBytes];
  for (int i = 0; i < sourceArray.length; ++i) {
    wrapBytes(nativeArr, i * dtype.numOfBytes, dtype.numOfBytes).putFloat(sourceArray[i]);
  }
  NDArray tmpArr = empty(shape(), this.dtype);
  Base.checkCall(Base._LIB.tvmArrayCopyFromJArray(nativeArr, tmpArr.handle, handle));
  tmpArr.release();
}


/**
 * Copy from a native array.
 * The NDArray type must by int64
 * @param sourceArray the source data
 */
public void copyFrom(long[] sourceArray) {

I used memcpy(from->data, static_cast<void *>(data), size) instead of from->data = static_cast<void *>(data); and verified the memory leak problem in my application has disappeared, however I’m not sure whether this is a generic solution for all cases;