Using external lib (cuDNN) with INT8 ops


#1

Do we currently support this use-case? It looks like the operator selection/profiling part of cuDNN works, but when running I get CUDNN_STATUS_BAD_PARAM. CC @masahi


#2

no, it looks like we are passing hard coded CUDNN_DATA_FLOAT as dtype to cuDNN calls :slight_smile: I’ll fix this soon.


#3

@eqy is cuDNN with Int8 supposed to work on a pascal card? It seems I can’t even create a conv descriptor when dtype is Int8 (cudnnSetConvolution2dDescriptor). algo selection is done on fp32 (hard coded, need to be fixed), with int8 nothing works.


#4

I do not know if cuDNN works, but tuning with TVM schedules works (I just wanted to try something quickly before doing tuning). I think the TVM performance is reasonable (~35TOP/s at batch size 128 on 1080 Ti), so it is not urgent.

We did recently get some 2080 Ti cards too though :slight_smile: