SSD Mobilenet performance Issue

Questions
#1

Hi,

I just converted the mobilenet model 608x608 to TVM
It is taking 0.2 seconds per frame and giving me 5 FPS on 1050 TI with full cuda cores usage.
Time per frame:
0.20082473754882812
0.19168806076049805
0.19301962852478027
0.19402718544006348
0.1933760643005371
0.1952970027923584
0.2037806510925293
0.1919691562652588
0.20432472229003906
0.20802545547485352
0.20062041282653809

nvidia-smi -i 0 --query-gpu=index,timestamp,utilization.gpu,power.draw,temperature.gpu --format=csv -l 1
0, 2019/04/22 15:39:58.050, 90 %, [Not Supported], 56
0, 2019/04/22 15:39:59.050, 88 %, [Not Supported], 56
0, 2019/04/22 15:40:00.051, 90 %, [Not Supported], 56
0, 2019/04/22 15:40:01.051, 88 %, [Not Supported], 56
0, 2019/04/22 15:40:02.051, 88 %, [Not Supported], 57
0, 2019/04/22 15:40:03.052, 96 %, [Not Supported], 57
0, 2019/04/22 15:40:04.052, 100 %, [Not Supported], 57
0, 2019/04/22 15:40:05.053, 91 %, [Not Supported], 57
0, 2019/04/22 15:40:06.053, 89 %, [Not Supported], 58
0, 2019/04/22 15:40:07.053, 88 %, [Not Supported], 58

Am i doing something wrong or is this current performance benchmark ?

Code:

def display(img, out, thresh=0.5):
pens = dict()
for det in out:
cid = int(det[0])
if cid < 0:
continue
score = det[1]
if score < thresh:
continue
scales = [img.shape[1], img.shape[0]] * 2
xmin, ymin, xmax, ymax = [int(p * s) for p, s in zip(det[2:6].tolist(), scales)]

    cv2.rectangle(img, (xmin, ymin), (xmax, ymax), (255,0,0), 2)
    cv2.putText(img, class_names[cid],(xmin,ymin), font, 1, (200,0,0), 3, cv2.LINE_AA)

cv2.imshow('frame', img)

cap = cv2.VideoCapture(“rtsp://admin:admin123@192.168.1.193:554/Streaming/Channels/101”)
while(cap.isOpened()):
t0 = time.time()
ret, image = cap.read()
#image = cv2.imread(frame)
img_data = cv2.resize(image, (data_shape[2], data_shape[3]))
img_data = img_data[:, :, (2, 1, 0)].astype(np.float32)
#img_data -= np.array([123, 117, 104])
img_data = np.transpose(np.array(img_data), (2, 0, 1))
img_data = np.expand_dims(img_data, axis=0)

module.run(data=img_data)
tvm_output = module.get_output(0)
#print(tvm_output)
display(image, tvm_output.asnumpy()[0], thresh=0.25)

if cv2.waitKey(1) & 0xFF == ord('q'):
    break

print(time.time()-t0)

cap.release()
cv2.destroyAllWindows()

#2

Have you tried to autotune?

#3

I use the cv2.rectangle and cv2.putText for display, but it’s speed is slower than the gluoncv func utils.viz.plot_bbox.

Do you have this problem? or could you share your complete codes of display?

#4

I am sorry i am a beginner and after trying many times i have failed in understanding auto tune.

  • I have exported the files for mobilenet ssd but without auto tune.

mobilenet_ssd_608_tvm.so
mobilenet_ssd_608_tvm.json
mobilenet_ssd_608_tvm.params

  • How do i pass the trained model as input to auto tune ?
  • If possible can you share working python code ?
#5

@kevinthesun

I am stuck on autotune and it keeps on running without saving any models…
Please reply.

#6

What is the output during the autotuning process?

#7

Autotune should be done before compilation and exporting. Did you have any issue following this tutorial: https://docs.tvm.ai/tutorials/autotvm/tune_relay_cuda.html#sphx-glr-tutorials-autotvm-tune-relay-cuda-py