[RPC] C++ RPC server hangs using BYOC flow

Hi, I’ve been looking into using the C++ RPC server for remote devices in order to remove the python dependency, making deployment to remote devices much more simple. Currently, I see it hang whenever trying to run an external module - standard TVM modules work fine. This locks up the server completely and it will not accept any new connections, I have to restart the server to run another module. The tests I’m using run perfectly using the python implementation of the RPC server, hence the thought that this is an issue with the c++ implementation. I’m not entirely sure how to go about debugging this, although it seems to lock up after successfully loading the module on the remote device.

I was wondering how well this implementation is maintained since it doesn’t look like a core part of TVM; did it get updated with changes to the RPC protocol? Is there anyone else using the c++ implementation also experiencing a similar issue?

Both the c++ and python shares the same impl of the RPC protocol(implemented in c++), cc @FrozenGene

Apologies, I think I’ve found the source of this issue although my thinking above was incorrect. I’ve been using the --no-fork option with the python implementation, whilst the c++ implementation doesn’t have this option. I must be causing a deadlock somewhere.

1 Like