Status of Relay VM

I am curious about the status of the Relay VM. How does it compare functionality- and performance-wise to the Graph Runtime? Are dynamics shapes officially supported? If so, are there any official examples or tutorials for how to use it and which ops have defined shape funcs?

Given that it’s the future of TVM runtime, I’d like to start moving my workloads to it if possible :blush:

Thanks!

1 Like

@jroesch do you have any guidance here?

VM currently is able to support all models that graph runtime can support. But it is able to execute more models that graph runtime cannot do, i.e. those with control flow or branch statements. We had some benchmarking before. The performance of most image classification models on CPU is very close for them. GPU has a bit issue for now because we don’t have a full memory analysis pass in the upstream yet. @jroesch may have some update on it.