Understanding The Relay Virtual Machine?

Hi All,

I have read and re-read the description of the Relay VM from here: https://docs.tvm.ai/dev/virtual_machine.html

It looks like Relay VM provides interesting tool, but I am having trouble to understand what is the use case of Relay VM? I am asking this because (based on my vague understanding) this might be a solution to the problem that I have.

Specifically, I am wondering if Relay allows to decompose (partition) large problem into sub problems automatically ? For example, let us say I have large NxN matrix multiplication, and I am wondering if Relay allows to automatically partition the NxN matrix into 4, and solve the 4 sub-matrix multiplications in parallel?

Thanks, John.

Hi, Relay VM is just another runtime for relay, it has full support of Relay IR(control flow, dynamic shape, algebraic data type, etc). But it can’t solve your problem.

1 Like

Thank you wweic,

It is super useful. I find these slides heree about Relay VM which I think is the most helpful resource so far. However, I really do not understand why we need another runtime?
I understand we can extend TVM with Relay to support dynamic shapes, but I am not sure how the Relay VM helping here?

What is the main use case of Relay VM?

Dynamic shape is a language feature of Relay, when it comes to execution, we need a runtime that is able to handle dynamic shape object(Tensor who shape is unknown until runtime). Graph Runtime does not support it, VM is a full fledged virtual machine(like JVM) that can support such features.

1 Like

Thank you so much for confirming!. It looks like very useful. Is there any kind of resources regarding these matter other than https://docs.tvm.ai/dev/virtual_machine.html ? I’d like to learn more about these VM. It seems very useful thing.

Right now this is the main doc. But please share what pieces of VM you’d like to know so we can think if we can add more documentation.