Understanding The Relay Virtual Machine?

Questions
#1

Hi All,

I have read and re-read the description of the Relay VM from here: https://docs.tvm.ai/dev/virtual_machine.html

It looks like Relay VM provides interesting tool, but I am having trouble to understand what is the use case of Relay VM? I am asking this because (based on my vague understanding) this might be a solution to the problem that I have.

Specifically, I am wondering if Relay allows to decompose (partition) large problem into sub problems automatically ? For example, let us say I have large NxN matrix multiplication, and I am wondering if Relay allows to automatically partition the NxN matrix into 4, and solve the 4 sub-matrix multiplications in parallel?

Thanks, John.

1 Like
#2

Hi, Relay VM is just another runtime for relay, it has full support of Relay IR(control flow, dynamic shape, algebraic data type, etc). But it can’t solve your problem.

1 Like
#3

Thank you wweic,

It is super useful. I find these slides heree about Relay VM which I think is the most helpful resource so far. However, I really do not understand why we need another runtime?
I understand we can extend TVM with Relay to support dynamic shapes, but I am not sure how the Relay VM helping here?

What is the main use case of Relay VM?

#4

Dynamic shape is a language feature of Relay, when it comes to execution, we need a runtime that is able to handle dynamic shape object(Tensor who shape is unknown until runtime). Graph Runtime does not support it, VM is a full fledged virtual machine(like JVM) that can support such features.

1 Like
#5

Thank you so much for confirming!. It looks like very useful. Is there any kind of resources regarding these matter other than https://docs.tvm.ai/dev/virtual_machine.html ? I’d like to learn more about these VM. It seems very useful thing.

#6

Right now this is the main doc. But please share what pieces of VM you’d like to know so we can think if we can add more documentation.