[VTA] Question about dependence queue

Hello, I have a question about VTA hardware.

While trying to understand how VTA hardware works, it is difficult to know how dependence queues work.

By extracting instructions with debug_flag=0x6, the following instructions are shown.

INSTRUCTION 0: NOP-STORE-STAGE
dep - pop prev: 0, pop next: 0, push prev: 1, push next: 0
l2g_queue = 0, g2l_queue = 0
s2g_queue = 1, g2s_queue = 0
INSTRUCTION 1: NOP-STORE-STAGE
dep - pop prev: 0, pop next: 0, push prev: 1, push next: 0
l2g_queue = 0, g2l_queue = 0
s2g_queue = 2, g2s_queue = 0
INSTRUCTION 2: LOAD UOP
dep - pop prev: 0, pop next: 1, push prev: 0, push next: 0
DRAM: 0x18600000, SRAM:0x0000
y: size=1, pad=[0, 0]
x: size=8, stride=8, pad=[0, 0]
l2g_queue = 0, g2l_queue = 0
s2g_queue = 1, g2s_queue = 0
INSTRUCTION 3: GEMM
dep - pop prev: 0, pop next: 0, push prev: 1, push next: 0
reset_out: 1
range (0, 8)
outer loop - iter: 56, wgt: 0, inp: 0, acc: 1
inner loop - iter: 2, wgt: 0, inp: 0, acc: 448
l2g_queue = 0, g2l_queue = 1
s2g_queue = 1, g2s_queue = 0
INSTRUCTION 4: LOAD UOP
dep - pop prev: 0, pop next: 1, push prev: 0, push next: 0
DRAM: 0x18600008, SRAM:0x0008
y: size=1, pad=[0, 0]
x: size=8, stride=8, pad=[0, 0]
l2g_queue = 0, g2l_queue = 1
s2g_queue = 0, g2s_queue = 0
INSTRUCTION 5: GEMM
dep - pop prev: 0, pop next: 0, push prev: 1, push next: 0
reset_out: 1
range (8, 16)
outer loop - iter: 56, wgt: 0, inp: 0, acc: 1
inner loop - iter: 2, wgt: 0, inp: 0, acc: 448
l2g_queue = 0, g2l_queue = 2
s2g_queue = 0, g2s_queue = 0
INSTRUCTION 6: LOAD INP
dep - pop prev: 0, pop next: 1, push prev: 0, push next: 0
DRAM: 0x06040000, SRAM:0x0000
y: size=9, pad=[1, 0]
x: size=56, stride=56, pad=[1, 1]
l2g_queue = 0, g2l_queue = 1
s2g_queue = 0, g2s_queue = 0
INSTRUCTION 7: LOAD WGT
dep - pop prev: 0, pop next: 0, push prev: 0, push next: 1
DRAM: 0x00600d00, SRAM:0x0000
y: size=2, pad=[0, 0]
x: size=9, stride=36, pad=[0, 0]
l2g_queue = 1, g2l_queue = 1
s2g_queue = 0, g2s_queue = 0
INSTRUCTION 8: LOAD INP
dep - pop prev: 0, pop next: 1, push prev: 0, push next: 0
DRAM: 0x06040000, SRAM:0x0244
y: size=9, pad=[1, 0]
x: size=56, stride=56, pad=[1, 1]
l2g_queue = 1, g2l_queue = 0
s2g_queue = 0, g2s_queue = 0
INSTRUCTION 9: LOAD WGT
dep - pop prev: 0, pop next: 0, push prev: 0, push next: 1
DRAM: 0x00600d48, SRAM:0x0012
y: size=2, pad=[0, 0]
x: size=9, stride=36, pad=[0, 0]
l2g_queue = 2, g2l_queue = 0
s2g_queue = 0, g2s_queue = 0

I guessed that it up to instruction 5 is a initializing phase since it has reset_out, and LOAD INP and LOAD WGT should be presented before LOAD UOP and GEMM.

So I listed what is happening in dependence queues.

However, even seeing these queues, Iā€™m not sure how task-level parallelism is achieved.

Can anyone tell me how this mechanism works?

Thank you for your help.

@jwlee the idea is that we want to communicate data dependencies between modules using queues. Read after Write (RAW) dependencies are enforced by pushing a token from the producer to the consumer. WAR dependencies are enforced by pushing a token from consumer to the producer.

You can think of each module using these dependence queues in the following pseudo-codey way:

# MODULE pseudo-code

insn_T insn = cmd.pop()
bool wait_on_producer = insn.pop_prev
bool wait_on_consumer = insn.pop_next
bool notify_producer = insn.push_prev
bool notify_consumer = insn.push_next
if wait_on_producer:
   while producer_raw_queue.empty() skip
   producer_raw_queue.pop()
if wait_on_consumer:
   while consumer_war_queue.empty() skip
   consumer_war_queue.pop()

# Do task

if notify_producer:
   producer_war_queue.push(1)
if notify_consumer:
   consumer_raw_queue.push(1)

This opens up task-level parallelism, if we can expose two parallel execution context in the TVM schedule which is what these lines do: https://github.com/dmlc/tvm/blob/master/vta/python/vta/top/vta_conv2d.py#L166-L169

Thierry

1 Like