Certain architectures (including Hexagon) support special kinds of memory, such as TCM. For our purposes, we’d like to be able to specify that a particular allocation should happen out of TCM instead of RAM (for example via tvm.decl_buffer, or tvm.ndarray).
I haven’t found anything like this already present in TVM, so I’m guessing I’d need to implement it. What is the best way to accomplish this?
It depends on where should the memory comes from. If we want to treat the memory as say DRAM, then we might want to specify a special kind of device API which maps a device type to the RAM.
If it is a storage that we want to use to store the intermediate result(like shared memory in GPU). Then we can use the special memory scope like vta to achieve similar goals
We’ll likely need both, but we can start with the scoped memory. Is this already done for vta? Do I need to write my own “storage rewriter” to replace all allocations of memory in that scope with custom ones?
The scoped memory is introduced during schedule phase(special memory tag), it is similar to how you tag shared memory.
See:
how VTA registers special memory scopes https://github.com/dmlc/tvm/blob/master/vta/python/vta/environment.py#L243
and how to use them in schedule https://docs.tvm.ai/vta/tutorials/vta_get_started.html#buffer-scopes
Hi,
I asked in another thread about the @tvm.register_func("tvm.info.mem.%s" % Environment.inp_scope) lines in the VTA example.
I know they are used to declare memory scopes, but could you give more information on what are “tvm.info.mem” and how they are processed in the compulation flow?
I mean is it really just a tag that the HW specific backend needs to replace with a certain memory address space?
Thanks