Memory types for allocation (e.g. TCM)


#1

Certain architectures (including Hexagon) support special kinds of memory, such as TCM. For our purposes, we’d like to be able to specify that a particular allocation should happen out of TCM instead of RAM (for example via tvm.decl_buffer, or tvm.ndarray).

I haven’t found anything like this already present in TVM, so I’m guessing I’d need to implement it. What is the best way to accomplish this?


#2

It depends on where should the memory comes from. If we want to treat the memory as say DRAM, then we might want to specify a special kind of device API which maps a device type to the RAM.

If it is a storage that we want to use to store the intermediate result(like shared memory in GPU). Then we can use the special memory scope like vta to achieve similar goals


#3

We’ll likely need both, but we can start with the scoped memory. Is this already done for vta? Do I need to write my own “storage rewriter” to replace all allocations of memory in that scope with custom ones?


#4

The scoped memory is introduced during schedule phase(special memory tag), it is similar to how you tag shared memory.

See:

how VTA registers special memory scopes https://github.com/dmlc/tvm/blob/master/vta/python/vta/environment.py#L243

and how to use them in schedule https://docs.tvm.ai/vta/tutorials/vta_get_started.html#buffer-scopes


#5

Hi,
I asked in another thread about the @tvm.register_func("tvm.info.mem.%s" % Environment.inp_scope) lines in the VTA example.
I know they are used to declare memory scopes, but could you give more information on what are “tvm.info.mem” and how they are processed in the compulation flow?
I mean is it really just a tag that the HW specific backend needs to replace with a certain memory address space?

Thanks