Currently, TVM doesn’t support a target dispatch mechanism in C++. As discussed in this PR, this can lead to design issues, duplicated code, and bugs.
In the example discussed in the PR,
schedule_extern must be implemented differently for C++ and Python. In Python, a schedule can be defined in
generic/extern.py. The code in this file can then call
schedule_injective, which will automatically call the right method for the current target. In other words, every target can use common code. In C++, this is not possible. Instead, the correct
schedule_injective function must be called explicitly, which means that each target needs its own implementation of
schedule_extern. This leads to each target having duplicated code.
If C++ were to have a target dispatch mechanism, we could write one
generic/extern.h, and use it from all targets in both Python and C++. This is easier to write, much less error-prone, and would allow clean refactoring.
I propose a target dispatch mechanism in C++. We would have a function like
topi::generic::dispatch::schedule_injective(args), which would get the correct function for the current target and call it with the correct arguments.
From some inspection, it seems that C++ TOPI already registers generic functions here, and this would be a natural place to add this functionality. I also think that this would benefit other areas of the code, as we could move more logic into C++ and have more consistency between the Python and C++ interfaces.
I am pretty new to this area of the code, and would appreciate some thoughts on this proposal, as well as guidance for implementation