Why TOPI is implemented in both c++ and python

Let’s take relu as an example:
In C++:

In Python:

It seems that these two parts have the same logic. What’s the point of this design?

Having an OP implemented in C++ is mainly for portability, i.e. doing so enables you portable it to various front-ends, imagining that you don’t want to use Python at all.

On the other hand, the Python version of OPs gives you more flexibility when looking for a best scheduling scheme.

In a long run, once the scheduling of an OP is stable, we can only use its C++ implementation for better portability. But I think for common tiny OPs like Relu, I think both implementations can co-live.

Thanks for the detailed explanations!

In a long run, once the scheduling of an OP is stable, we can only use its C++ implementation for better portability

I have another question, does the scheduling work only for a single OP or a subgraph?
If we get best performance of each OP, can we get the best performance of a subgraph of OPs?
I mean in some circumstances, the scheduling of a single OP can be great, but if we have a subgraph of OPs, and the best scheduling based on the whole subgraph may be changed for a OP.

So far the schedule is generic with respect to a type of subgraph. for example, schedule_injective will fuse all the injective operator in one op. schedule_conv2d will fuse the possible output element-wise ops.

There is definitely room for improvements, but the current subgraph pattern schedulers works well for our purpose so far, and we can have more discussion on what is being needed.

1 Like

I asked the same question in another form, and @masahi kindly responded to my question and it looks like to me that the TOPI C++ is redundant (If I understand correctly).

I also assume that NNVM used to use C++ implementations of TOPI, and Relay does not need C++ if I understand @masahi answer. Could you guys elaborate this more because it seems very important to understand how Relay/TOPI works in current release.

Generally we tried to put stable impls into c++ and those that are developing in python

1 Like

Thanks for the quick response.

I have debugged which seems to have python implementation only. However, when i step into topi.identity(), and I was expecting it to return from def identity(x) shown below. But it goes further, and it calls _LIB.TVMFuncCall() inside function.py which seems it further calls C++ function? So, my understanding that all python TOPI operators call C++ eventually?

  1. My topi.identity(A) code
    n = tvm.var("n")
    m = tvm.var("m")
    A = tvm.placeholder((n, m), name='A')

    A_id = topi.identity(A)
  1. topi.identity() implementation in Python.
def identity(x):
    """Take identity of input x.

    x : tvm.Tensor
        Input argument.

    y : tvm.Tensor
        The result.
    # pylint: disable=unnecessary-lambda
    return tvm.compute(x.shape, lambda *i: x(*i))
  1. tvm.compute(x.shape, lambda *i: x(*i)) is calling following function which seems interface to C++?
    def __call__(self, *args):
        """Call the function with positional arguments

        args : list
           The positional arguments to the function call.
        temp_args = []
        values, tcodes, num_args = _make_tvm_args(args, temp_args)
        ret_val = TVMValue()
        ret_tcode = ctypes.c_int()
        if _LIB.TVMFuncCall(
                self.handle, values, tcodes, ctypes.c_int(num_args),
                ctypes.byref(ret_val), ctypes.byref(ret_tcode)) != 0:
            raise get_last_ffi_error()
        _ = temp_args
        _ = args
        return RETURN_SWITCH[ret_tcode.value](ret_val)

Checkout https://github.com/tqchen/ffi-navigator to track python functions into cxx counter part

1 Like

Thank you very much. However, I am having hard time as well to install ffi-navigator. I outlined my problem here: Language Server Tool to Navigate Across PackedFunc FFI for IDEs like VSCode and Emacs