TOPI operator: upsampling in C vs. upsampling in python

Questions
#1

Hi All,

I am trying to understand the underlying structure of TOPI operators and how they were implemented.

For example, I have chased down upsampling and I see there are python and C++ versions of upsampling as follows, and I am wondering what is the relations between these two implementations? How they work togheter? I thought the python is merely an “INTEFFACE”, but I discovered that it actually implements upsampling ?

Thanks in advance,

Python code of upsampling under: \tvm\topi\python\topi\nn

def upsampling(data, scale_h, scale_w, layout="NCHW", method='nearest_neighbor',
               align_corners=False):
    """Perform upsampling on the data.
       Nearest neighbor and bilinear upsampling are supported.

    Parameters
    ----------
    inputs : tvm.Tensor
        inputs is a 4-D tensor with shape
        [batch, channel, in_height, in_width]
        or  [batch, in_height, in_width, channel]

    scale_h : float
        Scaling factor for height

    scale_w : float
        Scaling factor for width

    layout : string, optional
        either "NCHW" or "NHWC"

    method : {"bilinear", "nearest_neighbor", "bicubic"}
        Method to be used for upsampling.

    Returns
    -------
    output : tvm.Tensor
        4-D with shape [batch, channel, in_height*scale_h, in_width*scale_w]
        or [batch, in_height*scale, in_width*scale, channel]
    """
    base_layout = layout[0:4]
    if base_layout == "NCHW":
        out_shape = (simplify(topi.cast(tvm.round(data.shape[2] * scale_h), data.shape[2].dtype)),
                     simplify(topi.cast(tvm.round(data.shape[3] * scale_w), data.shape[3].dtype)))
    elif layout == "NHWC":
        out_shape = (simplify(topi.cast(tvm.round(data.shape[1] * scale_h), data.shape[1].dtype)),
                     simplify(topi.cast(tvm.round(data.shape[2] * scale_w), data.shape[2].dtype)))

    else:
        raise ValueError("not support this layout {} yet".format(layout))
    coord_trans = "align_corners" if align_corners else "asymmetric"
    return topi.image.resize(data, out_shape, layout=layout,
                             method=method, coordinate_transformation_mode=coord_trans)

C++code of upsampling under: \tvm\topi\include\topi\nn

namespace topi {
namespace nn {
using namespace tvm;
using namespace topi::image;

/*!
* \brief Upsample given tensor to given shape
*
* \param input The input tensor.
* \param shape Output shape to upsample.
* \param layout input layout
* \param mode Algorithm to use (NEAREST_NEIGHBOR / BILINEAR)
* \param name Name of the operation
* \param tag The tag to mark the operation
*
* \return A Tensor upsampled to given shape
*/
inline Tensor upsampling(const Tensor& input,
                         const Array<PrimExpr> shape,
                         std::string layout = "NCHW",
                         std::string mode = "NEAREST_NEIGHBOR",
                         std::string name = "tensor",
                         std::string tag = kInjective) {
  return resize(input, shape, layout, false, mode);
}

}  // namespace nn
}  // namespace topi
#endif  // TOPI_NN_UPSAMPLING_H_
#2

you can ignore the c++ implementation. It is a historical baggage and it was used only by NNVM until recently. Now that NNVM is gone, we can remove it. I’ll have a look.

1 Like
#3

Thank you very much, and this brings another question if that is Ok. Is that mean Relay does not use the C++ TOPI operators? I am trying to understand how to debug TOPI operators. I created my project in C++ hoping that I’d eliminate python so that I could reduce complexity

#4

you are hitting the dark corner here :slight_smile: The answer is some python operators are implemented in C++ while others are entirely in python. In topi/python directory, grep “cpp”. These are python topi that call into c++ implementation.

This is the historical baggage I’m talking about. There was a time when people tried to port python topi to c++ to reduce dependency on python. That sounds good on paper but that effort was half-done at some point (you can imagine nobody would port complicated operators and their schedules such as convolution or the whole AutoTVM infra to C++). Nowadays people do not seem to care and we are left with half-baked c++ implementation + cpp vs python inconsistency.

Personally I prefer python only solution.

1 Like
#5

Thank you @masahi, and now it is making sense.

Well, I still have one question. I do understand that current state of the TOPI, and I know that the old NNVM uses TOPI in computational graph. I am wondering if Relay uses python versions of TOPI or C++? Or does Relay is not restricted to use if python or C++?

#6

Relay doesn’t care if a particular topi it’s calling is implemented in python or C++. Remember I said "grep inside topi/python for cpp. These are python wrappers for corresponding C++ implementations. So in practice all topi calls from relay are first routed to python and then go to either python or c++ depending on which one they are implemented in.

Make sense now?

1 Like
#7

@masahi - thank you very much for clarification. Yes, now everything makes sense.

I’d like to understand the whole flow of TOPI operator. My goal is to understand the flow of topi operators from python to C++. I want to find the corresponding C++ implementation for particular target. So I created following example, and debugged it as shown below. My problem is that I can not find the corresponding C++ implementation of topi.sum().

  1. Here is my little example to debug topi.sum() function. I put a break point on topi.sum(), and steppend into it.
if __name__ == "__main__":
    print("TOPI test")

    C = topi.sum(A, axis=1)
    ts = tvm.create_schedule(C.op)
    print(tvm.lower(ts, [A], simple_mode=True))
  1. It goes top following function defined in reduction.py inside topi/python/topi which as you said has “cpp”. I believe this is where the C++ code is being called for the python topi.sum()
def sum(data, axis=None, keepdims=False):
    """Sum of array elements over a given axis or a list of axes

    Parameters
    ----------
    data : tvm.Tensor
        The input tvm tensor

    axis : None or int or tuple of int
        Axis or axes along which a sum is performed.
        The default, axis=None, will sum all of the elements of the input array.
        If axis is negative it counts from the last to the first axis.

    keepdims : bool
        If this is set to True, the axes which are reduced are left in the result as dimensions
        with size one.
        With this option, the result will broadcast correctly against the input array.

    Returns
    -------
    ret : tvm.Tensor
    """
    return cpp.sum(data, axis, keepdims)
  1. This is where I lost. One I step in to ‘cpp.sum(data, axis, keepdims)’, it goes following function, and I could not find the corresponding sum.topi() c++ function.
    def __call__(self, *args):
        """Call the function with positional arguments

        args : list
           The positional arguments to the function call.
        """
        temp_args = []
        values, tcodes, num_args = _make_tvm_args(args, temp_args)
        ret_val = TVMValue()
        ret_tcode = ctypes.c_int()
        if _LIB.TVMFuncCall(
                self.handle, values, tcodes, ctypes.c_int(num_args),
                ctypes.byref(ret_val), ctypes.byref(ret_tcode)) != 0:
            raise get_last_ffi_error()
        _ = temp_args
        _ = args
        return RETURN_SWITCH[ret_tcode.value](ret_val)