Compiler Errors for PixelShuffle layer

Development
#1

I am interested in using PyTorch’s PixelShuffle layer in my neural network design. I’ve been trying to run this layer (on its own) through the TVM compiler stack.

The model in PyTorch is defined as

model = nn.Sequential(nn.PixelShuffle(2),)

It is exported via ONNX and imported into NNVM using the ONNX frontend. Compiled using a host machine (x86_64-linux-gnu) targeting the CPU on a Jetson TX2 (target="llvm -target=aarch64-linux-gnu"). Using latest TVM source code and llvm-4.0.

If I use an input data shape that has a batch size of one, for example, (1, 32, 14, 14), I get the following compile error:

nnvm._base.NNVMError: Error in operator strided_slice1: [23:02:18] /home/dwofk/tvm/nnvm/src/top/tensor/transform.cc:919: Check failed: stride_vec[i] < 0 ? (end < begin) : (begin < end) : Input [Begin=1, End=2] is invalid for axis=0

If I increase the batch size to 16 and feed in an input data shape of (16, 32, 14, 14), I get this compile error:

nnvm._base.NNVMError: Error in operator concatenate0: [23:05:12] /home/dwofk/tvm/nnvm/src/top/tensor/transform.cc:120: Operator concatenate(axis=0, name=concatenate0) expects data1's shape to be [0,32,14,14], but got [1].

Are there any specific modifications that need to be made to the TVM source code to resolve these compiler errors for PixelShuffle?

I can provide the exported ONNX graph if that would be helpful.

#2

Yes, please share the exported ONNX graph.

#3

Here is the exported ONNX graph, for an input data shape of (1, 32, 14, 14).

ir_version: 3
producer_name: "pytorch"
producer_version: "0.3"
graph {
  node {
    input: "0"
    output: "1"
    op_type: "Shape"
  }
  node {
    input: "1"
    output: "2"
    op_type: "Slice"
    attribute {
      name: "axes"
      ints: 0
      type: INTS
    }
    attribute {
      name: "ends"
      ints: 1
      type: INTS
    }
    attribute {
      name: "starts"
      ints: 0
      type: INTS
    }
  }
  node {
    input: "2"
    output: "3"
    op_type: "Squeeze"
    attribute {
      name: "axes"
      ints: 0
      type: INTS
    }
  }
  node {
    input: "0"
    output: "4"
    op_type: "Shape"
  }
  node {
    input: "4"
    output: "5"
    op_type: "Slice"
    attribute {
      name: "axes"
      ints: 0
      type: INTS
    }
    attribute {
      name: "ends"
      ints: 2
      type: INTS
    }
    attribute {
      name: "starts"
      ints: 1
      type: INTS
    }
  }
  node {
    input: "5"
    output: "6"
    op_type: "Squeeze"
    attribute {
      name: "axes"
      ints: 0
      type: INTS
    }
  }
  node {
    input: "0"
    output: "7"
    op_type: "Shape"
  }
  node {
    input: "7"
    output: "8"
    op_type: "Slice"
    attribute {
      name: "axes"
      ints: 0
      type: INTS
    }
    attribute {
      name: "ends"
      ints: 3
      type: INTS
    }
    attribute {
      name: "starts"
      ints: 2
      type: INTS
    }
  }
  node {
    input: "8"
    output: "9"
    op_type: "Squeeze"
    attribute {
      name: "axes"
      ints: 0
      type: INTS
    }
  }
  node {
    input: "0"
    output: "10"
    op_type: "Shape"
  }
  node {
    input: "10"
    output: "11"
    op_type: "Slice"
    attribute {
      name: "axes"
      ints: 0
      type: INTS
    }
    attribute {
      name: "ends"
      ints: 4
      type: INTS
    }
    attribute {
      name: "starts"
      ints: 3
      type: INTS
    }
  }
  node {
    input: "11"
    output: "12"
    op_type: "Squeeze"
    attribute {
      name: "axes"
      ints: 0
      type: INTS
    }
  }
  node {
    output: "13"
    op_type: "Constant"
    attribute {
      name: "value"
      t {
        data_type: INT64
        raw_data: "\004\000\000\000\000\000\000\000"
      }
      type: TENSOR
    }
  }
  node {
    input: "6"
    input: "13"
    output: "14"
    op_type: "Div"
    attribute {
      name: "broadcast"
      i: 1
      type: INT
    }
    doc_string: "/home/dwofk/.local/lib/python3.5/site-packages/torch/tensor.py(347): __floordiv__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/functional.py(1835): pixel_shuffle\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/pixelshuffle.py(40): forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(465): _slow_forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(475): __call__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/container.py(91): forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(465): _slow_forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(475): __call__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/jit/__init__.py(109): forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(477): __call__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/jit/__init__.py(77): get_trace_graph\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(144): _trace_and_get_graph_from_model\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(177): _model_to_graph\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(226): _export\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(94): export\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/__init__.py(26): export\ntvm_compile.py(185): export_model\ntvm_compile.py(441): get_runtimes\ntvm_compile.py(504): main\ntvm_compile.py(507): <module>\n"
  }
  node {
    output: "15"
    op_type: "Constant"
    attribute {
      name: "value"
      t {
        data_type: INT64
        raw_data: "\002\000\000\000\000\000\000\000"
      }
      type: TENSOR
    }
  }
  node {
    input: "9"
    input: "15"
    output: "16"
    op_type: "Mul"
    attribute {
      name: "broadcast"
      i: 1
      type: INT
    }
    doc_string: "/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/functional.py(1837): pixel_shuffle\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/pixelshuffle.py(40): forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(465): _slow_forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(475): __call__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/container.py(91): forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(465): _slow_forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(475): __call__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/jit/__init__.py(109): forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(477): __call__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/jit/__init__.py(77): get_trace_graph\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(144): _trace_and_get_graph_from_model\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(177): _model_to_graph\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(226): _export\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(94): export\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/__init__.py(26): export\ntvm_compile.py(185): export_model\ntvm_compile.py(441): get_runtimes\ntvm_compile.py(504): main\ntvm_compile.py(507): <module>\n"
  }
  node {
    output: "17"
    op_type: "Constant"
    attribute {
      name: "value"
      t {
        data_type: INT64
        raw_data: "\002\000\000\000\000\000\000\000"
      }
      type: TENSOR
    }
  }
  node {
    input: "12"
    input: "17"
    output: "18"
    op_type: "Mul"
    attribute {
      name: "broadcast"
      i: 1
      type: INT
    }
    doc_string: "/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/functional.py(1838): pixel_shuffle\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/pixelshuffle.py(40): forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(465): _slow_forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(475): __call__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/container.py(91): forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(465): _slow_forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(475): __call__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/jit/__init__.py(109): forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(477): __call__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/jit/__init__.py(77): get_trace_graph\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(144): _trace_and_get_graph_from_model\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(177): _model_to_graph\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(226): _export\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(94): export\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/__init__.py(26): export\ntvm_compile.py(185): export_model\ntvm_compile.py(441): get_runtimes\ntvm_compile.py(504): main\ntvm_compile.py(507): <module>\n"
  }
  node {
    output: "19"
    op_type: "Constant"
    attribute {
      name: "value"
      t {
        data_type: INT64
        raw_data: "\002\000\000\000\000\000\000\000"
      }
      type: TENSOR
    }
  }
  node {
    output: "20"
    op_type: "Constant"
    attribute {
      name: "value"
      t {
        data_type: INT64
        raw_data: "\002\000\000\000\000\000\000\000"
      }
      type: TENSOR
    }
  }
  node {
    input: "3"
    output: "21"
    op_type: "Unsqueeze"
    attribute {
      name: "axes"
      ints: 0
      type: INTS
    }
  }
  node {
    input: "14"
    output: "22"
    op_type: "Unsqueeze"
    attribute {
      name: "axes"
      ints: 0
      type: INTS
    }
  }
  node {
    input: "19"
    output: "23"
    op_type: "Unsqueeze"
    attribute {
      name: "axes"
      ints: 0
      type: INTS
    }
  }
  node {
    input: "20"
    output: "24"
    op_type: "Unsqueeze"
    attribute {
      name: "axes"
      ints: 0
      type: INTS
    }
  }
  node {
    input: "9"
    output: "25"
    op_type: "Unsqueeze"
    attribute {
      name: "axes"
      ints: 0
      type: INTS
    }
  }
  node {
    input: "12"
    output: "26"
    op_type: "Unsqueeze"
    attribute {
      name: "axes"
      ints: 0
      type: INTS
    }
  }
  node {
    input: "21"
    input: "22"
    input: "23"
    input: "24"
    input: "25"
    input: "26"
    output: "27"
    op_type: "Concat"
    attribute {
      name: "axis"
      i: 0
      type: INT
    }
  }
  node {
    input: "0"
    input: "27"
    output: "28"
    op_type: "Reshape"
    doc_string: "/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/functional.py(1842): pixel_shuffle\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/pixelshuffle.py(40): forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(465): _slow_forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(475): __call__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/container.py(91): forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(465): _slow_forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(475): __call__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/jit/__init__.py(109): forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(477): __call__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/jit/__init__.py(77): get_trace_graph\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(144): _trace_and_get_graph_from_model\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(177): _model_to_graph\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(226): _export\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(94): export\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/__init__.py(26): export\ntvm_compile.py(185): export_model\ntvm_compile.py(441): get_runtimes\ntvm_compile.py(504): main\ntvm_compile.py(507): <module>\n"
  }
  node {
    input: "28"
    output: "29"
    op_type: "Transpose"
    attribute {
      name: "perm"
      ints: 0
      ints: 1
      ints: 4
      ints: 2
      ints: 5
      ints: 3
      type: INTS
    }
    doc_string: "/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/functional.py(1844): pixel_shuffle\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/pixelshuffle.py(40): forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(465): _slow_forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(475): __call__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/container.py(91): forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(465): _slow_forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(475): __call__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/jit/__init__.py(109): forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(477): __call__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/jit/__init__.py(77): get_trace_graph\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(144): _trace_and_get_graph_from_model\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(177): _model_to_graph\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(226): _export\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(94): export\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/__init__.py(26): export\ntvm_compile.py(185): export_model\ntvm_compile.py(441): get_runtimes\ntvm_compile.py(504): main\ntvm_compile.py(507): <module>\n"
  }
  node {
    input: "3"
    output: "30"
    op_type: "Unsqueeze"
    attribute {
      name: "axes"
      ints: 0
      type: INTS
    }
  }
  node {
    input: "14"
    output: "31"
    op_type: "Unsqueeze"
    attribute {
      name: "axes"
      ints: 0
      type: INTS
    }
  }
  node {
    input: "16"
    output: "32"
    op_type: "Unsqueeze"
    attribute {
      name: "axes"
      ints: 0
      type: INTS
    }
  }
  node {
    input: "18"
    output: "33"
    op_type: "Unsqueeze"
    attribute {
      name: "axes"
      ints: 0
      type: INTS
    }
  }
  node {
    input: "30"
    input: "31"
    input: "32"
    input: "33"
    output: "34"
    op_type: "Concat"
    attribute {
      name: "axis"
      i: 0
      type: INT
    }
  }
  node {
    input: "29"
    input: "34"
    output: "35"
    op_type: "Reshape"
    doc_string: "/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/functional.py(1845): pixel_shuffle\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/pixelshuffle.py(40): forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(465): _slow_forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(475): __call__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/container.py(91): forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(465): _slow_forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(475): __call__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/jit/__init__.py(109): forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(477): __call__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/jit/__init__.py(77): get_trace_graph\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(144): _trace_and_get_graph_from_model\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(177): _model_to_graph\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(226): _export\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(94): export\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/__init__.py(26): export\ntvm_compile.py(185): export_model\ntvm_compile.py(441): get_runtimes\ntvm_compile.py(504): main\ntvm_compile.py(507): <module>\n"
  }
  name: "torch-jit-export"
  input {
    name: "0"
    type {
      tensor_type {
        elem_type: FLOAT
        shape {
          dim {
            dim_value: 1
          }
          dim {
            dim_value: 32
          }
          dim {
            dim_value: 14
          }
          dim {
            dim_value: 14
          }
        }
      }
    }
  }
  output {
    name: "35"
    type {
      tensor_type {
        elem_type: FLOAT
        shape {
          dim {
            dim_value: 1
          }
          dim {
            dim_value: 8
          }
          dim {
            dim_value: 28
          }
          dim {
            dim_value: 28
          }
        }
      }
    }
  }
}
opset_import {
  version: 6
}

Running through the NNVM compiler also throws the following warnings before the compiler error is thrown.

Shape: Differently implemented in NNVM as a bypass (dummy operator)
Shape: Differently implemented in NNVM as a bypass (dummy operator)
Shape: Differently implemented in NNVM as a bypass (dummy operator)
Shape: Differently implemented in NNVM as a bypass (dummy operator)
#4

Understood, Thanks for the details.

The Shape operator earlier is a work around to support reshape case. Now it look like we should generalize it.
I see two options to handle this situation.

  • We could pass shape arguments to frontend which does an infer and transform the Shape to a param.
  • Or handle the Shape as part of precompute_prune (or before) to convert it to a param.

@tqchen , @masahi , @yzhliu and @Huyuwei welcome to comment and advice.

#5

Thank you, will be awaiting further comment.

#6

Hi, I was wondering whether there has been any update regarding support for a more generalized Shape layer in order for PixelShuffle to work in TVM.

Thanks!

#7

@dwofk will try to share a patch may be by tomorrow using first approach :slight_smile:

2 Likes
#8

Any updates here? We think the pixel shuffle layer can be very useful.

#9

@dwofk & @fangchangma can you try this above patch which generalizes the Shape operator.

#10

Thanks, I tried out the patch, and I no longer get the Error in operator strided_slice1 that I saw before. However, I still get an Error in operator concatenate0.

More specifically, I am compiling a model consisting of a single PixelShuffle layer, with an input shape of (1, 32, 14, 14). I also tried a larger batch size, i.e. input shape (16, 32, 14, 14). With both, I now get the following error:

nnvm._base.NNVMError: Error in operator concatenate0: [11:45:44] /home/dwofk/tvm/nnvm/src/top/tensor/transform.cc:120: Operator concatenate(axis=0, name=concatenate0) expects data1's shape to be [0,1], but got [1].

#11

This is tricky.

Can you share the onnx model?

#12

Yes, here is the onnx model for a PixelShuffle layer with input shape (1, 32, 14, 14).

ir_version: 3
producer_name: "pytorch"
producer_version: "0.3"
graph {
  node {
    input: "0"
    output: "1"
    op_type: "Shape"
  }
  node {
    input: "1"
    output: "2"
    op_type: "Slice"
    attribute {
      name: "axes"
      ints: 0
      type: INTS
    }
    attribute {
      name: "ends"
      ints: 1
      type: INTS
    }
    attribute {
      name: "starts"
      ints: 0
      type: INTS
    }
  }
  node {
    input: "2"
    output: "3"
    op_type: "Squeeze"
    attribute {
      name: "axes"
      ints: 0
      type: INTS
    }
  }
  node {
    input: "0"
    output: "4"
    op_type: "Shape"
  }
  node {
    input: "4"
    output: "5"
    op_type: "Slice"
    attribute {
      name: "axes"
      ints: 0
      type: INTS
    }
    attribute {
      name: "ends"
      ints: 2
      type: INTS
    }
    attribute {
      name: "starts"
      ints: 1
      type: INTS
    }
  }
  node {
    input: "5"
    output: "6"
    op_type: "Squeeze"
    attribute {
      name: "axes"
      ints: 0
      type: INTS
    }
  }
  node {
    input: "0"
    output: "7"
    op_type: "Shape"
  }
  node {
    input: "7"
    output: "8"
    op_type: "Slice"
    attribute {
      name: "axes"
      ints: 0
      type: INTS
    }
    attribute {
      name: "ends"
      ints: 3
      type: INTS
    }
    attribute {
      name: "starts"
      ints: 2
      type: INTS
    }
  }
  node {
    input: "8"
    output: "9"
    op_type: "Squeeze"
    attribute {
      name: "axes"
      ints: 0
      type: INTS
    }
  }
  node {
    input: "0"
    output: "10"
    op_type: "Shape"
  }
  node {
    input: "10"
    output: "11"
    op_type: "Slice"
    attribute {
      name: "axes"
      ints: 0
      type: INTS
    }
    attribute {
      name: "ends"
      ints: 4
      type: INTS
    }
    attribute {
      name: "starts"
      ints: 3
      type: INTS
    }
  }
  node {
    input: "11"
    output: "12"
    op_type: "Squeeze"
    attribute {
      name: "axes"
      ints: 0
      type: INTS
    }
  }
  node {
    output: "13"
    op_type: "Constant"
    attribute {
      name: "value"
      t {
        data_type: INT64
        raw_data: "\004\000\000\000\000\000\000\000"
      }
      type: TENSOR
    }
  }
  node {
    input: "6"
    input: "13"
    output: "14"
    op_type: "Div"
    attribute {
      name: "broadcast"
      i: 1
      type: INT
    }
    doc_string: "/home/dwofk/.local/lib/python3.5/site-packages/torch/tensor.py(347): __floordiv__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/functional.py(1835): pixel_shuffle\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/pixelshuffle.py(40): forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(465): _slow_forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(475): __call__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/container.py(91): forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(465): _slow_forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(475): __call__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/jit/__init__.py(109): forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(477): __call__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/jit/__init__.py(77): get_trace_graph\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(144): _trace_and_get_graph_from_model\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(177): _model_to_graph\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(226): _export\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(94): export\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/__init__.py(26): export\n/home/dwofk/sparse-to-dense.pytorch/tvm_compile/tvm_compile.py(179): export_model\n/home/dwofk/sparse-to-dense.pytorch/tvm_compile/tvm_compile.py(373): get_runtimes\n/home/dwofk/sparse-to-dense.pytorch/tvm_compile/tvm_compile.py(423): main\n/home/dwofk/sparse-to-dense.pytorch/tvm_compile/tvm_compile.py(426): <module>\n/usr/lib/python3.5/runpy.py(85): _run_code\n/usr/lib/python3.5/runpy.py(184): _run_module_as_main\n"
  }
  node {
    output: "15"
    op_type: "Constant"
    attribute {
      name: "value"
      t {
        data_type: INT64
        raw_data: "\002\000\000\000\000\000\000\000"
      }
      type: TENSOR
    }
  }
  node {
    input: "9"
    input: "15"
    output: "16"
    op_type: "Mul"
    attribute {
      name: "broadcast"
      i: 1
      type: INT
    }
    doc_string: "/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/functional.py(1837): pixel_shuffle\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/pixelshuffle.py(40): forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(465): _slow_forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(475): __call__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/container.py(91): forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(465): _slow_forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(475): __call__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/jit/__init__.py(109): forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(477): __call__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/jit/__init__.py(77): get_trace_graph\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(144): _trace_and_get_graph_from_model\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(177): _model_to_graph\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(226): _export\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(94): export\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/__init__.py(26): export\n/home/dwofk/sparse-to-dense.pytorch/tvm_compile/tvm_compile.py(179): export_model\n/home/dwofk/sparse-to-dense.pytorch/tvm_compile/tvm_compile.py(373): get_runtimes\n/home/dwofk/sparse-to-dense.pytorch/tvm_compile/tvm_compile.py(423): main\n/home/dwofk/sparse-to-dense.pytorch/tvm_compile/tvm_compile.py(426): <module>\n/usr/lib/python3.5/runpy.py(85): _run_code\n/usr/lib/python3.5/runpy.py(184): _run_module_as_main\n"
  }
  node {
    output: "17"
    op_type: "Constant"
    attribute {
      name: "value"
      t {
        data_type: INT64
        raw_data: "\002\000\000\000\000\000\000\000"
      }
      type: TENSOR
    }
  }
  node {
    input: "12"
    input: "17"
    output: "18"
    op_type: "Mul"
    attribute {
      name: "broadcast"
      i: 1
      type: INT
    }
    doc_string: "/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/functional.py(1838): pixel_shuffle\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/pixelshuffle.py(40): forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(465): _slow_forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(475): __call__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/container.py(91): forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(465): _slow_forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(475): __call__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/jit/__init__.py(109): forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(477): __call__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/jit/__init__.py(77): get_trace_graph\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(144): _trace_and_get_graph_from_model\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(177): _model_to_graph\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(226): _export\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(94): export\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/__init__.py(26): export\n/home/dwofk/sparse-to-dense.pytorch/tvm_compile/tvm_compile.py(179): export_model\n/home/dwofk/sparse-to-dense.pytorch/tvm_compile/tvm_compile.py(373): get_runtimes\n/home/dwofk/sparse-to-dense.pytorch/tvm_compile/tvm_compile.py(423): main\n/home/dwofk/sparse-to-dense.pytorch/tvm_compile/tvm_compile.py(426): <module>\n/usr/lib/python3.5/runpy.py(85): _run_code\n/usr/lib/python3.5/runpy.py(184): _run_module_as_main\n"
  }
  node {
    output: "19"
    op_type: "Constant"
    attribute {
      name: "value"
      t {
        data_type: INT64
        raw_data: "\002\000\000\000\000\000\000\000"
      }
      type: TENSOR
    }
  }
  node {
    output: "20"
    op_type: "Constant"
    attribute {
      name: "value"
      t {
        data_type: INT64
        raw_data: "\002\000\000\000\000\000\000\000"
      }
      type: TENSOR
    }
  }
  node {
    input: "3"
    output: "21"
    op_type: "Unsqueeze"
    attribute {
      name: "axes"
      ints: 0
      type: INTS
    }
  }
  node {
    input: "14"
    output: "22"
    op_type: "Unsqueeze"
    attribute {
      name: "axes"
      ints: 0
      type: INTS
    }
  }
  node {
    input: "19"
    output: "23"
    op_type: "Unsqueeze"
    attribute {
      name: "axes"
      ints: 0
      type: INTS
    }
  }
  node {
    input: "20"
    output: "24"
    op_type: "Unsqueeze"
    attribute {
      name: "axes"
      ints: 0
      type: INTS
    }
  }
  node {
    input: "9"
    output: "25"
    op_type: "Unsqueeze"
    attribute {
      name: "axes"
      ints: 0
      type: INTS
    }
  }
  node {
    input: "12"
    output: "26"
    op_type: "Unsqueeze"
    attribute {
      name: "axes"
      ints: 0
      type: INTS
    }
  }
  node {
    input: "21"
    input: "22"
    input: "23"
    input: "24"
    input: "25"
    input: "26"
    output: "27"
    op_type: "Concat"
    attribute {
      name: "axis"
      i: 0
      type: INT
    }
  }
  node {
    input: "0"
    input: "27"
    output: "28"
    op_type: "Reshape"
    doc_string: "/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/functional.py(1842): pixel_shuffle\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/pixelshuffle.py(40): forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(465): _slow_forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(475): __call__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/container.py(91): forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(465): _slow_forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(475): __call__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/jit/__init__.py(109): forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(477): __call__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/jit/__init__.py(77): get_trace_graph\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(144): _trace_and_get_graph_from_model\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(177): _model_to_graph\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(226): _export\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(94): export\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/__init__.py(26): export\n/home/dwofk/sparse-to-dense.pytorch/tvm_compile/tvm_compile.py(179): export_model\n/home/dwofk/sparse-to-dense.pytorch/tvm_compile/tvm_compile.py(373): get_runtimes\n/home/dwofk/sparse-to-dense.pytorch/tvm_compile/tvm_compile.py(423): main\n/home/dwofk/sparse-to-dense.pytorch/tvm_compile/tvm_compile.py(426): <module>\n/usr/lib/python3.5/runpy.py(85): _run_code\n/usr/lib/python3.5/runpy.py(184): _run_module_as_main\n"
  }
  node {
    input: "28"
    output: "29"
    op_type: "Transpose"
    attribute {
      name: "perm"
      ints: 0
      ints: 1
      ints: 4
      ints: 2
      ints: 5
      ints: 3
      type: INTS
    }
    doc_string: "/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/functional.py(1844): pixel_shuffle\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/pixelshuffle.py(40): forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(465): _slow_forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(475): __call__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/container.py(91): forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(465): _slow_forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(475): __call__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/jit/__init__.py(109): forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(477): __call__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/jit/__init__.py(77): get_trace_graph\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(144): _trace_and_get_graph_from_model\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(177): _model_to_graph\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(226): _export\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(94): export\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/__init__.py(26): export\n/home/dwofk/sparse-to-dense.pytorch/tvm_compile/tvm_compile.py(179): export_model\n/home/dwofk/sparse-to-dense.pytorch/tvm_compile/tvm_compile.py(373): get_runtimes\n/home/dwofk/sparse-to-dense.pytorch/tvm_compile/tvm_compile.py(423): main\n/home/dwofk/sparse-to-dense.pytorch/tvm_compile/tvm_compile.py(426): <module>\n/usr/lib/python3.5/runpy.py(85): _run_code\n/usr/lib/python3.5/runpy.py(184): _run_module_as_main\n"
  }
  node {
    input: "3"
    output: "30"
    op_type: "Unsqueeze"
    attribute {
      name: "axes"
      ints: 0
      type: INTS
    }
  }
  node {
    input: "14"
    output: "31"
    op_type: "Unsqueeze"
    attribute {
      name: "axes"
      ints: 0
      type: INTS
    }
  }
  node {
    input: "16"
    output: "32"
    op_type: "Unsqueeze"
    attribute {
      name: "axes"
      ints: 0
      type: INTS
    }
  }
  node {
    input: "18"
    output: "33"
    op_type: "Unsqueeze"
    attribute {
      name: "axes"
      ints: 0
      type: INTS
    }
  }
  node {
    input: "30"
    input: "31"
    input: "32"
    input: "33"
    output: "34"
    op_type: "Concat"
    attribute {
      name: "axis"
      i: 0
      type: INT
    }
  }
  node {
    input: "29"
    input: "34"
    output: "35"
    op_type: "Reshape"
    doc_string: "/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/functional.py(1845): pixel_shuffle\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/pixelshuffle.py(40): forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(465): _slow_forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(475): __call__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/container.py(91): forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(465): _slow_forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(475): __call__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/jit/__init__.py(109): forward\n/home/dwofk/.local/lib/python3.5/site-packages/torch/nn/modules/module.py(477): __call__\n/home/dwofk/.local/lib/python3.5/site-packages/torch/jit/__init__.py(77): get_trace_graph\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(144): _trace_and_get_graph_from_model\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(177): _model_to_graph\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(226): _export\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/utils.py(94): export\n/home/dwofk/.local/lib/python3.5/site-packages/torch/onnx/__init__.py(26): export\n/home/dwofk/sparse-to-dense.pytorch/tvm_compile/tvm_compile.py(179): export_model\n/home/dwofk/sparse-to-dense.pytorch/tvm_compile/tvm_compile.py(373): get_runtimes\n/home/dwofk/sparse-to-dense.pytorch/tvm_compile/tvm_compile.py(423): main\n/home/dwofk/sparse-to-dense.pytorch/tvm_compile/tvm_compile.py(426): <module>\n/usr/lib/python3.5/runpy.py(85): _run_code\n/usr/lib/python3.5/runpy.py(184): _run_module_as_main\n"
  }
  name: "torch-jit-export"
  input {
    name: "0"
    type {
      tensor_type {
        elem_type: FLOAT
        shape {
          dim {
            dim_value: 1
          }
          dim {
            dim_value: 32
          }
          dim {
            dim_value: 14
          }
          dim {
            dim_value: 14
          }
        }
      }
    }
  }
  output {
    name: "35"
    type {
      tensor_type {
        elem_type: FLOAT
        shape {
          dim {
            dim_value: 1
          }
          dim {
            dim_value: 8
          }
          dim {
            dim_value: 28
          }
          dim {
            dim_value: 28
          }
        }
      }
    }
  }
}
opset_import {
  version: 6
}
#13

Can you share in .onnx format?

It’s would be easy to replace in from_onnx.py and check.

#14

Do you mean the actual exported file? I’m linking it here. I used the .proto extension instead of .onnx when exporting.

1 Like
#15

@dwofk

I pushed a small bug fix for the concat issue.
It progressed further and ended up with another bigger issue.

The reshape operator with shape as an input tensor is not supported in current architecture.

I would need to think a bit more to address this situation in current fixed shape based compilation process.

#16

Thank you very much @srkreddy1238. Really appreciate you looking into it.

If any updates to the compilation process will be made that may make supporting this operator feasible, please do let us know. We’re very interested in using TVM to compile network designs with PixelShuffle layers, and will be hoping for any potential updates.

Thanks again!