Are external operations well-supported/tested in relay?

chuck-pilkington · July 5, 2019, 7:38pm

It would be good if the docs contain an example (similar to https://docs.tvm.ai/tutorials/language/extern_op.html#sphx-glr-tutorials-language-extern-op-py) of how to use external operations when using the relay interface. Also, not sure if this is well tested?

For example, I registered an extern relay op via RELAY_REGISTER_OP(“snps.subgraph”), and called it as per:

data1 = relay.var("data1", shape=data_shape)
data2 = relay.var("data2", shape=data_shape)
body = snps_subgraph(data=data1)
body = relay.add(body, data2)

this seems to run as expected. However, if I put the add operation before the snps_subgraph, I get errors during the schedule_injective processing. As I am very new to this stuff, not sure what is going wrong, if I am missing something, or if this is simply not yet supported.

jroesch · July 5, 2019, 9:35pm

Did you set its fusion attribute?

github.com

dmlc/tvm/blob/master/src/relay/op/tensor/transform.cc#L295


- **axis** : The axis along which the tensors are concatenated.


)code" TVM_ADD_FILELINE)
.set_attrs_type_key("relay.attrs.ConcatenateAttrs")
.set_num_inputs(1)
.add_argument("data", "Tensor", "The input list of tensors.")
.set_support_level(1)
.add_type_rel("Concatenate", ConcatenateRel)
.set_attr<FInferCorrectLayout>("FInferCorrectLayout", ConcatenateLayout)
.set_attr<FTVMCompute>("FTVMCompute", ConcatenateCompute)
.set_attr<TOpPattern>("TOpPattern", kInjective);


TVM_REGISTER_NODE_TYPE(StackAttrs);


bool StackRel(const Array<Type>& types,
            int num_inputs,
            const Attrs& attrs,
            const TypeReporter& reporter) {
// types: [data, result]
CHECK_EQ(types.size(), 2);
const auto* tensor_tuple = types[0].as<TupleTypeNode>();

You must make sure the schedule is correctly designed to handle the fusion pattern, if you incorrectly specify an operator as fusable the fusion algorithm will fail when scheduling the operation (what you are experiencing).

For example if you use a TVM external operation it should be marked opaque (i.e kOpaque) as it is not fusable.

If you have a reproducible error after correctly marking the operator, please send us a test case and we can introduce it as a regression.

chuck-pilkington · July 8, 2019, 5:40pm

Thanks for your help, indeed setting TOpPattern to kOpaque solved my issue.