Hi,
In this RFC i would like to propose new Frontend to support Chainer framework.
I have already raised one WIP PR.
Currently above PR is ready with followings:
1:> Tracing Chainer Graph.
2:> Convert all Params and Variables to TVM Vars.
3:> A naive converter map from Chainer to TVM Op.
I have multiple TODOs as well:
1:> Support Input provided names.
2:> Support multiple inputs.
3:> Support one model(Say Resnet) end-to-end.
4:> Form Tutorial.
The current code is tested successfully for RELU node alone(for the same test case also added).
I seek all TVM members expert advice here.
Thanks a lot!
besides the technical details, which we could cover a bit more here, it would be great to discuss the motivation, technical alternatives and possible engineering cost
Background:
Chainer is python based framework which provides a simple and intuitive and highly flexible framework for deep learning (Please refer).
Motivation:
My motivation to bring this frontend draws from 2 parts of benefits
Benefit from Chainer prospective:
As we all know the Chainer framework has stopped future development. Currently sits only on Maintenance phase. But that does not mean the existing users has reduced to 0.
Chainer framework does not have wide variety of target deployment like TVM. So definitely Chainer user can benefit from that.
Chainer does not have its own IR, so it can leverage TVM provided benefits for that.
Existing Chainer model zoo or if someone has his own researched and developed model, can be exported to TVM and provided in a production ready environment(makes it more reliable).
As the development is stopped now, so if someone is blocked in Chainer in terms of some runtime issue, can be solved through this new frontend bridge.
Benefit from TVM prospective:
Chainer has highly flexible training framework(as it support define-by-run methodology), i am pretty sure there will many technical challenges in implementing the Frontend. This will give us opportunity to test the TVM Relay strength for express-ability (or in other terms versatility).
This will help fellow Opensource community member to migrate using towards TVM(increasing TVM fan base ).
Alternatives:
Currently Chainer framework supports ONNX exporting.
So if any Chainer user wants to port into TVM, the path can be
Chainer -> ONNX -> TVM.
However as earlier mentioned the development is already stopped, so if any user faces any issue then above path is broken as there is no active support. So one has to re-implement everything from scratch in another framework.
Engineering Cost:
I am not sure how to put it. May be we can discuss about it(as it is first time i proposed in TVM community).
But for start i can say, it will take 2~3 weeks(may be sooner), to make the frontend ready for one end-to-end model importing. Then forth it can be taken forward to support all kinds.
Please correct me if any of above points are wrong. Its a learning always
I am excited to get expert opinions and advice from all TVM members.
Thanks for the discussion and offering to work on implementing this, @ANSHUMAN.TRIPATHY. Given that Chainer is no longer being developed, it seems like the ONNX converter should be very stable. Where would new issues be introduced? Can you give any examples of current problems when importing to TVM through ONNX?
@jwfromm: Thank you very much for sharing your view on this.
As i recall ONNX-Chainer has limited support in their current code. Please refer Code or Tutorial for the list.
So you can imagine there are some gaps to fill.
Apart from that, i faced some issues while trying to convert RCNN related models from Chainer to ONNX. Which seems their long standing issue.
Also to share my findings, i got many user posts who are trying to convert from Chainer to ONNX to TVM, where they faced lot of troubles. Which gave me confidence in bringing this frontend.
Hope my reply is up to your expectation. In case any thing else required. Please let me know. Thanks!
Your proposal is reasonable from perspective of existing Chainer users looking for deployment solution. However, I see little benefit from TVM side.
First of all, compared to other framework like PyTorch, the number of Chainer user had been very tiny even during Chainer was in active development. Now that Chainer development is in maintance mode, I would estimate the number of active Chainer users is close to zero. So “bringng Chainer users to TVM” doesn’t really appeal to us. The only reasonable use case is the need to deploy existing Chainer models. However, developing something entirely new in TVM to support such a legacy use case doesn’t seem right to me.
ONNX should be a good fit for Chainer users (who do not want to switch to PyTorch for some reason). Take a look at chainer-compiler https://github.com/pfnet-research/chainer-compiler. They use ONNX as a bridge between Chainer and their compiler. That means ONNX is “the” official IR adopted by the Chainer team. It also means their support of ONNX export should be good. You are correct in that chainer-onnx is not really in a good state, so the chainer-compiler team is developing a new ONNX export tool (actually two, see “ch2o” and “elichika” in https://github.com/pfnet-research/chainer-compiler/tree/master/chainer_compiler). Chainer -> ONNX path enables using our relatively mature and well supported ONNX frontend.
If you find a problem in Chainer to ONNX export, you should report or send a patch to them. If they are not supporting ONNX export either, maybe it is time for you to reconsider your decision of continuing to use Chainer. Rewriting your models to PyTorch should be more straightforward and better decision than developing another Chainer exporter or converter in TVM.
You need to be more specific than “pretty sure”. Chainer’s flexible API has been adopted by other frameworks and it is not unique to Chainer anymore. One of the interesting challenges when converting from “dynamic” frameworks is control flow. For that, we already demonstrated in our Torch frontend.
Note that if you use tracing, by definition you lose control flow and tracing only frontend is not really interesting. Preserving control flow is highly non trivial, you need to start by parsing Python source code to get an Python AST. (This is what Torchscript and chainer-compiler above do to preserve control flow) Are you prepared for this? Even if you are, it is out of scope from TVM anyway.
If you really want to develop this frontend, you can develop it as a utility library building on top of TVM. It doesn’t have to live inside TVM repo, since everything frontend code need is available in “userland”.
Thank you very much for sharing your views on this. It was really insightful.
Few points i want to reply as below:
Response: I agree Chainer has very less no of users compared to Pytorch. But maintenance phase does not mean user count is 0, however i cant put an official statement to it, as there is no record of showing how many user currently using it.
Deciding which frontend to support based on no of users, i believe should not be done, as there are various other dynamics to it. However i leave it to TVM members consensus.
Response: Thanks for the advice! But don’t you think the “out of scope from TVM” is applicable for all frontends. As all frontends first need to beakdown the model received and put all the pieces together to in terms of TVM.
Response: I am really not sure what you are trying to communicate here.
As i understand the TVM architecture and use cases. Frontends are never meant for TVM. Even without Frontends TVM core exists. TVM core never gains any benefit from adding new frontends except the points i mentioned in my second post.
As i understand the Frontends are mere gateways for other framework users to enable them availing the prominent benefits of TVM.
As you mentioned utility part here, i believe that is applicable for all frontends then.
My point here is TVM frontend should be open to all kind of users. More frontends shows more strength of TVM. And Chainer is not just a random experimental framework, it was released officially and used by multiple organizations as well.
I leave it to TVM member consensus , if they think there is no room any more for new frontend. Thanks!
No. For example, both Torchscript and ONNX has “If” and “While” IR node etc. to support control flow. It is framework’s job to lower Python control flow to corresponding IR constructs and TVM’s job is to convert these control flow IR constructs to Relay equivalent. But for you, Chainer lacks the first part, so you need to implement it if you want to support control flow. This should not be a TVM dev’s concern and that’s what I mean by “out of scope” of TVM. chainer-compiler already can already translate Python control flow to ONNX, so we shouldn’t duplicate this effort.
Yes, in principle frontend code can be separated from main TVM repo. But for frontends that have been justified to be added to TVM, it makes user’s and developer’s lives much easier if we put frontend code inside TVM (no need to install separate packages, prevent API break etc.).
But right now we are talking about whether or not to add Chainer frontend to TVM. What I mean is, even if we decide that we do not want to add chainer to our frontend, you can still go ahead and continue developing it outside of TVM. If enabling Chainer -> TVM path is your goal, this should be an acceptable solution.
Trying to summarizing the discussions, thanks @ANSHUMAN.TRIPATHY for bringing up the RFC :
S0: Currently chainer deployment can go through the ONNX path. chainer->ONNX->tvm allows us to leverage the more robust ONNX path.
S1: Because chainer -> ONNX stops developments, there is a concern that chainer->ONNX may no longer keep up to date, and was the main argument for introducing the chainer frontend.
Both arguments has valid points, it would be great to hear thoughts from others. In the meanwhile, @masahi suggested, such conversion support could always be added as a library built on top of the tvm
I agree with @masahi, it’s not obvious to me that the onnx importer will stop working and maintaining a new frontend is a lot of ongoing work. +1 to implementing this on top of TVM rather than inside of it if @ANSHUMAN.TRIPATHY wants to pursue this effort.
Agree with @masahi & @jwfromm. Looking at the limited users and no ongoing activity in Chainer community my take here is to keep Chainer above TVM. Also the effort of enhancing Chainer-ONNX converter may be lot less than a new direct frontend in TVM. Enhancing Chainer exporter benefits users of Chainer->ONNX -> Any other framework as ONNX is a goof bridge for many widely used frameworks.
).