Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add DefaultGraphStack #175

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

alexpantyukhin
Copy link
Contributor

No description provided.

@alexpantyukhin alexpantyukhin force-pushed the default_graph_storage branch 3 times, most recently from 38197c7 to 9e17ab1 Compare November 18, 2017 12:50
@migueldeicaza
Copy link
Owner

While I do like the principle, you just ran into the core problem that comes with this.

It is still cumbersome to use, because only a handful of operators can be overloaded, but the operations on the TFGraph are still ugly to invoke as an instance is still required.

One possible option (but I am not sure that I want to do that yet) is to create a "Graph" API, and make all the methods in that class static, allowing us to do "using static Tensorflow.Graph" and that could be a place where we could bring something like this.

@alexpantyukhin
Copy link
Contributor Author

In the python implementation Graph object is created by DefaultStack when there is no instance yet. Maybe we need something like this? Also I didn't see explicit createing Graph object in usage of python impl.

@cesarsouza
Copy link
Contributor

cesarsouza commented Nov 21, 2017

Hi @migueldeicaza,

Just to let you know, I am doing something similar to the option you have suggested within Keras#, if you would like to see how this approach would look.

In most of Keras# I static import a class containing a static property K that allows me to use the graph operations through it. I suppose that in the case of TF#, this could have been the DefaultGraphStack in this PR.

But then for operator overloading to work, I've created a Tensor class that wraps TFOutput and keeps a reference to which graph this TFOutput actually belongs to, and wrote the operator overloading methods such that they would use a graph from whichever argument it would be available.

I am not completely sure if this would work for all scenarios though.

Regards,
Cesar

@migueldeicaza
Copy link
Owner

I see what you are doing, it is just not clear to me that doing K.Operation is much better than g.Operation -- here I am assuming that the user would have done var g = new TFGraph. Perhaps the benefit is that there is a notion of a global, and users do not need to pass this TFGraph around.

If I end up going this way, it would likely take the form of the generator producing an additional set of methods that are proxies to the existing TFGraph methods.

I like where you are going with KerasSharp on operators. Let me ponder a bit how that would look like.

@cesarsouza
Copy link
Contributor

cesarsouza commented Nov 22, 2017

Hi Miguel,

it is just not clear to me that doing K.Operation is much better than g.Operation

Well, I am not completely sure myself either. I like TFS current approach because it is much more explicit, and you have less to wonder about what is going on under the hood when programming against it. Took me a while to figure out where all the magic from Keras+TF was coming from when I was using it from Python.

In Keras# it makes sense because I am using it to support the different backends (and because I wanted to porsue that idea of having a line-by-line port), but I am not sure it is as important in TFS. Actually, I think that the main issue in TFS is that most users are coming from Python and are looking for something that can be instantly recognizable given their past experience. And for this, I think there is something super simple that could be done: I suppose that users would feel instantly at home if the TFGraph variables were called "tf" in the examples currently in TFS, because they would instantly recognize that all tf.* operations from Python would be available from this scaring TFGraph class they have just read about. And I am supposing here that this would have almost the same benefit as having the notion of a global, since users would see things like tf.MatMul and immediately know how to use them.

Regarding the operators: I have more interesting examples in the learning algorithms if you would like to see how it looks like in practice (I forgot to link in my previous comment). I need to say that it is not completely possible to get rid of the explicit references to TFGraph because sometimes you need to pass more parameters to the operation (i.e. an explicit name).

The main problem with this approach is that I do not really know how feasible it is to make every TFOutput/TFTensor carry a reference to the TFGraph that created them. So far I haven't found any fundamental problem with this, but I guess I am just scratching the surface of TF's APIs at this point.

Regards,
Cesar

@alexpantyukhin
Copy link
Contributor Author

Should we close this PR?

@migueldeicaza
Copy link
Owner

I would like to keep this open, as I am still pondering what to do about this.

I do not have a good answer, but I would like to keep this as a reference as I think about this. One option I am considering is introducing a static class "EasyGraph" that would surface a bunch of methods that would allow a scenario like this to work.

Easygraph would have to replicate every bound method in TFGraph.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants