You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The tensorboard profiling capabilities are great, but it can be hard to tell precisely where in your computational graph the slowness comes from. To help with that, it would be fantastic if it were possible for the profiling tool to include some additional categorization of the performance data:
Associate ops & specific time spent with each phase (forward, loss, backward, optimizer) and each layer (Conv2d, BatchNorm, ReLU, etc) in the graph hierarchy.
Ideally the block diagram of the graph is hierarchically captured so that you can view ops & timings for e.g. your ResNetBlock1, then drill down to whatever constituent Conv/BN/ReLU/other blocks that may be within, recursively.
The training & inference time consumption can be very non-uniform across a graph, and having this breakdown would allow practitioners to pin-point specific problem-blocks in their networks, hopefully assisting in producing more efficient networks.
The tensorboard profiling capabilities are great, but it can be hard to tell precisely where in your computational graph the slowness comes from. To help with that, it would be fantastic if it were possible for the profiling tool to include some additional categorization of the performance data:
Associate ops & specific time spent with each phase (forward, loss, backward, optimizer) and each layer (Conv2d, BatchNorm, ReLU, etc) in the graph hierarchy.
Ideally the block diagram of the graph is hierarchically captured so that you can view ops & timings for e.g. your ResNetBlock1, then drill down to whatever constituent Conv/BN/ReLU/other blocks that may be within, recursively.
The training & inference time consumption can be very non-uniform across a graph, and having this breakdown would allow practitioners to pin-point specific problem-blocks in their networks, hopefully assisting in producing more efficient networks.
Thanks!
(coming from tensorflow/tensorboard#5749 - reposted here by request)
The text was updated successfully, but these errors were encountered: