Is there any monitor tool for cluster system resources ?

[salemmohammed]

Hi all,

I am running MXNet on a cluster machines. Is there any code or tool in MXNet system provide how resources are used ? example how much CPU, Memory, and network are being used while I am doing the model training.

I know there are many outside monitoring cluster tool such as Ganglia but I am wondering if there is something built-in.

Sincerely

[salemmohammed]

@ piiswrong

[formath]

I think MXNet should be just for computation. Other parts like resource management and schedule should be done in cluster management system like Yarn or k8s. Right?

[salemmohammed]

@formath
Right but Yarn is not going to work with ML because it's iterative in nature.

[formath]

@salemmohammed Yarn is ok for ML like TensorFlow on Yarn or MXNet on Yarn. I guess your meaning is MR or Spark on Yarn is not ok for ML. The only thing that MXNet on Yarn not works to your purpose is that the Yarn part of dmlc-core is very simple. You need a more complicated Yarn application to run MXNet. I have one in my company but now it's not open sourced.

[szha]

@apache/mxnet-committers: This issue has been inactive for the past 90 days. It has no label and needs triage.

For general "how-to" questions, our user forum (and Chinese version) is a good place to get help.

[zhiz21]

proposed labels: "Question", "Discussion", "Feature request"