./compile.sh: line 53: play: command not found #700

vempadh · 2020-09-25T09:18:16Z

[root@azitshdpmst01p dr-elephant-master]# ./compile.sh
Checking for required programs...
Program requirement is fulfilled!
Using the default configuration
Hadoop Version : 2.3.0
Spark Version : 1.4.0
Other opts set :
############################################################################
npm installation found, we'll compile with the new user interface
############################################################################

sleep 3
ember_assets=/usr/local/dr-elephant-master/public/assets
ember_resources_dir=/usr/local/dr-elephant-master/public/assets/ember
ember_web_directory=/usr/local/dr-elephant-master/web
cd /usr/local/dr-elephant-master/web
npm install
up to date in 5.689s
node_modules/bower/bin/bower install
bower ESUDO Cannot be run with sudo

Additional error details:
Since bower is a user command, there is no need to execute it with superuser permissions.
If you're having permission errors when using bower without sudo, please spend a few minutes learning more about how your system should work and make any necessary repairs.

http://www.joyent.com/blog/installing-node-and-npm
https://gist.github.com/isaacs/579814

You can however run a command with sudo using "--allow-root" option

node_modules/ember-cli/bin/ember build --prod
WARNING: Node v8.1.1 has currently not been tested against Ember CLI and may result in unexpected behaviour.
Could not start watchman; falling back to NodeWatcher for file system events.
Visit http://ember-cli.com/user-guide/#watchman for more info.
Built project successfully. Stored in "dist/".
File sizes:

dr-elephant.css: 8.61 KB (2.49 KB gzipped)
dr-elephant.js: 277.74 KB (28.33 KB gzipped)
aggregated-metrics.css: 207 B (156 B gzipped)
application-menu.css: 334 B (218 B gzipped)
application.css: 34 B (54 B gzipped)
breadcrumbs.css: 1.36 KB (566 B gzipped)
dashboard-header.css: 609 B (340 B gzipped)
heuristic-detail.css: 127 B (109 B gzipped)
heuristics-summary.css: 536 B (281 B gzipped)
jobs-severity.css: 333 B (202 B gzipped)
loading-panel.css: 129 B (126 B gzipped)
navbar.css: 920 B (282 B gzipped)
not-found.css: 76 B (74 B gzipped)
search-panel.css: 39 B (59 B gzipped)
search.css: 569 B (300 B gzipped)
user-detail.css: 988 B (435 B gzipped)
user-tabs.css: 725 B (321 B gzipped)
user-tags.css: 76 B (75 B gzipped)
vendor.css: 143.24 KB (24.08 KB gzipped)
vendor.js: 1.22 MB (357.84 KB gzipped)

rm -r /usr/local/dr-elephant-master/public/assets/ember
mkdir /usr/local/dr-elephant-master/public/assets/ember
cp dist/assets/dr-elephant.css /usr/local/dr-elephant-master/public/assets/ember/
cp dist/assets/dr-elephant.js /usr/local/dr-elephant-master/public/assets/ember/
cp dist/assets/vendor.js /usr/local/dr-elephant-master/public/assets/ember/
cp dist/assets/vendor.css /usr/local/dr-elephant-master/public/assets/ember/
cp -r dist/fonts /usr/local/dr-elephant-master/public/assets/
cd /usr/local/dr-elephant-master
trap exit SIGINT SIGTERM
set +x
Command is: play -Dhadoopversion=2.3.0 -Dsparkversion=1.4.0 clean compile test
./compile.sh: line 53: play: command not found
Build failed...
[root@azitshdpmst01p dr-elephant-master]#

vempadh · 2020-09-25T11:15:28Z

Hi Guys,
Please help me on this.
When i try to run ./compile.sh and getting error with Play command.

Run the main command alongwith the extra commands passed as arguments to compile.sh

echo "Command is: play $OPTS clean compile test $extra_commands"
play_command $OPTS clean compile test $extra_commands
if [ $? -ne 0 ]; then
echo "Build failed..."
exit 1;
fi

nelhaj · 2020-09-25T17:19:01Z

Hi,

You must have play or activator command installed.
Please follow Dr-Elephant Setup Instructions :
Quick-Setup-Instructions-(Must-Read)

PS : duplicated issue : 44

vempadh · 2020-09-26T04:00:42Z

Hi,
Play command is working after installation. But i was facing different issue after installation of Play.

[info] Compiling 98 Scala sources and 160 Java sources to /usr/local/dr-elephant-master/target/scala-2.10/classes...
[warn] /usr/local/dr-elephant-master/app/org/apache/spark/deploy/history/SparkDataCollection.scala:313: abstract type pattern T is unchecked since it is eliminated by erasure
[warn] seq.foreach { case (item: T) => list.add(item)}
[warn] ^
[warn] one warning found
java.io.IOException: Cannot run program "javac" (in directory "/usr/local/dr-elephant-master"): error=2, No such file or directory

Can you please help me on this.

vempadh · 2020-09-26T04:01:21Z

[error] (compile:compileIncremental) java.io.IOException: Cannot run program "javac" (in directory "/usr/local/dr-elephant-master"): error=2, No such file or directory
[error] Total time: 22 s, completed Sep 26, 2020 3:51:53 AM
Build failed..

vempadh · 2020-09-26T04:49:28Z

Can anyone help me on this. When i running compile.sh script and throwing below error in SparkDataCollection file in below path and error.

[warn] /usr/local/dr-elephant-master/app/org/apache/spark/deploy/history/SparkDataCollection.scala:313: abstract type pattern T is unchecked since it is eliminated by erasure
[warn] seq.foreach { case (item: T) => list.add(item)}
[warn] ^
[warn] one warning found
java.io.IOException: Cannot run program "javac" (in directory "/usr/local/dr-elephant-master"): error=2, No such file or directory

ShubhamGupta29 · 2020-09-28T19:38:13Z

@vempadh the real issue is

java.io.IOException: Cannot run program "javac" (in directory "/usr/local/dr-elephant-master"): error=2, No such file or directory

For SparkDataCollection, there is just a warning and nothing else. I have never faced this issue so not sure how to resolve this issue. From the message seems like Java is not available in the path. Kindly check by executing any simple Java script.

vempadh · 2020-09-29T09:24:11Z

@ShubhamGupta29 : Thanks for your replay.

We resolved the issue and started Dr.Elephant and can be see the Web UI as well. But we didn't find any jobs in UI level.
How do we fetch Spark and Hive Jobs in Elephant web UI?

Any suggestions on this.

vempadh · 2020-09-29T09:41:42Z

I was facing below error because of unable to see the Spark and Hive jobs in UI.

2020-09-29 09:00:33,745 - [ERROR] - from play.nettyException in New I/O worker #7
Exception caught in Netty
java.lang.IllegalArgumentException: invalid version format: H￲ﾣﾃ&>Aￃ￰LﾱￏￊPﾉ;}ￜEWﾃ￲*)+ￔ￷ﾩDﾃ>Eﾆﾜ(E￐￷E￞ ￺￺￀+￀/￀,￀0ￌﾩￌﾨ￀￀ﾜﾝ/5ﾓ▒
at org.jboss.netty.handler.codec.http.HttpVersion.(HttpVersion.java:102) ~[io.netty.netty-3.8.0.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpVersion.valueOf(HttpVersion.java:62) ~[io.netty.netty-3.8.0.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpRequestDecoder.createMessage(HttpRequestDecoder.java:75) ~[io.netty.netty-3.8.0.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpMessageDecoder.decode(HttpMessageDecoder.java:189) ~[io.netty.netty-3.8.0.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpMessageDecoder.decode(HttpMessageDecoder.java:101) ~[io.netty.netty-3.8.0.Final.jar:na]
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:500) ~[io.netty.netty-3.8.0.Final.jar:na]
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:435) ~[io.netty.netty-3.8.0.Final.jar:na]
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) ~[io.netty.netty-3.8.0.Final.jar:na]
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) ~[io.netty.netty-3.8.0.Final.jar:na]
at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) ~[io.netty.netty-3.8.0.Final.jar:na]
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108) ~[io.netty.netty-3.8.0.Final.jar:na]
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318) ~[io.netty.netty-3.8.0.Final.jar:na]
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89) ~[io.netty.netty-3.8.0.Final.jar:na]
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) ~[io.netty.netty-3.8.0.Final.jar:na]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_262]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_262]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_262]

vempadh · 2020-09-29T16:28:17Z

Hi All,
Can you please help me on the below issue.

09-29-2020 12:40:51 ERROR [dr-el-executor-thread-0] com.linkedin.drelephant.ElephantRunner : Failed to analyze TEZ application_1599981649784_4661
java.io.FileNotFoundException: http://azitshdpmst03p.ecolab.com:8188/ws/v1/timeline/TEZ_APPLICATION/tez_application_1599981649784_4661

Caused by: java.net.UnknownHostException: null

ShubhamGupta29 · 2020-09-29T18:43:04Z

@vempadh seems like the endpoint which Dr.Elephant is trying to call is not available, kindly check if some information exists at the link which you provided above as the failure reason.

ShubhamGupta29 · 2020-09-29T18:45:13Z

I was facing below error because of unable to see the Spark and Hive jobs in UI.

2020-09-29 09:00:33,745 - [ERROR] - from play.nettyException in New I/O worker #7
Exception caught in Netty
java.lang.IllegalArgumentException: invalid version format: H￲ﾣﾃ&>Aￃ￰LﾱￏￊPﾉ;}ￜEWﾃ￲*)+ￔ￷ﾩDﾃ>Eﾆﾜ(E￐￷E￞ ￺￺￀+￀/￀,￀0ￌﾩￌﾨ￀￀ﾜﾝ/5ﾓ▒
at org.jboss.netty.handler.codec.http.HttpVersion.(HttpVersion.java:102) ~[io.netty.netty-3.8.0.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpVersion.valueOf(HttpVersion.java:62) ~[io.netty.netty-3.8.0.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpRequestDecoder.createMessage(HttpRequestDecoder.java:75) ~[io.netty.netty-3.8.0.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpMessageDecoder.decode(HttpMessageDecoder.java:189) ~[io.netty.netty-3.8.0.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpMessageDecoder.decode(HttpMessageDecoder.java:101) ~[io.netty.netty-3.8.0.Final.jar:na]
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:500) ~[io.netty.netty-3.8.0.Final.jar:na]
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:435) ~[io.netty.netty-3.8.0.Final.jar:na]
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) ~[io.netty.netty-3.8.0.Final.jar:na]
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) ~[io.netty.netty-3.8.0.Final.jar:na]
at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) ~[io.netty.netty-3.8.0.Final.jar:na]
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108) ~[io.netty.netty-3.8.0.Final.jar:na]
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318) ~[io.netty.netty-3.8.0.Final.jar:na]
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89) ~[io.netty.netty-3.8.0.Final.jar:na]
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) ~[io.netty.netty-3.8.0.Final.jar:na]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_262]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_262]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_262]

What was the reason for this issue and how did you resolve it? It would be good if you can mention details so if some other user faces this issue than would be unblocked by your resolution.
Thanks.

vempadh · 2020-09-30T07:36:40Z

@ShubhamGupta29 : Still not yet resolved this issue.
Can you please help me on this. I verified all github links and didn't found any solution for this.

[error] p.nettyException - Exception caught in Netty
java.lang.IllegalArgumentException: invalid version format: ￥Vﾴￂﾗ'Nﾺﾏ￭ￄ9▒%ￜﾥVﾗￇ"￺￺￀+￀/￀,￀0ￌﾩￌﾨ￀￀ﾜﾝ/5
at org.jboss.netty.handler.codec.http.HttpVersion.(HttpVersion.java:102) ~[io.netty.netty-3.8.0.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpVersion.valueOf(HttpVersion.java:62) ~[io.netty.netty-3.8.0.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpRequestDecoder.createMessage(HttpRequestDecoder.java:75) ~[io.netty.netty-3.8.0.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpMessageDecoder.decode(HttpMessageDecoder.java:189) ~[io.netty.netty-3.8.0.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpMessageDecoder.decode(HttpMessageDecoder.java:101) ~[io.netty.netty-3.8.0.Final.jar:na]
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:500) ~[io.netty.netty-3.8.0.Final.jar:na]
[error] p.nettyException - Exception caught in Netty
java.lang.IllegalArgumentException: invalid version format: ￋ;ￊ￮￥ﾩﾡￔAK2�/"ﾺﾺ￀+￀/￀,￀0ￌﾩￌﾨ￀￀ﾜﾝ/5

ShubhamGupta29 · 2020-09-30T11:47:30Z

@vempadh check if you are calling the Dr.Elephant's REST endpoint have HTTPs? If yes then try without HTTPs

vempadh · 2020-09-30T12:43:37Z

@ShubhamGupta29 : I have login Dr.elephant web UI through http only.
Dr.elephant is not working through https .

Can you please provide more information about Dr.Elephant REST and do you need to made any configuration changes in dr.elephant.

ShubhamGupta29 · 2020-09-30T13:02:39Z

Kindly add dr-elephant/logs/application.log and dr-elephant/dr.log here. Also mention the changes you made to Dr.Elephant, especially if you changed the Play version. I am still of opinion that Dr.Elephant is getting requests with HTTPS instead of HTTP only.

vempadh · 2020-09-30T14:55:45Z

@ShubhamGupta29 : Attached requested logs. i didn't made any changes in HTTPS and HTTP in dr.elephant.
dr.log
application.log

vempadh · 2020-09-30T14:58:23Z

ShubhamGupta29 · 2020-09-30T16:54:33Z

So your UI is loading, so you don't need to concern for the Netty logs for now. The main issue is that there are not jobs or applications shown in Dr.Elephant. Check the logs at the location ../dr-elephant/logs/elephant/dr_elephant.log. In case Dr.Elephant is processing applications then you would see logs of pattern: Analysis of application_12345565_4546456 took N.

vempadh · 2020-09-30T17:45:56Z

@ShubhamGupta29 : Thank you!
Yes, Dr.Elephant is processing applications. But we are facing this issue : Failed to analyze SPARK application_1599981649784_3819

I have attached the dr-elephant.lof file.
dr_elephant.log

ShubhamGupta29 · 2020-09-30T20:39:40Z

@vempadh quite evident that Dr.E could not get the Spark History Url from SPARK_HOME or SPARK_CONF_DIR (if properly set). Kindly check for spark.yarn.historyServer.address in the Spark configs. Also for the TEZ url mentioned in the logs, check why it does not exist.

vempadh · 2020-10-01T05:45:52Z

@ShubhamGupta29 : The mentioned configuration parameter configured to spark-defaults.conf file in below.

spark.yarn.historyServer.address azitshdpgtw01p.ecolab.com:18081

Is it correct?

vempadh · 2020-10-01T11:02:32Z

@ShubhamGupta29 : Yes, i have set properly in both Spark_Home & Spark_Conf_Dir.
[root@azitshdpmst01p ~]# echo $SPARK_CONF_DIR
/etc/spark2/conf
[root@azitshdpmst01p ~]# echo $SPARK_HOME
/usr/hdp/current/spark2-client

I have attached few xml files and fetcher xml file as well.

XML files.zip

vempadh · 2020-10-01T11:13:23Z

vempadh · 2020-10-01T11:19:49Z

@shahrukhkhan489 : We are using timeline service version :1.5

vempadh · 2020-10-01T11:32:54Z

@ShubhamGupta29 : In yarn, we configured the below parameters.

Can you please help me on this

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

./compile.sh: line 53: play: command not found #700

./compile.sh: line 53: play: command not found #700

vempadh commented Sep 25, 2020

vempadh commented Sep 25, 2020

nelhaj commented Sep 25, 2020

vempadh commented Sep 26, 2020

vempadh commented Sep 26, 2020

vempadh commented Sep 26, 2020

ShubhamGupta29 commented Sep 28, 2020 •

edited

Loading

vempadh commented Sep 29, 2020

vempadh commented Sep 29, 2020

vempadh commented Sep 29, 2020

ShubhamGupta29 commented Sep 29, 2020

ShubhamGupta29 commented Sep 29, 2020

vempadh commented Sep 30, 2020

ShubhamGupta29 commented Sep 30, 2020 •

edited

Loading

vempadh commented Sep 30, 2020

ShubhamGupta29 commented Sep 30, 2020

vempadh commented Sep 30, 2020

vempadh commented Sep 30, 2020

ShubhamGupta29 commented Sep 30, 2020

vempadh commented Sep 30, 2020

ShubhamGupta29 commented Sep 30, 2020

vempadh commented Oct 1, 2020

vempadh commented Oct 1, 2020

vempadh commented Oct 1, 2020

vempadh commented Oct 1, 2020

vempadh commented Oct 1, 2020

./compile.sh: line 53: play: command not found #700

./compile.sh: line 53: play: command not found #700

Comments

vempadh commented Sep 25, 2020

vempadh commented Sep 25, 2020

Run the main command alongwith the extra commands passed as arguments to compile.sh

nelhaj commented Sep 25, 2020

vempadh commented Sep 26, 2020

vempadh commented Sep 26, 2020

vempadh commented Sep 26, 2020

ShubhamGupta29 commented Sep 28, 2020 • edited Loading

vempadh commented Sep 29, 2020

vempadh commented Sep 29, 2020

vempadh commented Sep 29, 2020

ShubhamGupta29 commented Sep 29, 2020

ShubhamGupta29 commented Sep 29, 2020

vempadh commented Sep 30, 2020

ShubhamGupta29 commented Sep 30, 2020 • edited Loading

vempadh commented Sep 30, 2020

ShubhamGupta29 commented Sep 30, 2020

vempadh commented Sep 30, 2020

vempadh commented Sep 30, 2020

ShubhamGupta29 commented Sep 30, 2020

vempadh commented Sep 30, 2020

ShubhamGupta29 commented Sep 30, 2020

vempadh commented Oct 1, 2020

vempadh commented Oct 1, 2020

vempadh commented Oct 1, 2020

vempadh commented Oct 1, 2020

vempadh commented Oct 1, 2020

ShubhamGupta29 commented Sep 28, 2020 •

edited

Loading

ShubhamGupta29 commented Sep 30, 2020 •

edited

Loading