Difficulty getting sparkR working in Zeppelin

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Difficulty getting sparkR working in Zeppelin

Roger Sherman
Hello there!

I'm trying to get SparkR running on Zeppelin. I'm using Spark 2.0.2 (built with Scala 2.11), R 3.4.0, and Zeppelin 0.6.2, on a MapR cluster, and having very little success. I'm having a difficult time googling the errors, as it seems a lot of people get these errors when the SPARK_HOME isn't set, but in my case it is.

The error in question is as follows:

org.apache.commons.exec.ExecuteException: Execution failed (Exit value: -559038737. Caused by java.io.IOException: Cannot run program "R" (in directory "."): error=2, No such file or directory)
	at org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:205)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Cannot run program "R" (in directory "."): error=2, No such file or directory
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1047)
	at java.lang.Runtime.exec(Runtime.java:617)
	at org.apache.commons.exec.launcher.Java13CommandLauncher.exec(Java13CommandLauncher.java:61)
	at org.apache.commons.exec.DefaultExecutor.launch(DefaultExecutor.java:279)
	at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:336)
	at org.apache.commons.exec.DefaultExecutor.access$200(DefaultExecutor.java:48)
	at org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:200)
	... 1 more
Caused by: java.io.IOException: error=2, No such file or directory
	at java.lang.UNIXProcess.forkAndExec(Native Method)
	at java.lang.UNIXProcess.<init>(UNIXProcess.java:187)
	at java.lang.ProcessImpl.start(ProcessImpl.java:130)
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028)
	... 7 more


I have r-base and r-base-dev installed across the cluster, on bare metal (running Ubuntu 14.04), across the cluster, If I
fire up just a sparkR shell, it works fine, and I can see the distributed job working in YARN.

Here's the env output:

SPARK_EXECUTOR_TIMEOUT=300
HOSTNAME=zeppelin-test5285.platform-dev.company.com.au
SPARK_HOME=/apps/spark/spark-2.0.2-bin-mapr5.1.0_yarn_fat_j7_2.11
ZEPPELIN_MEM=-Xms1024m -Xmx1024m -XX:MaxPermSize=512m
TERM=unknown
HOST=node.company.com.au.local
ZEPPELIN_INTERPRETER_DIR=/zeppelin/interpreter
JAVA_INTP_OPTS= -Dfile.encoding=UTF-8 -Dlog4j.configuration=file:///projects/platform-dev/apps/platform-dev-zeppelin-test5285/conf/log4j.properties -Dzeppelin.log.file=/zeppelin/logs/zeppelin-interpreter-sh-rsherman-zeppelin-test5285.platform-dev.company.com.au.log
HADOOP_HOME=/opt/mapr/hadoop/hadoop-2.7.0
SPARK_DRIVER_MEMORY=6g
PORT0=31399
APP_NAME=platform-dev-zeppelin-test5285
MESOS_TASK_ID=platform-dev-zeppelin-test5285.ce9d43e1-466a-11e7-8ff4-0242ac1f3802
ZEPPELIN_INTP_MEM=-Xms1024m -Xmx1024m -XX:MaxPermSize=512m
SPARK_CONF_DIR=/projects/platform-dev/apps/platform-dev-zeppelin-test5285/conf/spark
SPARK_EXECUTOR_MEMORY=12g
JAVA_OPTS= -Dfile.encoding=UTF-8 -Xms1024m -Xmx1024m -XX:MaxPermSize=512m -Dlog4j.configuration=file:///projects/platform-dev/apps/platform-dev-zeppelin-test5285/conf/log4j.properties -Dzeppelin.log.file=/zeppelin/logs/zeppelin-rsherman-zeppelin-test5285.platform-company.com.au.log -Dfile.encoding=UTF-8 -Xms1024m -Xmx1024m -XX:MaxPermSize=512m -Dlog4j.configuration=file:///projects/platform-dev/apps/platform-dev-zeppelin-test5285/conf/log4j.properties
USER=rsherman
SUDO_USER=root
ZEPPELIN_NOTEBOOK_DIR=/projects/platform-dev/apps/platform-dev-zeppelin-test5285/data/notebook
SUDO_UID=0
INITRD=no
APP_USERNAME=rsherman
SCALABLE=false
MAPR_TICKETFILE_LOCATION=/home/rsherman/.maprticket
ZEPPELIN_HOME=/zeppelin
ZEPPELIN_RUNNER=java
ZEPPELIN_WAR=/zeppelin/zeppelin-web/dist
ZEPPELIN_PID_DIR=/zeppelin/run
USERNAME=rsherman
APP_REVISION=10

Here are the flags set at build time:

$make_distro --mvn ${MAVEN_HOME}/bin/mvn --name ${ARTIFACT_NAME} --tgz -Pyarn -Phadoop-${HADOOP_VERSION} -Phive -Psparkr -Phive-thriftserver -Pspark-ganglia-lgpl ${HADOOP_PROVIDED_OPTION} -Dhadoop.version=2.7.0-mapr-1602 -Dyarn.version=2.7.0-mapr-1602 -Dzookeeper.version=3.4.5-mapr-1503 -Dscala-${SCALA_VERSION} -DskipTests -e

And finally, here is the full stacktrace leading up to the failure:

17/06/01 02:22:06 INFO SchedulerFactory: Job paragraph_1496283119069_1363557346 started by scheduler org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session900607537 17/06/01 02:22:06 INFO Paragraph: run paragraph 20170601-021159_529838667 using spark.r org.apache.zeppelin.interpreter.LazyOpenInterpreter@52a4904f 17/06/01 02:22:06 INFO SchedulerFactory: Job remoteInterpretJob_1496283726813 started by scheduler org.apache.zeppelin.spark.SparkRInterpreter936853935 17/06/01 02:22:06 INFO ZeppelinR: File /tmp/zeppelin_sparkr-2889736965928624810.R created 17/06/01 02:22:06 ERROR ZeppelinR: Execution failed (Exit value: -559038737. Caused by java.io.IOException: Cannot run program "R" (in directory "."): error=2, No such file or directory) org.apache.commons.exec.ExecuteException: Execution failed (Exit value: -559038737. Caused by java.io.IOException: Cannot run program "R" (in directory "."): error=2, No such file or directory) at org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:205) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.IOException: Cannot run program "R" (in directory "."): error=2, No such file or directory at java.lang.ProcessBuilder.start(ProcessBuilder.java:1047) at java.lang.Runtime.exec(Runtime.java:617) at org.apache.commons.exec.launcher.Java13CommandLauncher.exec(Java13CommandLauncher.java:61) at org.apache.commons.exec.DefaultExecutor.launch(DefaultExecutor.java:279) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:336) at org.apache.commons.exec.DefaultExecutor.access$200(DefaultExecutor.java:48) at org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:200) ... 1 more Caused by: java.io.IOException: error=2, No such file or directory at java.lang.UNIXProcess.forkAndExec(Native Method) at java.lang.UNIXProcess.<init>(UNIXProcess.java:187) at java.lang.ProcessImpl.start(ProcessImpl.java:130) at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028) ... 7 more 17/06/01 02:22:07 ERROR Job: Job failed org.apache.zeppelin.interpreter.InterpreterException: sparkr is not responding at org.apache.zeppelin.spark.ZeppelinR.waitForRScriptInitialized(ZeppelinR.java:295) at org.apache.zeppelin.spark.ZeppelinR.request(ZeppelinR.java:235) at org.apache.zeppelin.spark.ZeppelinR.eval(ZeppelinR.java:183) at org.apache.zeppelin.spark.ZeppelinR.open(ZeppelinR.java:172) at org.apache.zeppelin.spark.SparkRInterpreter.open(SparkRInterpreter.java:85) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:93) at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:341) at org.apache.zeppelin.scheduler.Job.run(Job.java:176) at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 17/06/01 02:22:07 INFO SchedulerFactory: Job remoteInterpretJob_1496283726813 finished by scheduler org.apache.zeppelin.spark.SparkRInterpreter936853935 17/06/01 02:22:07 INFO ZeppelinR: File /tmp/zeppelin_sparkr-5775729319721689200.R created 17/06/01 02:22:07 INFO NotebookServer: Job 20170601-021159_529838667 is finished 17/06/01 02:22:07 ERROR ZeppelinR: Execution failed (Exit value: -559038737. Caused by java.io.IOException: Cannot run program "R" (in directory "."): error=2, No such file or directory) org.apache.commons.exec.ExecuteException: Execution failed (Exit value: -559038737. Caused by java.io.IOException: Cannot run program "R" (in directory "."): error=2, No such file or directory) at org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:205) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.IOException: Cannot run program "R" (in directory "."): error=2, No such file or directory at java.lang.ProcessBuilder.start(ProcessBuilder.java:1047) at java.lang.Runtime.exec(Runtime.java:617) at org.apache.commons.exec.launcher.Java13CommandLauncher.exec(Java13CommandLauncher.java:61) at org.apache.commons.exec.DefaultExecutor.launch(DefaultExecutor.java:279) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:336) at org.apache.commons.exec.DefaultExecutor.access$200(DefaultExecutor.java:48) at org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:200) ... 1 more Caused by: java.io.IOException: error=2, No such file or directory at java.lang.UNIXProcess.forkAndExec(Native Method) at java.lang.UNIXProcess.<init>(UNIXProcess.java:187) at java.lang.ProcessImpl.start(ProcessImpl.java:130) at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028) ... 7 more 17/06/01 02:22:07 INFO SchedulerFactory: Job paragraph_1496283119069_1363557346 finished by scheduler org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session900607537 17/06/01 02:22:08 ERROR TThreadPoolServer: Error occurred during processing of message. org.apache.zeppelin.interpreter.InterpreterException: sparkr is not responding at org.apache.zeppelin.spark.ZeppelinR.waitForRScriptInitialized(ZeppelinR.java:295) at org.apache.zeppelin.spark.ZeppelinR.request(ZeppelinR.java:235) at org.apache.zeppelin.spark.ZeppelinR.eval(ZeppelinR.java:183) at org.apache.zeppelin.spark.ZeppelinR.open(ZeppelinR.java:172) at org.apache.zeppelin.spark.SparkRInterpreter.open(SparkRInterpreter.java:85) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getProgress(LazyOpenInterpreter.java:110) at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer.getProgress(RemoteInterpreterServer.java:404) at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Processor$getProgress.getResult(RemoteInterpreterService.java:1509) at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Processor$getProgress.getResult(RemoteInterpreterService.java:1494) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 17/06/01 02:22:08 ERROR JobProgressPoller: Can not get or update progress org.apache.zeppelin.interpreter.InterpreterException: org.apache.thrift.transport.TTransportException at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getProgress(RemoteInterpreter.java:373) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getProgress(LazyOpenInterpreter.java:111) at org.apache.zeppelin.notebook.Paragraph.progress(Paragraph.java:237) at org.apache.zeppelin.scheduler.JobProgressPoller.run(JobProgressPoller.java:51) Caused by: org.apache.thrift.transport.TTransportException at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132) at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86) at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429) at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318) at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69) at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.recv_getProgress(RemoteInterpreterService.java:296) at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.getProgress(RemoteInterpreterService.java:281) at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getProgress(RemoteInterpreter.java:370) ... 3 more


It's probably obvious that I don't have much experience with Zeppelin, so if I've left out anything obvious, just let me know and I'll be happy to add it.
anyone could point me in the right direction, it would be greatly appreciated.


Best Regards,

Roger Sherman
Reply | Threaded
Open this post in threaded view
|

Re: Difficulty getting sparkR working in Zeppelin

Jeff Zhang

It looks likes due to R could not be found. Try to set zeppelin.R.cmd to where R you installed in interpreter setting.



Roger Sherman <[hidden email]>于2017年6月2日周五 上午7:03写道:
Hello there!

I'm trying to get SparkR running on Zeppelin. I'm using Spark 2.0.2 (built with Scala 2.11), R 3.4.0, and Zeppelin 0.6.2, on a MapR cluster, and having very little success. I'm having a difficult time googling the errors, as it seems a lot of people get these errors when the SPARK_HOME isn't set, but in my case it is.

The error in question is as follows:

org.apache.commons.exec.ExecuteException: Execution failed (Exit value: -559038737. Caused by java.io.IOException: Cannot run program "R" (in directory "."): error=2, No such file or directory)
	at org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:205)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Cannot run program "R" (in directory "."): error=2, No such file or directory
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1047)
	at java.lang.Runtime.exec(Runtime.java:617)
	at org.apache.commons.exec.launcher.Java13CommandLauncher.exec(Java13CommandLauncher.java:61)
	at org.apache.commons.exec.DefaultExecutor.launch(DefaultExecutor.java:279)
	at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:336)
	at org.apache.commons.exec.DefaultExecutor.access$200(DefaultExecutor.java:48)
	at org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:200)
	... 1 more
Caused by: java.io.IOException: error=2, No such file or directory
	at java.lang.UNIXProcess.forkAndExec(Native Method)
	at java.lang.UNIXProcess.<init>(UNIXProcess.java:187)
	at java.lang.ProcessImpl.start(ProcessImpl.java:130)
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028)
	... 7 more


I have r-base and r-base-dev installed across the cluster, on bare metal (running Ubuntu 14.04), across the cluster, If I
fire up just a sparkR shell, it works fine, and I can see the distributed job working in YARN.

Here's the env output:

SPARK_EXECUTOR_TIMEOUT=300
HOSTNAME=zeppelin-test5285.platform-dev.company.com.au
SPARK_HOME=/apps/spark/spark-2.0.2-bin-mapr5.1.0_yarn_fat_j7_2.11
ZEPPELIN_MEM=-Xms1024m -Xmx1024m -XX:MaxPermSize=512m
TERM=unknown
HOST=node.company.com.au.local
ZEPPELIN_INTERPRETER_DIR=/zeppelin/interpreter
JAVA_INTP_OPTS= -Dfile.encoding=UTF-8 -Dlog4j.configuration=file:///projects/platform-dev/apps/platform-dev-zeppelin-test5285/conf/log4j.properties -Dzeppelin.log.file=/zeppelin/logs/zeppelin-interpreter-sh-rsherman-zeppelin-test5285.platform-dev.company.com.au.log
HADOOP_HOME=/opt/mapr/hadoop/hadoop-2.7.0
SPARK_DRIVER_MEMORY=6g
PORT0=31399
APP_NAME=platform-dev-zeppelin-test5285
MESOS_TASK_ID=platform-dev-zeppelin-test5285.ce9d43e1-466a-11e7-8ff4-0242ac1f3802
ZEPPELIN_INTP_MEM=-Xms1024m -Xmx1024m -XX:MaxPermSize=512m
SPARK_CONF_DIR=/projects/platform-dev/apps/platform-dev-zeppelin-test5285/conf/spark
SPARK_EXECUTOR_MEMORY=12g
JAVA_OPTS= -Dfile.encoding=UTF-8 -Xms1024m -Xmx1024m -XX:MaxPermSize=512m -Dlog4j.configuration=file:///projects/platform-dev/apps/platform-dev-zeppelin-test5285/conf/log4j.properties -Dzeppelin.log.file=/zeppelin/logs/zeppelin-rsherman-zeppelin-test5285.platform-company.com.au.log -Dfile.encoding=UTF-8 -Xms1024m -Xmx1024m -XX:MaxPermSize=512m -Dlog4j.configuration=file:///projects/platform-dev/apps/platform-dev-zeppelin-test5285/conf/log4j.properties
USER=rsherman
SUDO_USER=root
ZEPPELIN_NOTEBOOK_DIR=/projects/platform-dev/apps/platform-dev-zeppelin-test5285/data/notebook
SUDO_UID=0
INITRD=no
APP_USERNAME=rsherman
SCALABLE=false
MAPR_TICKETFILE_LOCATION=/home/rsherman/.maprticket
ZEPPELIN_HOME=/zeppelin
ZEPPELIN_RUNNER=java
ZEPPELIN_WAR=/zeppelin/zeppelin-web/dist
ZEPPELIN_PID_DIR=/zeppelin/run
USERNAME=rsherman
APP_REVISION=10

Here are the flags set at build time:

$make_distro --mvn ${MAVEN_HOME}/bin/mvn --name ${ARTIFACT_NAME} --tgz -Pyarn -Phadoop-${HADOOP_VERSION} -Phive -Psparkr -Phive-thriftserver -Pspark-ganglia-lgpl ${HADOOP_PROVIDED_OPTION} -Dhadoop.version=2.7.0-mapr-1602 -Dyarn.version=2.7.0-mapr-1602 -Dzookeeper.version=3.4.5-mapr-1503 -Dscala-${SCALA_VERSION} -DskipTests -e

And finally, here is the full stacktrace leading up to the failure:

17/06/01 02:22:06 INFO SchedulerFactory: Job paragraph_1496283119069_1363557346 started by scheduler org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session900607537 17/06/01 02:22:06 INFO Paragraph: run paragraph 20170601-021159_529838667 using spark.r org.apache.zeppelin.interpreter.LazyOpenInterpreter@52a4904f 17/06/01 02:22:06 INFO SchedulerFactory: Job remoteInterpretJob_1496283726813 started by scheduler org.apache.zeppelin.spark.SparkRInterpreter936853935 17/06/01 02:22:06 INFO ZeppelinR: File /tmp/zeppelin_sparkr-2889736965928624810.R created 17/06/01 02:22:06 ERROR ZeppelinR: Execution failed (Exit value: -559038737. Caused by java.io.IOException: Cannot run program "R" (in directory "."): error=2, No such file or directory) org.apache.commons.exec.ExecuteException: Execution failed (Exit value: -559038737. Caused by java.io.IOException: Cannot run program "R" (in directory "."): error=2, No such file or directory) at org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:205) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.IOException: Cannot run program "R" (in directory "."): error=2, No such file or directory at java.lang.ProcessBuilder.start(ProcessBuilder.java:1047) at java.lang.Runtime.exec(Runtime.java:617) at org.apache.commons.exec.launcher.Java13CommandLauncher.exec(Java13CommandLauncher.java:61) at org.apache.commons.exec.DefaultExecutor.launch(DefaultExecutor.java:279) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:336) at org.apache.commons.exec.DefaultExecutor.access$200(DefaultExecutor.java:48) at org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:200) ... 1 more Caused by: java.io.IOException: error=2, No such file or directory at java.lang.UNIXProcess.forkAndExec(Native Method) at java.lang.UNIXProcess.<init>(UNIXProcess.java:187) at java.lang.ProcessImpl.start(ProcessImpl.java:130) at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028) ... 7 more 17/06/01 02:22:07 ERROR Job: Job failed org.apache.zeppelin.interpreter.InterpreterException: sparkr is not responding at org.apache.zeppelin.spark.ZeppelinR.waitForRScriptInitialized(ZeppelinR.java:295) at org.apache.zeppelin.spark.ZeppelinR.request(ZeppelinR.java:235) at org.apache.zeppelin.spark.ZeppelinR.eval(ZeppelinR.java:183) at org.apache.zeppelin.spark.ZeppelinR.open(ZeppelinR.java:172) at org.apache.zeppelin.spark.SparkRInterpreter.open(SparkRInterpreter.java:85) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:93) at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:341) at org.apache.zeppelin.scheduler.Job.run(Job.java:176) at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 17/06/01 02:22:07 INFO SchedulerFactory: Job remoteInterpretJob_1496283726813 finished by scheduler org.apache.zeppelin.spark.SparkRInterpreter936853935 17/06/01 02:22:07 INFO ZeppelinR: File /tmp/zeppelin_sparkr-5775729319721689200.R created 17/06/01 02:22:07 INFO NotebookServer: Job 20170601-021159_529838667 is finished 17/06/01 02:22:07 ERROR ZeppelinR: Execution failed (Exit value: -559038737. Caused by java.io.IOException: Cannot run program "R" (in directory "."): error=2, No such file or directory) org.apache.commons.exec.ExecuteException: Execution failed (Exit value: -559038737. Caused by java.io.IOException: Cannot run program "R" (in directory "."): error=2, No such file or directory) at org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:205) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.IOException: Cannot run program "R" (in directory "."): error=2, No such file or directory at java.lang.ProcessBuilder.start(ProcessBuilder.java:1047) at java.lang.Runtime.exec(Runtime.java:617) at org.apache.commons.exec.launcher.Java13CommandLauncher.exec(Java13CommandLauncher.java:61) at org.apache.commons.exec.DefaultExecutor.launch(DefaultExecutor.java:279) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:336) at org.apache.commons.exec.DefaultExecutor.access$200(DefaultExecutor.java:48) at org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:200) ... 1 more Caused by: java.io.IOException: error=2, No such file or directory at java.lang.UNIXProcess.forkAndExec(Native Method) at java.lang.UNIXProcess.<init>(UNIXProcess.java:187) at java.lang.ProcessImpl.start(ProcessImpl.java:130) at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028) ... 7 more 17/06/01 02:22:07 INFO SchedulerFactory: Job paragraph_1496283119069_1363557346 finished by scheduler org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session900607537 17/06/01 02:22:08 ERROR TThreadPoolServer: Error occurred during processing of message. org.apache.zeppelin.interpreter.InterpreterException: sparkr is not responding at org.apache.zeppelin.spark.ZeppelinR.waitForRScriptInitialized(ZeppelinR.java:295) at org.apache.zeppelin.spark.ZeppelinR.request(ZeppelinR.java:235) at org.apache.zeppelin.spark.ZeppelinR.eval(ZeppelinR.java:183) at org.apache.zeppelin.spark.ZeppelinR.open(ZeppelinR.java:172) at org.apache.zeppelin.spark.SparkRInterpreter.open(SparkRInterpreter.java:85) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getProgress(LazyOpenInterpreter.java:110) at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer.getProgress(RemoteInterpreterServer.java:404) at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Processor$getProgress.getResult(RemoteInterpreterService.java:1509) at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Processor$getProgress.getResult(RemoteInterpreterService.java:1494) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 17/06/01 02:22:08 ERROR JobProgressPoller: Can not get or update progress org.apache.zeppelin.interpreter.InterpreterException: org.apache.thrift.transport.TTransportException at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getProgress(RemoteInterpreter.java:373) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getProgress(LazyOpenInterpreter.java:111) at org.apache.zeppelin.notebook.Paragraph.progress(Paragraph.java:237) at org.apache.zeppelin.scheduler.JobProgressPoller.run(JobProgressPoller.java:51) Caused by: org.apache.thrift.transport.TTransportException at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132) at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86) at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429) at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318) at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69) at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.recv_getProgress(RemoteInterpreterService.java:296) at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.getProgress(RemoteInterpreterService.java:281) at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getProgress(RemoteInterpreter.java:370) ... 3 more


It's probably obvious that I don't have much experience with Zeppelin, so if I've left out anything obvious, just let me know and I'll be happy to add it.
anyone could point me in the right direction, it would be greatly appreciated.


Best Regards,

Roger Sherman