Cannot fetch dependencies when using Zeppelin on EMR

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Cannot fetch dependencies when using Zeppelin on EMR

Johnathan Kool
to users
Hello, I've encountered a repeatable error when using Zeppelin on EMR.  I have tested using emr-0.5.6 and 0.5.8 (Zeppelin 0.7.1 and 0.7.2 respectively).

To generate the error, boot an EMR cluster, install the jdbc interpreter (sudo /usr/lib/zeppelin/bin/install-interpreter.sh --name jdbc), and then attempt to add a dependency (e.g.
org.apache.hive:hive-jdbc:0.14.0).  An error then pops up:
Error setting properties for interpreter 'jdbc.hive': Cannot fetch dependencies for org.apache.hive:hive-jdbc:0.14.0.

Looking in the logs doesn't revel much, it just says:  

ERROR [2017-08-18 12:34:55,738] ({Thread-19} InterpreterSettingManager.java[run]:582) - Error while downloading repos for interpreter group : jdbc, go to interpreter setting page click on edit and save it again to make this interpreter work properly. : Cannot fetch dependencies for org.apache.hive:hive-jdbc:0.14.0 org.sonatype.aether.RepositoryException: Cannot fetch dependencies for org.apache.hive:hive-jdbc:0.14.0 at org.apache.zeppelin.dep.DependencyResolver.getArtifactsWithDep(DependencyResolver.java:181) at org.apache.zeppelin.dep.DependencyResolver.loadFromMvn(DependencyResolver.java:131) at org.apache.zeppelin.dep.DependencyResolver.load(DependencyResolver.java:79) at org.apache.zeppelin.dep.DependencyResolver.load(DependencyResolver.java:96) at org.apache.zeppelin.dep.DependencyResolver.load(DependencyResolver.java:88) at org.apache.zeppelin.interpreter.InterpreterSettingManager$3.run(InterpreterSettingManager.java:574)

After some testing, it looks like the error is introduced during the jdbc installation step.  Dependencies can be added before this step without a problem.  There may be something going on with the installation script.  JAVA_HOME is set by default as /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.141-1.b16.32.amzn1.x86_64, but I have also tried testing using Java 1.7.

Any advice would be appreciated,
Reply | Threaded
Open this post in threaded view
|

Re: Cannot fetch dependencies when using Zeppelin on EMR

smartsi
Reply | Threaded
Open this post in threaded view
|

Re: Cannot fetch dependencies when using Zeppelin on EMR

Jeff Zhang
Please be more specific, e.g. what kind of error do you see ?

smartsi <[hidden email]> 于2020年10月18日周日 下午2:29写道:
how solve the error?



--
Sent from: http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/


--
Best Regards

Jeff Zhang
Reply | Threaded
Open this post in threaded view
|

Re: Cannot fetch dependencies when using Zeppelin on EMR

Austin Heyne

I have trouble with this as well but have never tried to fix it, as a work around, you can place the jar on the master and reference the local path rather than the GAV.

-Austin

On 10/18/20 6:09 AM, Jeff Zhang wrote:
Please be more specific, e.g. what kind of error do you see ?

smartsi <[hidden email]> 于2020年10月18日周日 下午2:29写道:
how solve the error?



--
Sent from: http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/


--
Best Regards

Jeff Zhang