Hive interpreter Error as soon as Hive query uses MapRed

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Hive interpreter Error as soon as Hive query uses MapRed

Meier, Alexander
Hi list

I’m trying to get a Hive interpreter correctly running on a CDH 5.7 Cluster with Spark 1.6. Simple queries are running fine, but as soon as a query needs a MapRed tasks in order to complete, the query fails with:

java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
        at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:279)
        at org.apache.commons.dbcp2.DelegatingStatement.execute(DelegatingStatement.java:291)
        at org.apache.commons.dbcp2.DelegatingStatement.execute(DelegatingStatement.java:291)
        at org.apache.zeppelin.jdbc.JDBCInterpreter.executeSql(JDBCInterpreter.java:580)
        at org.apache.zeppelin.jdbc.JDBCInterpreter.interpret(JDBCInterpreter.java:692)
        at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:95)
        at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:490)
        at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
        at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
etc…

I’ve got the interpreter set up as follows:

Properties
name value
default.driver org.apache.hive.jdbc.HiveDriver
default.url jdbc:hive2://[hostname]:10000
hive.driver org.apache.hive.jdbc.HiveDriver
hive.url jdbc:hive2://[hostname]:10000
zeppelin.interpreter.localRepo /opt/zeppelin/local-repo/2CJ4XM2Z4

Dependencies
artifact
/opt/cloudera/parcels/CDH/lib/hive/lib/hive-jdbc.jar
/opt/cloudera/parcels/CDH/lib/hive/lib/hive-service.jar
/opt/cloudera/parcels/CDH/lib/hadoop/client/hadoop-common.jar
/opt/cloudera/parcels/CDH/lib/hive/lib/hive-common.jar
/opt/cloudera/parcels/CDH/lib/hive/lib/hive-metastore.jar


Unfortunately I haven’t found any help googling around… anyone here with some helpful input?

Best regards and many thanks in advance,
Alex
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Hive interpreter Error as soon as Hive query uses MapRed

Jongyoul Lee
Can you check your script works in native hive environment?

On Fri, May 19, 2017 at 10:20 AM, Meier, Alexander <[hidden email]> wrote:
Hi list

I’m trying to get a Hive interpreter correctly running on a CDH 5.7 Cluster with Spark 1.6. Simple queries are running fine, but as soon as a query needs a MapRed tasks in order to complete, the query fails with:

java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
        at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:279)
        at org.apache.commons.dbcp2.DelegatingStatement.execute(DelegatingStatement.java:291)
        at org.apache.commons.dbcp2.DelegatingStatement.execute(DelegatingStatement.java:291)
        at org.apache.zeppelin.jdbc.JDBCInterpreter.executeSql(JDBCInterpreter.java:580)
        at org.apache.zeppelin.jdbc.JDBCInterpreter.interpret(JDBCInterpreter.java:692)
        at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:95)
        at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:490)
        at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
        at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
etc…

I’ve got the interpreter set up as follows:

Properties
name                                            value
default.driver                                  org.apache.hive.jdbc.HiveDriver
default.url                                     jdbc:hive2://[hostname]:10000
hive.driver                                     org.apache.hive.jdbc.HiveDriver
hive.url                                                jdbc:hive2://[hostname]:10000
zeppelin.interpreter.localRepo  /opt/zeppelin/local-repo/2CJ4XM2Z4

Dependencies
artifact
/opt/cloudera/parcels/CDH/lib/hive/lib/hive-jdbc.jar
/opt/cloudera/parcels/CDH/lib/hive/lib/hive-service.jar
/opt/cloudera/parcels/CDH/lib/hadoop/client/hadoop-common.jar
/opt/cloudera/parcels/CDH/lib/hive/lib/hive-common.jar
/opt/cloudera/parcels/CDH/lib/hive/lib/hive-metastore.jar


Unfortunately I haven’t found any help googling around… anyone here with some helpful input?

Best regards and many thanks in advance,
Alex



--
이종열, Jongyoul Lee, 李宗烈
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Hive interpreter Error as soon as Hive query uses MapRed

Meier, Alexander
Yes, the script (i.e. The select statement) runs fine in hive cli, hue and also in spark sql ( spark sql also in zeppelin). 
Just not when using the hive interpreter in zeppelin. 



Sent from my iPhone

Am 19.05.2017 um 19:35 schrieb Jongyoul Lee <[hidden email]>:

Can you check your script works in native hive environment?

On Fri, May 19, 2017 at 10:20 AM, Meier, Alexander <[hidden email]> wrote:
Hi list

I’m trying to get a Hive interpreter correctly running on a CDH 5.7 Cluster with Spark 1.6. Simple queries are running fine, but as soon as a query needs a MapRed tasks in order to complete, the query fails with:

java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
        at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:279)
        at org.apache.commons.dbcp2.DelegatingStatement.execute(DelegatingStatement.java:291)
        at org.apache.commons.dbcp2.DelegatingStatement.execute(DelegatingStatement.java:291)
        at org.apache.zeppelin.jdbc.JDBCInterpreter.executeSql(JDBCInterpreter.java:580)
        at org.apache.zeppelin.jdbc.JDBCInterpreter.interpret(JDBCInterpreter.java:692)
        at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:95)
        at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:490)
        at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
        at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
etc…

I’ve got the interpreter set up as follows:

Properties
name                                            value
default.driver                                  org.apache.hive.jdbc.HiveDriver
default.url                                     jdbc:hive2://[hostname]:10000
hive.driver                                     org.apache.hive.jdbc.HiveDriver
hive.url                                                jdbc:hive2://[hostname]:10000
zeppelin.interpreter.localRepo  /opt/zeppelin/local-repo/2CJ4XM2Z4

Dependencies
artifact
/opt/cloudera/parcels/CDH/lib/hive/lib/hive-jdbc.jar
/opt/cloudera/parcels/CDH/lib/hive/lib/hive-service.jar
/opt/cloudera/parcels/CDH/lib/hadoop/client/hadoop-common.jar
/opt/cloudera/parcels/CDH/lib/hive/lib/hive-common.jar
/opt/cloudera/parcels/CDH/lib/hive/lib/hive-metastore.jar


Unfortunately I haven’t found any help googling around… anyone here with some helpful input?

Best regards and many thanks in advance,
Alex



--
이종열, Jongyoul Lee, 李宗烈
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Hive interpreter Error as soon as Hive query uses MapRed

Ben Vogan
I am running CDH 5.7 and Spark 1.6 as well and hive is working for me with the following configuration:

Properties
namevalue
common.max_count1000
default.driverorg.apache.hive.jdbc.HiveDriver
default.password
default.urljdbc:hive2://hdfs004:10000
default.userhive
zeppelin.interpreter.localRepo/services/zeppelin/zeppelin-0.7.1/local-repo/2CECB8FBV
zeppelin.jdbc.auth.type
zeppelin.jdbc.concurrent.max_connection10
zeppelin.jdbc.concurrent.usetrue
zeppelin.jdbc.keytab.location
zeppelin.jdbc.principal
Dependencies
artifactexclude
org.apache.hive:hive-jdbc:0.14.0
org.apache.hadoop:hadoop-common:2.6.0

I admit to not having spent time figuring out whether there are any edge cases that are broken because I am using the open source version of the odbc driver vs using the cloudera jars.  However, it definitely returns results from complex select queries and has no issues with DDL statements that I've tried.

Good luck!
--Ben

On Fri, May 19, 2017 at 12:10 PM, Meier, Alexander <[hidden email]> wrote:
Yes, the script (i.e. The select statement) runs fine in hive cli, hue and also in spark sql ( spark sql also in zeppelin). 
Just not when using the hive interpreter in zeppelin. 



Sent from my iPhone

Am 19.05.2017 um 19:35 schrieb Jongyoul Lee <[hidden email]>:

Can you check your script works in native hive environment?

On Fri, May 19, 2017 at 10:20 AM, Meier, Alexander <[hidden email]> wrote:
Hi list

I’m trying to get a Hive interpreter correctly running on a CDH 5.7 Cluster with Spark 1.6. Simple queries are running fine, but as soon as a query needs a MapRed tasks in order to complete, the query fails with:

java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
        at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:279)
        at org.apache.commons.dbcp2.DelegatingStatement.execute(DelegatingStatement.java:291)
        at org.apache.commons.dbcp2.DelegatingStatement.execute(DelegatingStatement.java:291)
        at org.apache.zeppelin.jdbc.JDBCInterpreter.executeSql(JDBCInterpreter.java:580)
        at org.apache.zeppelin.jdbc.JDBCInterpreter.interpret(JDBCInterpreter.java:692)
        at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:95)
        at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:490)
        at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
        at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
etc…

I’ve got the interpreter set up as follows:

Properties
name                                            value
default.driver                                  org.apache.hive.jdbc.HiveDriver
default.url                                     jdbc:hive2://[hostname]:10000
hive.driver                                     org.apache.hive.jdbc.HiveDriver
hive.url                                                jdbc:hive2://[hostname]:10000
zeppelin.interpreter.localRepo  /opt/zeppelin/local-repo/2CJ4XM2Z4

Dependencies
artifact
/opt/cloudera/parcels/CDH/lib/hive/lib/hive-jdbc.jar
/opt/cloudera/parcels/CDH/lib/hive/lib/hive-service.jar
/opt/cloudera/parcels/CDH/lib/hadoop/client/hadoop-common.jar
/opt/cloudera/parcels/CDH/lib/hive/lib/hive-common.jar
/opt/cloudera/parcels/CDH/lib/hive/lib/hive-metastore.jar


Unfortunately I haven’t found any help googling around… anyone here with some helpful input?

Best regards and many thanks in advance,
Alex



--
이종열, Jongyoul Lee, 李宗烈



--
BENJAMIN VOGAN | Data Platform Team Lead

Loading...