Zeppelin postgres interpreter throws NPE

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Zeppelin postgres interpreter throws NPE

Richard Xin
on AWS EMR
1. Tested remote postgres connection using psql, works OK
2. added postges interpreter via Interpreter UI
3. create new note:
%postgresql   (or %psql)
select * from test.batch_report; 
got NPE

4. here are what's in the log:

WARN [2017-08-03 20:22:46,859] ({qtp459296537-18} LoginRestApi.java[postLogin]:119) - {"status":"OK","message":"","body":{"principal":"richard.xin","ticket":"d86ab3d9-3976-43a4-a274-df50ee3b627a","roles":"[]"}}
INFO [2017-08-03 20:22:50,370] ({qtp459296537-21} NotebookServer.java[sendNote]:705) - New operation from 172.17.197.49 : 57990 : richard.xin : GET_NOTE : 2CNXQE5CR
INFO [2017-08-03 20:22:50,656] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:188) - Create interpreter instance spark for note 2CNXQE5CR
INFO [2017-08-03 20:22:50,661] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.spark.SparkInterpreter 2080711349 created
INFO [2017-08-03 20:22:50,662] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.spark.PySparkInterpreter 506689376 created
INFO [2017-08-03 20:22:50,662] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.spark.SparkSqlInterpreter 1462545045 created
INFO [2017-08-03 20:22:50,776] ({qtp459296537-22} InterpreterFactory.java[createInterpretersForNote]:188) - Create interpreter instance postgresql for note 2CNXQE5CR
INFO [2017-08-03 20:22:50,776] ({qtp459296537-22} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.postgresql.PostgreSqlInterpreter 340437485 created
INFO [2017-08-03 20:22:53,610] ({pool-2-thread-2} SchedulerFactory.java[jobStarted]:131) - Job paragraph_1501783535283_1713771734 started by scheduler org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session392982422
INFO [2017-08-03 20:22:53,611] ({pool-2-thread-2} Paragraph.java[jobRun]:362) - run paragraph 20170803-180535_552293631 using psql org.apache.zeppelin.interpreter.LazyOpenInterpreter@144aa9ed
INFO [2017-08-03 20:22:53,620] ({pool-2-thread-2} RemoteInterpreterManagedProcess.java[start]:126) - Run interpreter process [/usr/lib/zeppelin/bin/interpreter.sh, -d, /usr/lib/zeppelin/interpreter/postgresql, -p, 42610, -l, /usr/lib/zeppelin/local-repo/2CPQEJPGC]
INFO [2017-08-03 20:22:54,190] ({pool-2-thread-2} RemoteInterpreter.java[init]:221) - Create remote interpreter org.apache.zeppelin.postgresql.PostgreSqlInterpreter
INFO [2017-08-03 20:22:54,370] ({pool-2-thread-2} RemoteInterpreter.java[pushAngularObjectRegistryToRemote]:551) - Push local angular object registry from ZeppelinServer to remote interpreter group 2CPQEJPGC:shared_process
WARN [2017-08-03 20:22:54,606] ({pool-2-thread-2} NotebookServer.java[afterStatusChange]:2058) - Job 20170803-180535_552293631 is finished, status: ERROR, exception: null, result: %text java.lang.NullPointerException
    at org.apache.zeppelin.postgresql.PostgreSqlInterpreter.executeSql(PostgreSqlInterpreter.java:202)
    at org.apache.zeppelin.postgresql.PostgreSqlInterpreter.interpret(PostgreSqlInterpreter.java:289)
    at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:97)
    at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:498)
    at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
    at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

INFO [2017-08-03 20:22:54,673] ({pool-2-thread-2} SchedulerFactory.java[jobFinished]:137) - Job paragraph_1501783535283_1713771734 finished by scheduler org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session392982422



Reply | Threaded
Open this post in threaded view
|

Re: Zeppelin postgres interpreter throws NPE

Richard Xin
oh, just figured it out myself
it seems that the property I aded in interpreter config postgres.database is not recognized by zeppelin, I solved it by adding dbname to url.

anyway, the exception is better to be handled and emit more friendly error message.


On Thursday, August 3, 2017, 1:35:04 PM PDT, Richard Xin <[hidden email]> wrote:


on AWS EMR
1. Tested remote postgres connection using psql, works OK
2. added postges interpreter via Interpreter UI
3. create new note:
%postgresql   (or %psql)
select * from test.batch_report; 
got NPE

4. here are what's in the log:

WARN [2017-08-03 20:22:46,859] ({qtp459296537-18} LoginRestApi.java[postLogin]:119) - {"status":"OK","message":"","body":{"principal":"richard.xin","ticket":"d86ab3d9-3976-43a4-a274-df50ee3b627a","roles":"[]"}}
INFO [2017-08-03 20:22:50,370] ({qtp459296537-21} NotebookServer.java[sendNote]:705) - New operation from 172.17.197.49 : 57990 : richard.xin : GET_NOTE : 2CNXQE5CR
INFO [2017-08-03 20:22:50,656] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:188) - Create interpreter instance spark for note 2CNXQE5CR
INFO [2017-08-03 20:22:50,661] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.spark.SparkInterpreter 2080711349 created
INFO [2017-08-03 20:22:50,662] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.spark.PySparkInterpreter 506689376 created
INFO [2017-08-03 20:22:50,662] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.spark.SparkSqlInterpreter 1462545045 created
INFO [2017-08-03 20:22:50,776] ({qtp459296537-22} InterpreterFactory.java[createInterpretersForNote]:188) - Create interpreter instance postgresql for note 2CNXQE5CR
INFO [2017-08-03 20:22:50,776] ({qtp459296537-22} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.postgresql.PostgreSqlInterpreter 340437485 created
INFO [2017-08-03 20:22:53,610] ({pool-2-thread-2} SchedulerFactory.java[jobStarted]:131) - Job paragraph_1501783535283_1713771734 started by scheduler org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session392982422
INFO [2017-08-03 20:22:53,611] ({pool-2-thread-2} Paragraph.java[jobRun]:362) - run paragraph 20170803-180535_552293631 using psql org.apache.zeppelin.interpreter.LazyOpenInterpreter@144aa9ed
INFO [2017-08-03 20:22:53,620] ({pool-2-thread-2} RemoteInterpreterManagedProcess.java[start]:126) - Run interpreter process [/usr/lib/zeppelin/bin/interpreter.sh, -d, /usr/lib/zeppelin/interpreter/postgresql, -p, 42610, -l, /usr/lib/zeppelin/local-repo/2CPQEJPGC]
INFO [2017-08-03 20:22:54,190] ({pool-2-thread-2} RemoteInterpreter.java[init]:221) - Create remote interpreter org.apache.zeppelin.postgresql.PostgreSqlInterpreter
INFO [2017-08-03 20:22:54,370] ({pool-2-thread-2} RemoteInterpreter.java[pushAngularObjectRegistryToRemote]:551) - Push local angular object registry from ZeppelinServer to remote interpreter group 2CPQEJPGC:shared_process
WARN [2017-08-03 20:22:54,606] ({pool-2-thread-2} NotebookServer.java[afterStatusChange]:2058) - Job 20170803-180535_552293631 is finished, status: ERROR, exception: null, result: %text java.lang.NullPointerException
    at org.apache.zeppelin.postgresql.PostgreSqlInterpreter.executeSql(PostgreSqlInterpreter.java:202)
    at org.apache.zeppelin.postgresql.PostgreSqlInterpreter.interpret(PostgreSqlInterpreter.java:289)
    at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:97)
    at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:498)
    at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
    at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

INFO [2017-08-03 20:22:54,673] ({pool-2-thread-2} SchedulerFactory.java[jobFinished]:137) - Job paragraph_1501783535283_1713771734 finished by scheduler org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session392982422



Reply | Threaded
Open this post in threaded view
|

Re: Zeppelin postgres interpreter throws NPE

Jianfeng (Jeff) Zhang
Agree, you can create a ticket for better exception message. And the error is better to be displayed in frontend




Best Regard,
Jeff Zhang


From: Richard Xin <[hidden email]>
Reply-To: "[hidden email]" <[hidden email]>
Date: Friday, August 4, 2017 at 4:40 AM
To: "[hidden email]" <[hidden email]>
Subject: Re: Zeppelin postgres interpreter throws NPE

oh, just figured it out myself
it seems that the property I aded in interpreter config postgres.database is not recognized by zeppelin, I solved it by adding dbname to url.

anyway, the exception is better to be handled and emit more friendly error message.


On Thursday, August 3, 2017, 1:35:04 PM PDT, Richard Xin <[hidden email]> wrote:


on AWS EMR
1. Tested remote postgres connection using psql, works OK
2. added postges interpreter via Interpreter UI
3. create new note:
%postgresql   (or %psql)
select * from test.batch_report; 
got NPE

4. here are what's in the log:

WARN [2017-08-03 20:22:46,859] ({qtp459296537-18} LoginRestApi.java[postLogin]:119) - {"status":"OK","message":"","body":{"principal":"richard.xin","ticket":"d86ab3d9-3976-43a4-a274-df50ee3b627a","roles":"[]"}}
INFO [2017-08-03 20:22:50,370] ({qtp459296537-21} NotebookServer.java[sendNote]:705) - New operation from 172.17.197.49 : 57990 : richard.xin : GET_NOTE : 2CNXQE5CR
INFO [2017-08-03 20:22:50,656] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:188) - Create interpreter instance spark for note 2CNXQE5CR
INFO [2017-08-03 20:22:50,661] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.spark.SparkInterpreter 2080711349 created
INFO [2017-08-03 20:22:50,662] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.spark.PySparkInterpreter 506689376 created
INFO [2017-08-03 20:22:50,662] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.spark.SparkSqlInterpreter 1462545045 created
INFO [2017-08-03 20:22:50,776] ({qtp459296537-22} InterpreterFactory.java[createInterpretersForNote]:188) - Create interpreter instance postgresql for note 2CNXQE5CR
INFO [2017-08-03 20:22:50,776] ({qtp459296537-22} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.postgresql.PostgreSqlInterpreter 340437485 created
INFO [2017-08-03 20:22:53,610] ({pool-2-thread-2} SchedulerFactory.java[jobStarted]:131) - Job paragraph_1501783535283_1713771734 started by scheduler org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session392982422
INFO [2017-08-03 20:22:53,611] ({pool-2-thread-2} Paragraph.java[jobRun]:362) - run paragraph 20170803-180535_552293631 using psql org.apache.zeppelin.interpreter.LazyOpenInterpreter@144aa9ed
INFO [2017-08-03 20:22:53,620] ({pool-2-thread-2} RemoteInterpreterManagedProcess.java[start]:126) - Run interpreter process [/usr/lib/zeppelin/bin/interpreter.sh, -d, /usr/lib/zeppelin/interpreter/postgresql, -p, 42610, -l, /usr/lib/zeppelin/local-repo/2CPQEJPGC]
INFO [2017-08-03 20:22:54,190] ({pool-2-thread-2} RemoteInterpreter.java[init]:221) - Create remote interpreter org.apache.zeppelin.postgresql.PostgreSqlInterpreter
INFO [2017-08-03 20:22:54,370] ({pool-2-thread-2} RemoteInterpreter.java[pushAngularObjectRegistryToRemote]:551) - Push local angular object registry from ZeppelinServer to remote interpreter group 2CPQEJPGC:shared_process
WARN [2017-08-03 20:22:54,606] ({pool-2-thread-2} NotebookServer.java[afterStatusChange]:2058) - Job 20170803-180535_552293631 is finished, status: ERROR, exception: null, result: %text java.lang.NullPointerException
    at org.apache.zeppelin.postgresql.PostgreSqlInterpreter.executeSql(PostgreSqlInterpreter.java:202)
    at org.apache.zeppelin.postgresql.PostgreSqlInterpreter.interpret(PostgreSqlInterpreter.java:289)
    at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:97)
    at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:498)
    at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
    at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

INFO [2017-08-03 20:22:54,673] ({pool-2-thread-2} SchedulerFactory.java[jobFinished]:137) - Job paragraph_1501783535283_1713771734 finished by scheduler org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session392982422



Reply | Threaded
Open this post in threaded view
|

Re: Zeppelin postgres interpreter throws NPE

Richard Xin


On Thursday, August 3, 2017, 4:53:05 PM PDT, Jianfeng (Jeff) Zhang <[hidden email]> wrote:


Agree, you can create a ticket for better exception message. And the error is better to be displayed in frontend




Best Regard,
Jeff Zhang


From: Richard Xin <[hidden email]>
Reply-To: "[hidden email]" <[hidden email]>
Date: Friday, August 4, 2017 at 4:40 AM
To: "[hidden email]" <[hidden email]>
Subject: Re: Zeppelin postgres interpreter throws NPE

oh, just figured it out myself
it seems that the property I aded in interpreter config postgres.database is not recognized by zeppelin, I solved it by adding dbname to url.

anyway, the exception is better to be handled and emit more friendly error message.


On Thursday, August 3, 2017, 1:35:04 PM PDT, Richard Xin <[hidden email]> wrote:


on AWS EMR
1. Tested remote postgres connection using psql, works OK
2. added postges interpreter via Interpreter UI
3. create new note:
%postgresql   (or %psql)
select * from test.batch_report; 
got NPE

4. here are what's in the log:

WARN [2017-08-03 20:22:46,859] ({qtp459296537-18} LoginRestApi.java[postLogin]:119) - {"status":"OK","message":"","body":{"principal":"richard.xin","ticket":"d86ab3d9-3976-43a4-a274-df50ee3b627a","roles":"[]"}}
INFO [2017-08-03 20:22:50,370] ({qtp459296537-21} NotebookServer.java[sendNote]:705) - New operation from 172.17.197.49 : 57990 : richard.xin : GET_NOTE : 2CNXQE5CR
INFO [2017-08-03 20:22:50,656] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:188) - Create interpreter instance spark for note 2CNXQE5CR
INFO [2017-08-03 20:22:50,661] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.spark.SparkInterpreter 2080711349 created
INFO [2017-08-03 20:22:50,662] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.spark.PySparkInterpreter 506689376 created
INFO [2017-08-03 20:22:50,662] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.spark.SparkSqlInterpreter 1462545045 created
INFO [2017-08-03 20:22:50,776] ({qtp459296537-22} InterpreterFactory.java[createInterpretersForNote]:188) - Create interpreter instance postgresql for note 2CNXQE5CR
INFO [2017-08-03 20:22:50,776] ({qtp459296537-22} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.postgresql.PostgreSqlInterpreter 340437485 created
INFO [2017-08-03 20:22:53,610] ({pool-2-thread-2} SchedulerFactory.java[jobStarted]:131) - Job paragraph_1501783535283_1713771734 started by scheduler org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session392982422
INFO [2017-08-03 20:22:53,611] ({pool-2-thread-2} Paragraph.java[jobRun]:362) - run paragraph 20170803-180535_552293631 using psql org.apache.zeppelin.interpreter.LazyOpenInterpreter@144aa9ed
INFO [2017-08-03 20:22:53,620] ({pool-2-thread-2} RemoteInterpreterManagedProcess.java[start]:126) - Run interpreter process [/usr/lib/zeppelin/bin/interpreter.sh, -d, /usr/lib/zeppelin/interpreter/postgresql, -p, 42610, -l, /usr/lib/zeppelin/local-repo/2CPQEJPGC]
INFO [2017-08-03 20:22:54,190] ({pool-2-thread-2} RemoteInterpreter.java[init]:221) - Create remote interpreter org.apache.zeppelin.postgresql.PostgreSqlInterpreter
INFO [2017-08-03 20:22:54,370] ({pool-2-thread-2} RemoteInterpreter.java[pushAngularObjectRegistryToRemote]:551) - Push local angular object registry from ZeppelinServer to remote interpreter group 2CPQEJPGC:shared_process
WARN [2017-08-03 20:22:54,606] ({pool-2-thread-2} NotebookServer.java[afterStatusChange]:2058) - Job 20170803-180535_552293631 is finished, status: ERROR, exception: null, result: %text java.lang.NullPointerException
    at org.apache.zeppelin.postgresql.PostgreSqlInterpreter.executeSql(PostgreSqlInterpreter.java:202)
    at org.apache.zeppelin.postgresql.PostgreSqlInterpreter.interpret(PostgreSqlInterpreter.java:289)
    at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:97)
    at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:498)
    at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
    at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

INFO [2017-08-03 20:22:54,673] ({pool-2-thread-2} SchedulerFactory.java[jobFinished]:137) - Job paragraph_1501783535283_1713771734 finished by scheduler org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session392982422