Zeppelin postgres interpreter throws NPE

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Zeppelin postgres interpreter throws NPE

Richard Xin
on AWS EMR
1. Tested remote postgres connection using psql, works OK
2. added postges interpreter via Interpreter UI
3. create new note:
%postgresql   (or %psql)
select * from test.batch_report; 
got NPE

4. here are what's in the log:

WARN [2017-08-03 20:22:46,859] ({qtp459296537-18} LoginRestApi.java[postLogin]:119) - {"status":"OK","message":"","body":{"principal":"richard.xin","ticket":"d86ab3d9-3976-43a4-a274-df50ee3b627a","roles":"[]"}}
INFO [2017-08-03 20:22:50,370] ({qtp459296537-21} NotebookServer.java[sendNote]:705) - New operation from 172.17.197.49 : 57990 : richard.xin : GET_NOTE : 2CNXQE5CR
INFO [2017-08-03 20:22:50,656] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:188) - Create interpreter instance spark for note 2CNXQE5CR
INFO [2017-08-03 20:22:50,661] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.spark.SparkInterpreter 2080711349 created
INFO [2017-08-03 20:22:50,662] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.spark.PySparkInterpreter 506689376 created
INFO [2017-08-03 20:22:50,662] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.spark.SparkSqlInterpreter 1462545045 created
INFO [2017-08-03 20:22:50,776] ({qtp459296537-22} InterpreterFactory.java[createInterpretersForNote]:188) - Create interpreter instance postgresql for note 2CNXQE5CR
INFO [2017-08-03 20:22:50,776] ({qtp459296537-22} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.postgresql.PostgreSqlInterpreter 340437485 created
INFO [2017-08-03 20:22:53,610] ({pool-2-thread-2} SchedulerFactory.java[jobStarted]:131) - Job paragraph_1501783535283_1713771734 started by scheduler org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session392982422
INFO [2017-08-03 20:22:53,611] ({pool-2-thread-2} Paragraph.java[jobRun]:362) - run paragraph 20170803-180535_552293631 using psql org.apache.zeppelin.interpreter.LazyOpenInterpreter@144aa9ed
INFO [2017-08-03 20:22:53,620] ({pool-2-thread-2} RemoteInterpreterManagedProcess.java[start]:126) - Run interpreter process [/usr/lib/zeppelin/bin/interpreter.sh, -d, /usr/lib/zeppelin/interpreter/postgresql, -p, 42610, -l, /usr/lib/zeppelin/local-repo/2CPQEJPGC]
INFO [2017-08-03 20:22:54,190] ({pool-2-thread-2} RemoteInterpreter.java[init]:221) - Create remote interpreter org.apache.zeppelin.postgresql.PostgreSqlInterpreter
INFO [2017-08-03 20:22:54,370] ({pool-2-thread-2} RemoteInterpreter.java[pushAngularObjectRegistryToRemote]:551) - Push local angular object registry from ZeppelinServer to remote interpreter group 2CPQEJPGC:shared_process
WARN [2017-08-03 20:22:54,606] ({pool-2-thread-2} NotebookServer.java[afterStatusChange]:2058) - Job 20170803-180535_552293631 is finished, status: ERROR, exception: null, result: %text java.lang.NullPointerException
    at org.apache.zeppelin.postgresql.PostgreSqlInterpreter.executeSql(PostgreSqlInterpreter.java:202)
    at org.apache.zeppelin.postgresql.PostgreSqlInterpreter.interpret(PostgreSqlInterpreter.java:289)
    at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:97)
    at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:498)
    at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
    at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

INFO [2017-08-03 20:22:54,673] ({pool-2-thread-2} SchedulerFactory.java[jobFinished]:137) - Job paragraph_1501783535283_1713771734 finished by scheduler org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session392982422



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Zeppelin postgres interpreter throws NPE

Richard Xin
oh, just figured it out myself
it seems that the property I aded in interpreter config postgres.database is not recognized by zeppelin, I solved it by adding dbname to url.

anyway, the exception is better to be handled and emit more friendly error message.


On Thursday, August 3, 2017, 1:35:04 PM PDT, Richard Xin <[hidden email]> wrote:


on AWS EMR
1. Tested remote postgres connection using psql, works OK
2. added postges interpreter via Interpreter UI
3. create new note:
%postgresql   (or %psql)
select * from test.batch_report; 
got NPE

4. here are what's in the log:

WARN [2017-08-03 20:22:46,859] ({qtp459296537-18} LoginRestApi.java[postLogin]:119) - {"status":"OK","message":"","body":{"principal":"richard.xin","ticket":"d86ab3d9-3976-43a4-a274-df50ee3b627a","roles":"[]"}}
INFO [2017-08-03 20:22:50,370] ({qtp459296537-21} NotebookServer.java[sendNote]:705) - New operation from 172.17.197.49 : 57990 : richard.xin : GET_NOTE : 2CNXQE5CR
INFO [2017-08-03 20:22:50,656] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:188) - Create interpreter instance spark for note 2CNXQE5CR
INFO [2017-08-03 20:22:50,661] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.spark.SparkInterpreter 2080711349 created
INFO [2017-08-03 20:22:50,662] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.spark.PySparkInterpreter 506689376 created
INFO [2017-08-03 20:22:50,662] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.spark.SparkSqlInterpreter 1462545045 created
INFO [2017-08-03 20:22:50,776] ({qtp459296537-22} InterpreterFactory.java[createInterpretersForNote]:188) - Create interpreter instance postgresql for note 2CNXQE5CR
INFO [2017-08-03 20:22:50,776] ({qtp459296537-22} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.postgresql.PostgreSqlInterpreter 340437485 created
INFO [2017-08-03 20:22:53,610] ({pool-2-thread-2} SchedulerFactory.java[jobStarted]:131) - Job paragraph_1501783535283_1713771734 started by scheduler org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session392982422
INFO [2017-08-03 20:22:53,611] ({pool-2-thread-2} Paragraph.java[jobRun]:362) - run paragraph 20170803-180535_552293631 using psql org.apache.zeppelin.interpreter.LazyOpenInterpreter@144aa9ed
INFO [2017-08-03 20:22:53,620] ({pool-2-thread-2} RemoteInterpreterManagedProcess.java[start]:126) - Run interpreter process [/usr/lib/zeppelin/bin/interpreter.sh, -d, /usr/lib/zeppelin/interpreter/postgresql, -p, 42610, -l, /usr/lib/zeppelin/local-repo/2CPQEJPGC]
INFO [2017-08-03 20:22:54,190] ({pool-2-thread-2} RemoteInterpreter.java[init]:221) - Create remote interpreter org.apache.zeppelin.postgresql.PostgreSqlInterpreter
INFO [2017-08-03 20:22:54,370] ({pool-2-thread-2} RemoteInterpreter.java[pushAngularObjectRegistryToRemote]:551) - Push local angular object registry from ZeppelinServer to remote interpreter group 2CPQEJPGC:shared_process
WARN [2017-08-03 20:22:54,606] ({pool-2-thread-2} NotebookServer.java[afterStatusChange]:2058) - Job 20170803-180535_552293631 is finished, status: ERROR, exception: null, result: %text java.lang.NullPointerException
    at org.apache.zeppelin.postgresql.PostgreSqlInterpreter.executeSql(PostgreSqlInterpreter.java:202)
    at org.apache.zeppelin.postgresql.PostgreSqlInterpreter.interpret(PostgreSqlInterpreter.java:289)
    at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:97)
    at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:498)
    at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
    at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

INFO [2017-08-03 20:22:54,673] ({pool-2-thread-2} SchedulerFactory.java[jobFinished]:137) - Job paragraph_1501783535283_1713771734 finished by scheduler org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session392982422



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Zeppelin postgres interpreter throws NPE

Jianfeng (Jeff) Zhang
Agree, you can create a ticket for better exception message. And the error is better to be displayed in frontend




Best Regard,
Jeff Zhang


From: Richard Xin <[hidden email]>
Reply-To: "[hidden email]" <[hidden email]>
Date: Friday, August 4, 2017 at 4:40 AM
To: "[hidden email]" <[hidden email]>
Subject: Re: Zeppelin postgres interpreter throws NPE

oh, just figured it out myself
it seems that the property I aded in interpreter config postgres.database is not recognized by zeppelin, I solved it by adding dbname to url.

anyway, the exception is better to be handled and emit more friendly error message.


On Thursday, August 3, 2017, 1:35:04 PM PDT, Richard Xin <[hidden email]> wrote:


on AWS EMR
1. Tested remote postgres connection using psql, works OK
2. added postges interpreter via Interpreter UI
3. create new note:
%postgresql   (or %psql)
select * from test.batch_report; 
got NPE

4. here are what's in the log:

WARN [2017-08-03 20:22:46,859] ({qtp459296537-18} LoginRestApi.java[postLogin]:119) - {"status":"OK","message":"","body":{"principal":"richard.xin","ticket":"d86ab3d9-3976-43a4-a274-df50ee3b627a","roles":"[]"}}
INFO [2017-08-03 20:22:50,370] ({qtp459296537-21} NotebookServer.java[sendNote]:705) - New operation from 172.17.197.49 : 57990 : richard.xin : GET_NOTE : 2CNXQE5CR
INFO [2017-08-03 20:22:50,656] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:188) - Create interpreter instance spark for note 2CNXQE5CR
INFO [2017-08-03 20:22:50,661] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.spark.SparkInterpreter 2080711349 created
INFO [2017-08-03 20:22:50,662] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.spark.PySparkInterpreter 506689376 created
INFO [2017-08-03 20:22:50,662] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.spark.SparkSqlInterpreter 1462545045 created
INFO [2017-08-03 20:22:50,776] ({qtp459296537-22} InterpreterFactory.java[createInterpretersForNote]:188) - Create interpreter instance postgresql for note 2CNXQE5CR
INFO [2017-08-03 20:22:50,776] ({qtp459296537-22} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.postgresql.PostgreSqlInterpreter 340437485 created
INFO [2017-08-03 20:22:53,610] ({pool-2-thread-2} SchedulerFactory.java[jobStarted]:131) - Job paragraph_1501783535283_1713771734 started by scheduler org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session392982422
INFO [2017-08-03 20:22:53,611] ({pool-2-thread-2} Paragraph.java[jobRun]:362) - run paragraph 20170803-180535_552293631 using psql org.apache.zeppelin.interpreter.LazyOpenInterpreter@144aa9ed
INFO [2017-08-03 20:22:53,620] ({pool-2-thread-2} RemoteInterpreterManagedProcess.java[start]:126) - Run interpreter process [/usr/lib/zeppelin/bin/interpreter.sh, -d, /usr/lib/zeppelin/interpreter/postgresql, -p, 42610, -l, /usr/lib/zeppelin/local-repo/2CPQEJPGC]
INFO [2017-08-03 20:22:54,190] ({pool-2-thread-2} RemoteInterpreter.java[init]:221) - Create remote interpreter org.apache.zeppelin.postgresql.PostgreSqlInterpreter
INFO [2017-08-03 20:22:54,370] ({pool-2-thread-2} RemoteInterpreter.java[pushAngularObjectRegistryToRemote]:551) - Push local angular object registry from ZeppelinServer to remote interpreter group 2CPQEJPGC:shared_process
WARN [2017-08-03 20:22:54,606] ({pool-2-thread-2} NotebookServer.java[afterStatusChange]:2058) - Job 20170803-180535_552293631 is finished, status: ERROR, exception: null, result: %text java.lang.NullPointerException
    at org.apache.zeppelin.postgresql.PostgreSqlInterpreter.executeSql(PostgreSqlInterpreter.java:202)
    at org.apache.zeppelin.postgresql.PostgreSqlInterpreter.interpret(PostgreSqlInterpreter.java:289)
    at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:97)
    at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:498)
    at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
    at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

INFO [2017-08-03 20:22:54,673] ({pool-2-thread-2} SchedulerFactory.java[jobFinished]:137) - Job paragraph_1501783535283_1713771734 finished by scheduler org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session392982422



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Zeppelin postgres interpreter throws NPE

Richard Xin


On Thursday, August 3, 2017, 4:53:05 PM PDT, Jianfeng (Jeff) Zhang <[hidden email]> wrote:


Agree, you can create a ticket for better exception message. And the error is better to be displayed in frontend




Best Regard,
Jeff Zhang


From: Richard Xin <[hidden email]>
Reply-To: "[hidden email]" <[hidden email]>
Date: Friday, August 4, 2017 at 4:40 AM
To: "[hidden email]" <[hidden email]>
Subject: Re: Zeppelin postgres interpreter throws NPE

oh, just figured it out myself
it seems that the property I aded in interpreter config postgres.database is not recognized by zeppelin, I solved it by adding dbname to url.

anyway, the exception is better to be handled and emit more friendly error message.


On Thursday, August 3, 2017, 1:35:04 PM PDT, Richard Xin <[hidden email]> wrote:


on AWS EMR
1. Tested remote postgres connection using psql, works OK
2. added postges interpreter via Interpreter UI
3. create new note:
%postgresql   (or %psql)
select * from test.batch_report; 
got NPE

4. here are what's in the log:

WARN [2017-08-03 20:22:46,859] ({qtp459296537-18} LoginRestApi.java[postLogin]:119) - {"status":"OK","message":"","body":{"principal":"richard.xin","ticket":"d86ab3d9-3976-43a4-a274-df50ee3b627a","roles":"[]"}}
INFO [2017-08-03 20:22:50,370] ({qtp459296537-21} NotebookServer.java[sendNote]:705) - New operation from 172.17.197.49 : 57990 : richard.xin : GET_NOTE : 2CNXQE5CR
INFO [2017-08-03 20:22:50,656] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:188) - Create interpreter instance spark for note 2CNXQE5CR
INFO [2017-08-03 20:22:50,661] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.spark.SparkInterpreter 2080711349 created
INFO [2017-08-03 20:22:50,662] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.spark.PySparkInterpreter 506689376 created
INFO [2017-08-03 20:22:50,662] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.spark.SparkSqlInterpreter 1462545045 created
INFO [2017-08-03 20:22:50,776] ({qtp459296537-22} InterpreterFactory.java[createInterpretersForNote]:188) - Create interpreter instance postgresql for note 2CNXQE5CR
INFO [2017-08-03 20:22:50,776] ({qtp459296537-22} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.postgresql.PostgreSqlInterpreter 340437485 created
INFO [2017-08-03 20:22:53,610] ({pool-2-thread-2} SchedulerFactory.java[jobStarted]:131) - Job paragraph_1501783535283_1713771734 started by scheduler org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session392982422
INFO [2017-08-03 20:22:53,611] ({pool-2-thread-2} Paragraph.java[jobRun]:362) - run paragraph 20170803-180535_552293631 using psql org.apache.zeppelin.interpreter.LazyOpenInterpreter@144aa9ed
INFO [2017-08-03 20:22:53,620] ({pool-2-thread-2} RemoteInterpreterManagedProcess.java[start]:126) - Run interpreter process [/usr/lib/zeppelin/bin/interpreter.sh, -d, /usr/lib/zeppelin/interpreter/postgresql, -p, 42610, -l, /usr/lib/zeppelin/local-repo/2CPQEJPGC]
INFO [2017-08-03 20:22:54,190] ({pool-2-thread-2} RemoteInterpreter.java[init]:221) - Create remote interpreter org.apache.zeppelin.postgresql.PostgreSqlInterpreter
INFO [2017-08-03 20:22:54,370] ({pool-2-thread-2} RemoteInterpreter.java[pushAngularObjectRegistryToRemote]:551) - Push local angular object registry from ZeppelinServer to remote interpreter group 2CPQEJPGC:shared_process
WARN [2017-08-03 20:22:54,606] ({pool-2-thread-2} NotebookServer.java[afterStatusChange]:2058) - Job 20170803-180535_552293631 is finished, status: ERROR, exception: null, result: %text java.lang.NullPointerException
    at org.apache.zeppelin.postgresql.PostgreSqlInterpreter.executeSql(PostgreSqlInterpreter.java:202)
    at org.apache.zeppelin.postgresql.PostgreSqlInterpreter.interpret(PostgreSqlInterpreter.java:289)
    at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:97)
    at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:498)
    at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
    at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

INFO [2017-08-03 20:22:54,673] ({pool-2-thread-2} SchedulerFactory.java[jobFinished]:137) - Job paragraph_1501783535283_1713771734 finished by scheduler org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session392982422



Loading...