Cannot connect to Flink cluster (TTransportException)

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Cannot connect to Flink cluster (TTransportException)

Jens Grassel
Hi,

we've tried to connect to a flink (1.1.3) cluster (started with
`start-cluster.sh`) from zeppelin 0.7.3 (bin-all).
The cluster is running at localhost:6123 and we configured the
interpreter using:

[X] Connect to existing process
Host : localhost
Port : 6123

But if we try to run flink code from a notebook (for example the
included flink example notebook) we get the following error message:

<---snip--->

ERROR [2017-11-08 13:21:09,289] ({pool-2-thread-2} RemoteInterpreter.java[init]:236) - Failed to create interpreter: org.apache.zeppelin.flink.FlinkInterpreter
ERROR [2017-11-08 13:21:09,290] ({pool-2-thread-2} RemoteInterpreter.java[open]:275) - Failed to initialize interpreter: org.apache.zeppelin.flink.FlinkInterpreter. Remove it from interpreterGroup
ERROR [2017-11-08 13:21:09,300] ({pool-2-thread-2} Job.java[run]:188) - Job failed
org.apache.zeppelin.interpreter.InterpreterException: org.apache.thrift.transport.TTransportException
        at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:450)
        at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:111)
        at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:387)
        at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
        at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:329)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.thrift.transport.TTransportException
        at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
        at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
        at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
        at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)
        at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
        at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
        at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.recv_getFormType(RemoteInterpreterService.java:337)
        at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.getFormType(RemoteInterpreterService.java:323)
        at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:446)
        ... 11 more

<---snip--->

Any ideas? We haven't found anything online. :-(

Regards,

Jens

--
CTO, Wegtam GmbH, 08. Nebelung 2017, 13:53
Homepage : https://www.wegtam.com

Those who talk don't know.  Those who don't talk, know.

attachment0 (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Cannot connect to Flink cluster (TTransportException)

Jeff Zhang

There's another log file which is interpreter log under the logs folder, could you check that file ?


Jens Grassel <[hidden email]>于2017年11月8日周三 下午9:01写道:
Hi,

we've tried to connect to a flink (1.1.3) cluster (started with
`start-cluster.sh`) from zeppelin 0.7.3 (bin-all).
The cluster is running at localhost:6123 and we configured the
interpreter using:

[X] Connect to existing process
Host : localhost
Port : 6123

But if we try to run flink code from a notebook (for example the
included flink example notebook) we get the following error message:

<---snip--->

ERROR [2017-11-08 13:21:09,289] ({pool-2-thread-2} RemoteInterpreter.java[init]:236) - Failed to create interpreter: org.apache.zeppelin.flink.FlinkInterpreter
ERROR [2017-11-08 13:21:09,290] ({pool-2-thread-2} RemoteInterpreter.java[open]:275) - Failed to initialize interpreter: org.apache.zeppelin.flink.FlinkInterpreter. Remove it from interpreterGroup
ERROR [2017-11-08 13:21:09,300] ({pool-2-thread-2} Job.java[run]:188) - Job failed
org.apache.zeppelin.interpreter.InterpreterException: org.apache.thrift.transport.TTransportException
        at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:450)
        at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:111)
        at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:387)
        at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
        at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:329)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.thrift.transport.TTransportException
        at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
        at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
        at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
        at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)
        at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
        at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
        at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.recv_getFormType(RemoteInterpreterService.java:337)
        at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.getFormType(RemoteInterpreterService.java:323)
        at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:446)
        ... 11 more

<---snip--->

Any ideas? We haven't found anything online. :-(

Regards,

Jens

--
CTO, Wegtam GmbH, 08. Nebelung 2017, 13:53
Homepage : https://www.wegtam.com

Those who talk don't know.  Those who don't talk, know.
Reply | Threaded
Open this post in threaded view
|

Re: Cannot connect to Flink cluster (TTransportException)

Jens Grassel
Hi,

On Wed, 08 Nov 2017 13:23:11 +0000
Jeff Zhang <[hidden email]> wrote:

JZ> There's another log file which is interpreter log under the logs
JZ> folder, could you check that file ?

there are just two logfiles in that folder:

zeppelin-whatever.log
zeppelin-whatever.out

However if we configure the interpreter to use the embedded (included)
one then a logfile named zeppelin-interpreter-flink-whatever.log is
created.

Regards,

Jens

--
CTO, Wegtam GmbH, 08. Nebelung 2017, 14:39
Homepage : https://www.wegtam.com

Johnson's First Law:
        When any mechanical contrivance fails, it will do so at the
        most inconvenient possible time.

attachment0 (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Cannot connect to Flink cluster (TTransportException)

Jeff Zhang

zeppelin-interpreter-flink-whatever.log  is what I mean

Jens Grassel <[hidden email]>于2017年11月8日周三 下午9:42写道:
Hi,

On Wed, 08 Nov 2017 13:23:11 +0000
Jeff Zhang <[hidden email]> wrote:

JZ> There's another log file which is interpreter log under the logs
JZ> folder, could you check that file ?

there are just two logfiles in that folder:

zeppelin-whatever.log
zeppelin-whatever.out

However if we configure the interpreter to use the embedded (included)
one then a logfile named zeppelin-interpreter-flink-whatever.log is
created.

Regards,

Jens

--
CTO, Wegtam GmbH, 08. Nebelung 2017, 14:39
Homepage : https://www.wegtam.com

Johnson's First Law:
        When any mechanical contrivance fails, it will do so at the
        most inconvenient possible time.
Reply | Threaded
Open this post in threaded view
|

Re: Cannot connect to Flink cluster (TTransportException)

Jens Grassel
On Wed, 08 Nov 2017 13:52:50 +0000
Jeff Zhang <[hidden email]> wrote:

JZ> zeppelin-interpreter-flink-whatever.log  is what I mean

As mentioned this log file is only created by the embedded flink
interpreter. If the "Connect to existing process" option is used the
file is not created and even if it exist no data is written to it.
The file you mentioned only contains data from the embedded flink which
runs fine but no error messages or warnings regarding the connection to
an existing flink process.

Regards,

Jens

--
CTO, Wegtam GmbH, 08. Nebelung 2017, 14:56
Homepage : https://www.wegtam.com

Real programmers don't write in BASIC.  Actually, no programmers write
in BASIC after reaching puberty.

attachment0 (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Cannot connect to Flink cluster (TTransportException)

Jeff Zhang

I think you misunderstand the connecting existing process, it means connecting to an existing zeppelin interpreter process. Here it is flink interpreter process. But seems 
you are configure it to connect to flink cluster. I am not expert in flink, for how to connect to flink cluster, someone else may be able to help you.





Jens Grassel <[hidden email]>于2017年11月8日周三 下午10:00写道:
On Wed, 08 Nov 2017 13:52:50 +0000
Jeff Zhang <[hidden email]> wrote:

JZ> zeppelin-interpreter-flink-whatever.log  is what I mean

As mentioned this log file is only created by the embedded flink
interpreter. If the "Connect to existing process" option is used the
file is not created and even if it exist no data is written to it.
The file you mentioned only contains data from the embedded flink which
runs fine but no error messages or warnings regarding the connection to
an existing flink process.

Regards,

Jens

--
CTO, Wegtam GmbH, 08. Nebelung 2017, 14:56
Homepage : https://www.wegtam.com

Real programmers don't write in BASIC.  Actually, no programmers write
in BASIC after reaching puberty.
Reply | Threaded
Open this post in threaded view
|

Re: Cannot connect to Flink cluster (TTransportException)

Jens Grassel
Hi,

On Wed, 08 Nov 2017 14:47:00 +0000
Jeff Zhang <[hidden email]> wrote:

JZ> I think you misunderstand the connecting existing process, it means
JZ> connecting to an existing zeppelin interpreter process. Here it is
JZ> flink interpreter process. But seems
JZ> you are configure it to connect to flink cluster. I am not expert
JZ> in flink, for how to connect to flink cluster, someone else may be
JZ> able to help you.

thanks for the enlightenment. I already guessed that but it is good to
have this confirmed.

Is there a way to cluster zeppelin like flink?

On the other hand it would be great if there are resources how to
connect to an existing flink cluster because that is what we have
already. ;-)

Regards,

Jens

--
CTO, Wegtam GmbH, 09. Nebelung 2017, 08:13
Homepage : https://www.wegtam.com

Go to a movie tonight.
Darkness becomes you.

attachment0 (849 bytes) Download Attachment