Multi tenancy - isolated interpreters block each other?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Multi tenancy - isolated interpreters block each other?

Wade Jensen

Hello,

 

I have been experimenting with multi user modes with 0.7.1 in EMR. I set the Spark interpreter scope to be isolated per user, and then ran two different users side by side in different browser tabs. All the privacy and sharing features worked, but whenever I tried to run a spark sql query in both notes at the same time, one query will hang at 0% until the other is completed. (Note the queries where on different data, so it is not that the data was locked by one query).

 

Sorry if this is documented explicitly somewhere, but is this expected behaviour? Or should I be able to run multiple queries with separate interpreters in parallel?

 

I looked in the YARN scheduler GUI and could see two separate Spark processes, so I would have thought this should work.

 

Kind regards,

Wade Jensen

Reply | Threaded
Open this post in threaded view
|

Re: Multi tenancy - isolated interpreters block each other?

Jongyoul Lee
Hi,

Thanks for reporting this bug. Can you make a jira issue and describe how to reproduce it?

Thanks,
Jongyoul

On Tue, Jun 6, 2017 at 9:35 AM, Wade Jensen <[hidden email]> wrote:

Hello,

 

I have been experimenting with multi user modes with 0.7.1 in EMR. I set the Spark interpreter scope to be isolated per user, and then ran two different users side by side in different browser tabs. All the privacy and sharing features worked, but whenever I tried to run a spark sql query in both notes at the same time, one query will hang at 0% until the other is completed. (Note the queries where on different data, so it is not that the data was locked by one query).

 

Sorry if this is documented explicitly somewhere, but is this expected behaviour? Or should I be able to run multiple queries with separate interpreters in parallel?

 

I looked in the YARN scheduler GUI and could see two separate Spark processes, so I would have thought this should work.

 

Kind regards,

Wade Jensen




--
이종열, Jongyoul Lee, 李宗烈
Reply | Threaded
Open this post in threaded view
|

Re: Multi tenancy - isolated interpreters block each other?

Fabian Böhnlein
Hi Wade,

how's the experimenting going? We usually run scoped 'Per Note'.

Have you played with 

zeppelin.spark.concurrentSQL

wich is false by default?

Best,
Fabian

On Fri, 9 Jun 2017 at 13:40 Jongyoul Lee <[hidden email]> wrote:
Hi,

Thanks for reporting this bug. Can you make a jira issue and describe how to reproduce it?

Thanks,
Jongyoul

On Tue, Jun 6, 2017 at 9:35 AM, Wade Jensen <[hidden email]> wrote:

Hello,

 

I have been experimenting with multi user modes with 0.7.1 in EMR. I set the Spark interpreter scope to be isolated per user, and then ran two different users side by side in different browser tabs. All the privacy and sharing features worked, but whenever I tried to run a spark sql query in both notes at the same time, one query will hang at 0% until the other is completed. (Note the queries where on different data, so it is not that the data was locked by one query).

 

Sorry if this is documented explicitly somewhere, but is this expected behaviour? Or should I be able to run multiple queries with separate interpreters in parallel?

 

I looked in the YARN scheduler GUI and could see two separate Spark processes, so I would have thought this should work.

 

Kind regards,

Wade Jensen




--
이종열, Jongyoul Lee, 李宗烈