multiple instances of the same interpreter type

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

multiple instances of the same interpreter type

Ruslan Dautkhanov
A user managed somehow to launch multiple instances of spark interpreter under the same Zeppelin server.

See a snippet of `pstree` output:

  |-java,6360,wabramov -Dfile.encoding=UTF-8 -Xms1024m -Xmx2048m -XX:MaxPermSize=512m-Dlog4j.configuration=file:///home/wabramov/
  |   |-interpreter.sh,4510 /opt/zeppelin/zeppelin-active/bin/interpreter.sh -d /opt/zeppelin/zeppelin-active/interpreter/spark -p 45986 -l/opt/zeppe
  |   |   `-interpreter.sh,4523 /opt/zeppelin/zeppelin-active/bin/interpreter.sh -d /opt/zeppelin/zeppelin-active/interpreter/spark -p 45986 -l/opt/zeppe
  |   |       `-java,4524 -cp/etc/hive/conf/:/opt/zeppelin/zeppelin-active/interpreter/spark/*:/opt/zeppelin/zeppelin-active/zeppeli
  |   |-interpreter.sh,5097 /opt/zeppelin/zeppelin-active/bin/interpreter.sh -d /opt/zeppelin/zeppelin-active/interpreter/spark -p 39752 -l/opt/zeppe
  |   |   `-interpreter.sh,5110 /opt/zeppelin/zeppelin-active/bin/interpreter.sh -d /opt/zeppelin/zeppelin-active/interpreter/spark -p 39752 -l/opt/zeppe
  |   |       `-java,5111 -cp/etc/hive/conf/:/opt/zeppelin/zeppelin-active/interpreter/spark/*:/opt/zeppelin/zeppelin-active/zeppeli


I see another user has three (3) instances running of %sh interpreter.

Is this a known issue?


--
Ruslan Dautkhanov
Reply | Threaded
Open this post in threaded view
|

Re: multiple instances of the same interpreter type

Jeff Zhang

What mode do you use ?



Ruslan Dautkhanov <[hidden email]>于2017年4月7日周五 上午12:49写道:
A user managed somehow to launch multiple instances of spark interpreter under the same Zeppelin server.

See a snippet of `pstree` output:

  |-java,6360,wabramov -Dfile.encoding=UTF-8 -Xms1024m -Xmx2048m -XX:MaxPermSize=512m-Dlog4j.configuration=file:///home/wabramov/
  |   |-interpreter.sh,4510 /opt/zeppelin/zeppelin-active/bin/interpreter.sh -d /opt/zeppelin/zeppelin-active/interpreter/spark -p 45986 -l/opt/zeppe
  |   |   `-interpreter.sh,4523 /opt/zeppelin/zeppelin-active/bin/interpreter.sh -d /opt/zeppelin/zeppelin-active/interpreter/spark -p 45986 -l/opt/zeppe
  |   |       `-java,4524 -cp/etc/hive/conf/:/opt/zeppelin/zeppelin-active/interpreter/spark/*:/opt/zeppelin/zeppelin-active/zeppeli
  |   |-interpreter.sh,5097 /opt/zeppelin/zeppelin-active/bin/interpreter.sh -d /opt/zeppelin/zeppelin-active/interpreter/spark -p 39752 -l/opt/zeppe
  |   |   `-interpreter.sh,5110 /opt/zeppelin/zeppelin-active/bin/interpreter.sh -d /opt/zeppelin/zeppelin-active/interpreter/spark -p 39752 -l/opt/zeppe
  |   |       `-java,5111 -cp/etc/hive/conf/:/opt/zeppelin/zeppelin-active/interpreter/spark/*:/opt/zeppelin/zeppelin-active/zeppeli


I see another user has three (3) instances running of %sh interpreter.

Is this a known issue?


--
Ruslan Dautkhanov
Reply | Threaded
Open this post in threaded view
|

Re: multiple instances of the same interpreter type

Ruslan Dautkhanov
We have each user running their own Zeppelin instances, 
so everyone has Spark interpreter group defined as 

      "option": {
        ..
        "perNote": "shared",
        "perUser": "shared",
        ..
      }

which translates to "interpreter will be instantiated 
 in 
process."



--
Ruslan Dautkhanov

On Thu, Apr 6, 2017 at 6:34 PM, Jeff Zhang <[hidden email]> wrote:

What mode do you use ?



Ruslan Dautkhanov <[hidden email]>于2017年4月7日周五 上午12:49写道:
A user managed somehow to launch multiple instances of spark interpreter under the same Zeppelin server.

See a snippet of `pstree` output:

  |-java,6360,wabramov -Dfile.encoding=UTF-8 -Xms1024m -Xmx2048m -XX:MaxPermSize=512m-Dlog4j.configuration=file:///home/wabramov/
  |   |-interpreter.sh,4510 /opt/zeppelin/zeppelin-active/bin/interpreter.sh -d /opt/zeppelin/zeppelin-active/interpreter/spark -p 45986 -l/opt/zeppe
  |   |   `-interpreter.sh,4523 /opt/zeppelin/zeppelin-active/bin/interpreter.sh -d /opt/zeppelin/zeppelin-active/interpreter/spark -p 45986 -l/opt/zeppe
  |   |       `-java,4524 -cp/etc/hive/conf/:/opt/zeppelin/zeppelin-active/interpreter/spark/*:/opt/zeppelin/zeppelin-active/zeppeli
  |   |-interpreter.sh,5097 /opt/zeppelin/zeppelin-active/bin/interpreter.sh -d /opt/zeppelin/zeppelin-active/interpreter/spark -p 39752 -l/opt/zeppe
  |   |   `-interpreter.sh,5110 /opt/zeppelin/zeppelin-active/bin/interpreter.sh -d /opt/zeppelin/zeppelin-active/interpreter/spark -p 39752 -l/opt/zeppe
  |   |       `-java,5111 -cp/etc/hive/conf/:/opt/zeppelin/zeppelin-active/interpreter/spark/*:/opt/zeppelin/zeppelin-active/zeppeli


I see another user has three (3) instances running of %sh interpreter.

Is this a known issue?


--
Ruslan Dautkhanov

Reply | Threaded
Open this post in threaded view
|

Re: multiple instances of the same interpreter type

Rick Moritz
Which version of Zeppelin are you running?
There was a bug, where killing Zeppelin wouldn't actually kill the interpreters, because the kill signal wasn't passed through, but that was fixed in 0.7.1: https://issues.apache.org/jira/browse/ZEPPELIN-2258

On Fri, Apr 7, 2017 at 5:31 PM, Ruslan Dautkhanov <[hidden email]> wrote:
We have each user running their own Zeppelin instances, 
so everyone has Spark interpreter group defined as 

      "option": {
        ..
        "perNote": "shared",
        "perUser": "shared",
        ..
      }

which translates to "interpreter will be instantiated 
 in 
process."



--
Ruslan Dautkhanov

On Thu, Apr 6, 2017 at 6:34 PM, Jeff Zhang <[hidden email]> wrote:

What mode do you use ?



Ruslan Dautkhanov <[hidden email]>于2017年4月7日周五 上午12:49写道:
A user managed somehow to launch multiple instances of spark interpreter under the same Zeppelin server.

See a snippet of `pstree` output:

  |-java,6360,wabramov -Dfile.encoding=UTF-8 -Xms1024m -Xmx2048m -XX:MaxPermSize=512m-Dlog4j.configuration=file:///home/wabramov/
  |   |-interpreter.sh,4510 /opt/zeppelin/zeppelin-active/bin/interpreter.sh -d /opt/zeppelin/zeppelin-active/interpreter/spark -p 45986 -l/opt/zeppe
  |   |   `-interpreter.sh,4523 /opt/zeppelin/zeppelin-active/bin/interpreter.sh -d /opt/zeppelin/zeppelin-active/interpreter/spark -p 45986 -l/opt/zeppe
  |   |       `-java,4524 -cp/etc/hive/conf/:/opt/zeppelin/zeppelin-active/interpreter/spark/*:/opt/zeppelin/zeppelin-active/zeppeli
  |   |-interpreter.sh,5097 /opt/zeppelin/zeppelin-active/bin/interpreter.sh -d /opt/zeppelin/zeppelin-active/interpreter/spark -p 39752 -l/opt/zeppe
  |   |   `-interpreter.sh,5110 /opt/zeppelin/zeppelin-active/bin/interpreter.sh -d /opt/zeppelin/zeppelin-active/interpreter/spark -p 39752 -l/opt/zeppe
  |   |       `-java,5111 -cp/etc/hive/conf/:/opt/zeppelin/zeppelin-active/interpreter/spark/*:/opt/zeppelin/zeppelin-active/zeppeli


I see another user has three (3) instances running of %sh interpreter.

Is this a known issue?


--
Ruslan Dautkhanov