User Impersonation Configuration

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

User Impersonation Configuration

Yeshwanth Jagini
Hi Users,

I am trying to setup Zeppelin for multiple users.
and i found there are multiple configurations in different places. started tinkering with them and i didn't had any luck .
here's my setup and configuration.

Zeppelin server is running as root.

i edited zeppelin-env.sh  and uncommented 

export ZEPPELIN_IMPERSONATE_CMD='sudo -H -u ${ZEPPELIN_IMPERSONATE_USER} bash -c'       # Optional, when user want to run interpreter as end web user. eg) 'sudo -H -u ${ZEPPELIN_IMPERSONATE_USER} bash -c '
export ZEPPELIN_IMPERSONATE_SPARK_PROXY_USER=true  #Optional, by default is true; can be set to false if you don't want to use --proxy-user option with Spark interpreter when impersonation enabled

and  the interpreter settings are as follows

Inline image 1


for different combination of configurations i am getting different types of errors

if i do not specify impersonation configuration in zeppelin-env.sh  and specify impersonation in interpreter setting i am getting a org.apache.zeppelin.interpreter.InterpreterException: Host key verification failed.
if i specify both zeppelin impersonation configuration and interpreter impersonation config, it's throwing error as user1 cannot impersonate user1

if i do not specify any impersonation configuration at all , interpreter is launching spark-submit as root. that's expected.

could some one please explain me how to set impersonation config and   which configuration i am messing up here

 
Thanks,
Yeshwanth Jagini
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: User Impersonation Configuration

Prabhjyot Singh-2
Hi Yeshwant,

Which version of Zeppelin are you on? 

If you are on latest then you don't need to do any of ZEPPELIN_IMPERSONATE_CMD or ZEPPELIN_IMPERSONATE_SPARK_PROXY_USER. Just by enabling User Impersonation check-box should be sufficient.

Can you confirm by `ps aux | grep spark`. This is what I see on my machine;

prabhjyotsingh@MACHINE:~/ps-zeppelin/logs$ ps aux | grep spark
prabhjyotsingh    2496   0.2  3.9  5179540 657660 s000  S    12:08PM   0:30.68 /Library/Java/JavaVirtualMachines/jdk1.8.0_102.jdk/Contents/Home/bin/java -cp /Users/prabhjyotsingh/ps-zeppelin/interpreter/spark/*:/Users/prabhjyotsingh/ps-zeppelin/zeppelin-interpreter/target/lib/*:/Users/prabhjyotsingh/ps-zeppelin/zeppelin-interpreter/target/classes/:/Users/prabhjyotsingh/ps-zeppelin/zeppelin-interpreter/target/test-classes/:/Users/prabhjyotsingh/ps-zeppelin/zeppelin-zengine/target/test-classes/:/Users/prabhjyotsingh/ps-zeppelin/interpreter/spark/zeppelin-spark_2.10-0.8.0-SNAPSHOT.jar:/Users/prabhjyotsingh/spark-2.0.0-bin-hadoop2.7/conf/:/Users/prabhjyotsingh/spark-2.0.0-bin-hadoop2.7/jars/* -Xmx1g -Dfile.encoding=UTF-8 -Dlog4j.configuration=file:///Users/prabhjyotsingh/ps-zeppelin/conf/log4j.properties -Dzeppelin.log.file=/Users/prabhjyotsingh/ps-zeppelin/logs/zeppelin-interpreter-spark-user1-spark-prabhjyotsingh-HW11610.local.log org.apache.spark.deploy.SparkSubmit --conf spark.driver.extraClassPath=:/Users/prabhjyotsingh/ps-zeppelin/interpreter/spark/*:/Users/prabhjyotsingh/ps-zeppelin/zeppelin-interpreter/target/lib/*::/Users/prabhjyotsingh/ps-zeppelin/zeppelin-interpreter/target/classes:/Users/prabhjyotsingh/ps-zeppelin/zeppelin-interpreter/target/test-classes:/Users/prabhjyotsingh/ps-zeppelin/zeppelin-zengine/target/test-classes:/Users/prabhjyotsingh/ps-zeppelin/interpreter/spark/zeppelin-spark_2.10-0.8.0-SNAPSHOT.jar --conf spark.driver.extraJavaOptions= -Dfile.encoding=UTF-8 -Dlog4j.configuration=file:///Users/prabhjyotsingh/ps-zeppelin/conf/log4j.properties -Dzeppelin.log.file=/Users/prabhjyotsingh/ps-zeppelin/logs/zeppelin-interpreter-spark-user1-spark-prabhjyotsingh-HW11610.local.log --class org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer --proxy-user user1 /Users/prabhjyotsingh/ps-zeppelin/interpreter/spark/zeppelin-spark_2.10-0.8.0-SNAPSHOT.jar 50911
prabhjyotsingh    2508   0.0  0.0  2445100    860 s000  S+   12:08PM   0:00.00 grep spark
prabhjyotsingh    2495   0.0  0.0  2465144    764 s000  S    12:08PM   0:00.00 /bin/bash /Users/prabhjyotsingh/ps-zeppelin/bin/interpreter.sh -d /Users/prabhjyotsingh/ps-zeppelin/interpreter/spark -p 50911 -u user1 -l /Users/prabhjyotsingh/ps-zeppelin/local-repo/2CEZC4JXN -g spark
prabhjyotsingh    2484   0.0  0.0  2465144   1368 s000  S    12:08PM   0:00.01 /bin/bash /Users/prabhjyotsingh/ps-zeppelin/bin/interpreter.sh -d /Users/prabhjyotsingh/ps-zeppelin/interpreter/spark -p 50911 -u user1 -l /Users/prabhjyotsingh/ps-zeppelin/local-repo/2CEZC4JXN -g spark



On 10 May 2017 at 06:10, Yeshwanth Jagini <[hidden email]> wrote:
Hi Users,

I am trying to setup Zeppelin for multiple users.
and i found there are multiple configurations in different places. started tinkering with them and i didn't had any luck .
here's my setup and configuration.

Zeppelin server is running as root.

i edited zeppelin-env.sh  and uncommented 

export ZEPPELIN_IMPERSONATE_CMD='sudo -H -u ${ZEPPELIN_IMPERSONATE_USER} bash -c'       # Optional, when user want to run interpreter as end web user. eg) 'sudo -H -u ${ZEPPELIN_IMPERSONATE_USER} bash -c '
export ZEPPELIN_IMPERSONATE_SPARK_PROXY_USER=true  #Optional, by default is true; can be set to false if you don't want to use --proxy-user option with Spark interpreter when impersonation enabled

and  the interpreter settings are as follows

Inline image 1


for different combination of configurations i am getting different types of errors

if i do not specify impersonation configuration in zeppelin-env.sh  and specify impersonation in interpreter setting i am getting a org.apache.zeppelin.interpreter.InterpreterException: Host key verification failed.
if i specify both zeppelin impersonation configuration and interpreter impersonation config, it's throwing error as user1 cannot impersonate user1

if i do not specify any impersonation configuration at all , interpreter is launching spark-submit as root. that's expected.

could some one please explain me how to set impersonation config and   which configuration i am messing up here

 
Thanks,
Yeshwanth Jagini



--

Warm Regards, 

Prabhjyot Singh
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: User Impersonation Configuration

Yeshwanth Jagini
Hi prabhjyot,
thanks for your reply.

i am using zeppelin 0.7.0 version.
when i do not specify impersonation config in zeppelin-env.sh and only in interpreter setting,
it is throwing following  exception

ERROR [2017-05-10 17:26:30,551] ({pool-2-thread-3} Job.java[run]:188) - Job failed
org.apache.zeppelin.interpreter.InterpreterException: Host key verification failed.

at org.apache.zeppelin.interpreter.remote.RemoteInterpreterManagedProcess.start(RemoteInterpreterManagedProcess.java:143)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.reference(RemoteInterpreterProcess.java:73)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.open(RemoteInterpreter.java:258)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:423)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:106)
at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:387)
at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:329)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:473)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)



i am running zeppelin as root user, root user doesn't had a password less ssh setup where as the end web user user1 has.

how should i proceed now?

Thanks,
Yeshwanth Jagini










On Wed, May 10, 2017 at 1:45 AM, Prabhjyot Singh <[hidden email]> wrote:
Hi Yeshwant,

Which version of Zeppelin are you on? 

If you are on latest then you don't need to do any of ZEPPELIN_IMPERSONATE_CMD or ZEPPELIN_IMPERSONATE_SPARK_PROXY_USER. Just by enabling User Impersonation check-box should be sufficient.

Can you confirm by `ps aux | grep spark`. This is what I see on my machine;

prabhjyotsingh@MACHINE:~/ps-zeppelin/logs$ ps aux | grep spark
prabhjyotsingh    2496   0.2  3.9  5179540 657660 s000  S    12:08PM   0:30.68 /Library/Java/JavaVirtualMachines/jdk1.8.0_102.jdk/Contents/Home/bin/java -cp /Users/prabhjyotsingh/ps-zeppelin/interpreter/spark/*:/Users/prabhjyotsingh/ps-zeppelin/zeppelin-interpreter/target/lib/*:/Users/prabhjyotsingh/ps-zeppelin/zeppelin-interpreter/target/classes/:/Users/prabhjyotsingh/ps-zeppelin/zeppelin-interpreter/target/test-classes/:/Users/prabhjyotsingh/ps-zeppelin/zeppelin-zengine/target/test-classes/:/Users/prabhjyotsingh/ps-zeppelin/interpreter/spark/zeppelin-spark_2.10-0.8.0-SNAPSHOT.jar:/Users/prabhjyotsingh/spark-2.0.0-bin-hadoop2.7/conf/:/Users/prabhjyotsingh/spark-2.0.0-bin-hadoop2.7/jars/* -Xmx1g -Dfile.encoding=UTF-8 -Dlog4j.configuration=file:///Users/prabhjyotsingh/ps-zeppelin/conf/log4j.properties -Dzeppelin.log.file=/Users/prabhjyotsingh/ps-zeppelin/logs/zeppelin-interpreter-spark-user1-spark-prabhjyotsingh-HW11610.local.log org.apache.spark.deploy.SparkSubmit --conf spark.driver.extraClassPath=:/Users/prabhjyotsingh/ps-zeppelin/interpreter/spark/*:/Users/prabhjyotsingh/ps-zeppelin/zeppelin-interpreter/target/lib/*::/Users/prabhjyotsingh/ps-zeppelin/zeppelin-interpreter/target/classes:/Users/prabhjyotsingh/ps-zeppelin/zeppelin-interpreter/target/test-classes:/Users/prabhjyotsingh/ps-zeppelin/zeppelin-zengine/target/test-classes:/Users/prabhjyotsingh/ps-zeppelin/interpreter/spark/zeppelin-spark_2.10-0.8.0-SNAPSHOT.jar --conf spark.driver.extraJavaOptions= -Dfile.encoding=UTF-8 -Dlog4j.configuration=file:///Users/prabhjyotsingh/ps-zeppelin/conf/log4j.properties -Dzeppelin.log.file=/Users/prabhjyotsingh/ps-zeppelin/logs/zeppelin-interpreter-spark-user1-spark-prabhjyotsingh-HW11610.local.log --class org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer --proxy-user user1 /Users/prabhjyotsingh/ps-zeppelin/interpreter/spark/zeppelin-spark_2.10-0.8.0-SNAPSHOT.jar 50911
prabhjyotsingh    2508   0.0  0.0  2445100    860 s000  S+   12:08PM   0:00.00 grep spark
prabhjyotsingh    2495   0.0  0.0  2465144    764 s000  S    12:08PM   0:00.00 /bin/bash /Users/prabhjyotsingh/ps-zeppelin/bin/interpreter.sh -d /Users/prabhjyotsingh/ps-zeppelin/interpreter/spark -p 50911 -u user1 -l /Users/prabhjyotsingh/ps-zeppelin/local-repo/2CEZC4JXN -g spark
prabhjyotsingh    2484   0.0  0.0  2465144   1368 s000  S    12:08PM   0:00.01 /bin/bash /Users/prabhjyotsingh/ps-zeppelin/bin/interpreter.sh -d /Users/prabhjyotsingh/ps-zeppelin/interpreter/spark -p 50911 -u user1 -l /Users/prabhjyotsingh/ps-zeppelin/local-repo/2CEZC4JXN -g spark



On 10 May 2017 at 06:10, Yeshwanth Jagini <[hidden email]> wrote:
Hi Users,

I am trying to setup Zeppelin for multiple users.
and i found there are multiple configurations in different places. started tinkering with them and i didn't had any luck .
here's my setup and configuration.

Zeppelin server is running as root.

i edited zeppelin-env.sh  and uncommented 

export ZEPPELIN_IMPERSONATE_CMD='sudo -H -u ${ZEPPELIN_IMPERSONATE_USER} bash -c'       # Optional, when user want to run interpreter as end web user. eg) 'sudo -H -u ${ZEPPELIN_IMPERSONATE_USER} bash -c '
export ZEPPELIN_IMPERSONATE_SPARK_PROXY_USER=true  #Optional, by default is true; can be set to false if you don't want to use --proxy-user option with Spark interpreter when impersonation enabled

and  the interpreter settings are as follows

Inline image 1


for different combination of configurations i am getting different types of errors

if i do not specify impersonation configuration in zeppelin-env.sh  and specify impersonation in interpreter setting i am getting a org.apache.zeppelin.interpreter.InterpreterException: Host key verification failed.
if i specify both zeppelin impersonation configuration and interpreter impersonation config, it's throwing error as user1 cannot impersonate user1

if i do not specify any impersonation configuration at all , interpreter is launching spark-submit as root. that's expected.

could some one please explain me how to set impersonation config and   which configuration i am messing up here

 
Thanks,
Yeshwanth Jagini



--

Warm Regards, 

Prabhjyot Singh



--
Thanks,
Yeshwanth Jagini
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: User Impersonation Configuration

Prabhjyot Singh
From the exception, it looks like you are on 0.7.1. Which has the above-mentioned patch (https://github.com/apache/zeppelin/pull/1840).

Without ZEPPELIN_IMPERSONATE_CMD and ZEPPELIN_IMPERSONATE_SPARK_PROXY_USER set it should work just fine. However, if you want to setup password-less login, you can use this doc, its was written for Zeppelin-0.6.0 but it works fine https://community.hortonworks.com/content/kbentry/81069/how-to-enable-user-impersonation-for-sh-interprete.html.

Also, can you quickly check if you are able to connect to spark with user impersonation?


On 10 May 2017 at 23:05, Yeshwanth Jagini <[hidden email]> wrote:
Hi prabhjyot,
thanks for your reply.

i am using zeppelin 0.7.0 version.
when i do not specify impersonation config in zeppelin-env.sh and only in interpreter setting,
it is throwing following  exception

ERROR [2017-05-10 17:26:30,551] ({pool-2-thread-3} Job.java[run]:188) - Job failed
org.apache.zeppelin.interpreter.InterpreterException: Host key verification failed.

at org.apache.zeppelin.interpreter.remote.RemoteInterpreterManagedProcess.start(RemoteInterpreterManagedProcess.java:143)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.reference(RemoteInterpreterProcess.java:73)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.open(RemoteInterpreter.java:258)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:423)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:106)
at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:387)
at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:329)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:473)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)



i am running zeppelin as root user, root user doesn't had a password less ssh setup where as the end web user user1 has.

how should i proceed now?

Thanks,
Yeshwanth Jagini










On Wed, May 10, 2017 at 1:45 AM, Prabhjyot Singh <[hidden email]> wrote:
Hi Yeshwant,

Which version of Zeppelin are you on? 

If you are on latest then you don't need to do any of ZEPPELIN_IMPERSONATE_CMD or ZEPPELIN_IMPERSONATE_SPARK_PROXY_USER. Just by enabling User Impersonation check-box should be sufficient.

Can you confirm by `ps aux | grep spark`. This is what I see on my machine;

prabhjyotsingh@MACHINE:~/ps-zeppelin/logs$ ps aux | grep spark
prabhjyotsingh    2496   0.2  3.9  5179540 657660 s000  S    12:08PM   0:30.68 /Library/Java/JavaVirtualMachines/jdk1.8.0_102.jdk/Contents/Home/bin/java -cp /Users/prabhjyotsingh/ps-zeppelin/interpreter/spark/*:/Users/prabhjyotsingh/ps-zeppelin/zeppelin-interpreter/target/lib/*:/Users/prabhjyotsingh/ps-zeppelin/zeppelin-interpreter/target/classes/:/Users/prabhjyotsingh/ps-zeppelin/zeppelin-interpreter/target/test-classes/:/Users/prabhjyotsingh/ps-zeppelin/zeppelin-zengine/target/test-classes/:/Users/prabhjyotsingh/ps-zeppelin/interpreter/spark/zeppelin-spark_2.10-0.8.0-SNAPSHOT.jar:/Users/prabhjyotsingh/spark-2.0.0-bin-hadoop2.7/conf/:/Users/prabhjyotsingh/spark-2.0.0-bin-hadoop2.7/jars/* -Xmx1g -Dfile.encoding=UTF-8 -Dlog4j.configuration=file:///Users/prabhjyotsingh/ps-zeppelin/conf/log4j.properties -Dzeppelin.log.file=/Users/prabhjyotsingh/ps-zeppelin/logs/zeppelin-interpreter-spark-user1-spark-prabhjyotsingh-HW11610.local.log org.apache.spark.deploy.SparkSubmit --conf spark.driver.extraClassPath=:/Users/prabhjyotsingh/ps-zeppelin/interpreter/spark/*:/Users/prabhjyotsingh/ps-zeppelin/zeppelin-interpreter/target/lib/*::/Users/prabhjyotsingh/ps-zeppelin/zeppelin-interpreter/target/classes:/Users/prabhjyotsingh/ps-zeppelin/zeppelin-interpreter/target/test-classes:/Users/prabhjyotsingh/ps-zeppelin/zeppelin-zengine/target/test-classes:/Users/prabhjyotsingh/ps-zeppelin/interpreter/spark/zeppelin-spark_2.10-0.8.0-SNAPSHOT.jar --conf spark.driver.extraJavaOptions= -Dfile.encoding=UTF-8 -Dlog4j.configuration=file:///Users/prabhjyotsingh/ps-zeppelin/conf/log4j.properties -Dzeppelin.log.file=/Users/prabhjyotsingh/ps-zeppelin/logs/zeppelin-interpreter-spark-user1-spark-prabhjyotsingh-HW11610.local.log --class org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer --proxy-user user1 /Users/prabhjyotsingh/ps-zeppelin/interpreter/spark/zeppelin-spark_2.10-0.8.0-SNAPSHOT.jar 50911
prabhjyotsingh    2508   0.0  0.0  2445100    860 s000  S+   12:08PM   0:00.00 grep spark
prabhjyotsingh    2495   0.0  0.0  2465144    764 s000  S    12:08PM   0:00.00 /bin/bash /Users/prabhjyotsingh/ps-zeppelin/bin/interpreter.sh -d /Users/prabhjyotsingh/ps-zeppelin/interpreter/spark -p 50911 -u user1 -l /Users/prabhjyotsingh/ps-zeppelin/local-repo/2CEZC4JXN -g spark
prabhjyotsingh    2484   0.0  0.0  2465144   1368 s000  S    12:08PM   0:00.01 /bin/bash /Users/prabhjyotsingh/ps-zeppelin/bin/interpreter.sh -d /Users/prabhjyotsingh/ps-zeppelin/interpreter/spark -p 50911 -u user1 -l /Users/prabhjyotsingh/ps-zeppelin/local-repo/2CEZC4JXN -g spark



On 10 May 2017 at 06:10, Yeshwanth Jagini <[hidden email]> wrote:
Hi Users,

I am trying to setup Zeppelin for multiple users.
and i found there are multiple configurations in different places. started tinkering with them and i didn't had any luck .
here's my setup and configuration.

Zeppelin server is running as root.

i edited zeppelin-env.sh  and uncommented 

export ZEPPELIN_IMPERSONATE_CMD='sudo -H -u ${ZEPPELIN_IMPERSONATE_USER} bash -c'       # Optional, when user want to run interpreter as end web user. eg) 'sudo -H -u ${ZEPPELIN_IMPERSONATE_USER} bash -c '
export ZEPPELIN_IMPERSONATE_SPARK_PROXY_USER=true  #Optional, by default is true; can be set to false if you don't want to use --proxy-user option with Spark interpreter when impersonation enabled

and  the interpreter settings are as follows

Inline image 1


for different combination of configurations i am getting different types of errors

if i do not specify impersonation configuration in zeppelin-env.sh  and specify impersonation in interpreter setting i am getting a org.apache.zeppelin.interpreter.InterpreterException: Host key verification failed.
if i specify both zeppelin impersonation configuration and interpreter impersonation config, it's throwing error as user1 cannot impersonate user1

if i do not specify any impersonation configuration at all , interpreter is launching spark-submit as root. that's expected.

could some one please explain me how to set impersonation config and   which configuration i am messing up here

 
Thanks,
Yeshwanth Jagini



--

Warm Regards, 

Prabhjyot Singh



--
Thanks,
Yeshwanth Jagini



--
Thankx and Regards,

Prabhjyot Singh
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: User Impersonation Configuration

Prabhjyot Singh-2

On 11 May 2017 at 08:39, Prabhjyot Singh <[hidden email]> wrote:
Also, can you quickly check if you are able to connect to spark with user impersonation?

typo, can you quickly check if you are able to connect to spark without user impersonation?


--

Warm Regards, 

Prabhjyot Singh
Loading...