Zeppelin 0.7.1,config file not being read

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Zeppelin 0.7.1,config file not being read

kebarcla
Hello,

I'm running Zep 0.7.1 on the new EMR 5.5 on AWS: I have all the usual big data tools deployed. I'd like to set up users to access the cluster.

In following the steps documented in 'Apache Shiro authentication for Apache Zeppelin' I'm finding that my changes to conf/shiro.ini or zeppelin-site.xml on the master node seem to be ignored: I don't see them under 'Configuration' on the user drop-down.

After restarting Zeppelin server, I'm still automatically logged in as 'anonymous.'
How does one log out, and log back in as another user?
How to get Zeppelin server to recognize config file changes?

Thanks for any pointers,
Ken

# shiro.ini changes:

< newuser = Password!, admin
< #/api/version = anon
< /api/interpreter/** = authc, roles[admin]
< /api/configurations/** = authc, roles[admin]
< /api/credential/** = authc, roles[admin]
---
> /api/version = anon
> #/api/interpreter/** = authc, roles[admin]
> #/api/configurations/** = authc, roles[admin]
> #/api/credential/** = authc, roles[admin]
Reply | Threaded
Open this post in threaded view
|

Re: Zeppelin 0.7.1,config file not being read

Jaideep Singh
Hello,

I am also new to this but you can try changing
> /api/version = authc,roles[admin]

Thanks,
Jaideep Singh



On Saturday, May 27, 2017, kebarcla <[hidden email]> wrote:
Hello,

I'm running Zep 0.7.1 on the new EMR 5.5 on AWS: I have all the usual big
data tools deployed. I'd like to set up users to access the cluster.

In following the steps documented in 'Apache Shiro authentication for Apache
Zeppelin' I'm finding that my changes to conf/shiro.ini or zeppelin-site.xml
on the master node seem to be ignored: I don't see them under
'Configuration' on the user drop-down.

After restarting Zeppelin server, I'm still automatically logged in as
'anonymous.'
How does one log out, and log back in as another user?
How to get Zeppelin server to recognize config file changes?

Thanks for any pointers,
Ken

# shiro.ini changes:

< newuser = Password!, admin
< #/api/version = anon
< /api/interpreter/** = authc, roles[admin]
< /api/configurations/** = authc, roles[admin]
< /api/credential/** = authc, roles[admin]
---
> /api/version = anon
> #/api/interpreter/** = authc, roles[admin]
> #/api/configurations/** = authc, roles[admin]
> #/api/credential/** = authc, roles[admin]



--
View this message in context: http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/Zeppelin-0-7-1-config-file-not-being-read-tp5695.html
Sent from the Apache Zeppelin Users (incubating) mailing list mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|

Re: Zeppelin 0.7.1,config file not being read

Jonathan
I have not tried using Shiro with Zeppelin yet, but one thing I'll point out is that for Zeppelin on EMR, configuration should be done through zeppelin-env.sh rather than zeppelin-site.xml. All Zeppelin configuration properties can be set via either file, so we (I'm on the EMR team, btw) decided to support just one of them. At least, zeppelin-env.sh is the only one you can change via the Configuration JSON you may supply when creating a cluster. Nothing stops you from ssh'ing to the master node and changing the files yourself.

~ Jonathan
On Fri, May 26, 2017 at 6:15 PM Jaideep Singh <[hidden email]> wrote:
Hello,

I am also new to this but you can try changing
> /api/version = authc,roles[admin]

Thanks,
Jaideep Singh




On Saturday, May 27, 2017, kebarcla <[hidden email]> wrote:
Hello,

I'm running Zep 0.7.1 on the new EMR 5.5 on AWS: I have all the usual big
data tools deployed. I'd like to set up users to access the cluster.

In following the steps documented in 'Apache Shiro authentication for Apache
Zeppelin' I'm finding that my changes to conf/shiro.ini or zeppelin-site.xml
on the master node seem to be ignored: I don't see them under
'Configuration' on the user drop-down.

After restarting Zeppelin server, I'm still automatically logged in as
'anonymous.'
How does one log out, and log back in as another user?
How to get Zeppelin server to recognize config file changes?

Thanks for any pointers,
Ken

# shiro.ini changes:

< newuser = Password!, admin
< #/api/version = anon
< /api/interpreter/** = authc, roles[admin]
< /api/configurations/** = authc, roles[admin]
< /api/credential/** = authc, roles[admin]
---
> /api/version = anon
> #/api/interpreter/** = authc, roles[admin]
> #/api/configurations/** = authc, roles[admin]
> #/api/credential/** = authc, roles[admin]



--
View this message in context: http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/Zeppelin-0-7-1-config-file-not-being-read-tp5695.html
Sent from the Apache Zeppelin Users (incubating) mailing list mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|

Re: Zeppelin 0.7.1,config file not being read

kebarcla
In reply to this post by Jaideep Singh

Seems logical – I gave it a shot, but it didn’t affect the outcome. I seem to have some fundamental disconnect between my configuration & my running instance. I’m even passing the config directory as an argument:

 

/usr/lib/zeppelin/bin/zeppelin-daemon.sh --config /etc/zeppelin/conf start

 

Thanks though Jaideep

 

From: Jaideep Singh <[hidden email]>
Reply-To: "[hidden email]" <[hidden email]>
Date: Friday, May 26, 2017 at 6:15 PM
To: "[hidden email]" <[hidden email]>
Cc: "[hidden email]" <[hidden email]>
Subject: Re: Zeppelin 0.7.1,config file not being read

 

Hello,

 

I am also new to this but you can try changing

> /api/version = authc,roles[admin]

Thanks,

Jaideep Singh



On Saturday, May 27, 2017, kebarcla <[hidden email]> wrote:

Hello,

I'm running Zep 0.7.1 on the new EMR 5.5 on AWS: I have all the usual big
data tools deployed. I'd like to set up users to access the cluster.

In following the steps documented in 'Apache Shiro authentication for Apache
Zeppelin' I'm finding that my changes to conf/shiro.ini or zeppelin-site.xml
on the master node seem to be ignored: I don't see them under
'Configuration' on the user drop-down.

After restarting Zeppelin server, I'm still automatically logged in as
'anonymous.'
How does one log out, and log back in as another user?
How to get Zeppelin server to recognize config file changes?

Thanks for any pointers,
Ken

# shiro.ini changes:

< newuser = Password!, admin
< #/api/version = anon
< /api/interpreter/** = authc, roles[admin]
< /api/configurations/** = authc, roles[admin]
< /api/credential/** = authc, roles[admin]
---
> /api/version = anon
> #/api/interpreter/** = authc, roles[admin]
> #/api/configurations/** = authc, roles[admin]
> #/api/credential/** = authc, roles[admin]



--
View this message in context: http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/Zeppelin-0-7-1-config-file-not-being-read-tp5695.html
Sent from the Apache Zeppelin Users (incubating) mailing list mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|

Re: Zeppelin 0.7.1,config file not being read

kebarcla
In reply to this post by Jonathan

You seem like right person to talk  to Jonathan!

 

Something as close as possible to SSO for users of our EMR cluster who use Spark & Hive is what we’re seeking. EMR is so powerful & easy to configure that creating a cluster of any size and type is trivial. Now we need to be provide data scientists a means to use it, and notebook environments like Jupyter and Zeppelin seem ideal. So this is what we’re trying first.

 

I initially tried to use Hue as my interface, and successfully integrated it with LDAP so that I could authenticate users and groups in  the UI – beautiful. And I can navigate to an S3 bucket from Hue where I have Zep notebooks stored, but couldn’t figure out how to then bring them up in  Zeppelin. If I could do that my problem would be solved, because I could have users log in as themselves, then bring up their notebooks from a common bucket.

 

Next thought was to forget about LDAP and just see if I could configure some users in Zep directly via shiro. To that end I logged onto the master node and added this to shiro.ini:

 

/api/version = authc,roles[admin]

/api/interpreter/** = authc, roles[admin]

/api/configurations/** = authc, roles[admin]

/api/credential/** = authc, roles[admin]

 

Then created a local user for myself with adduser and put this in zeppelin-env.sh:

 

export ZEPPELIN_NOTEBOOK_S3_BUCKET=zeppelin

export ZEPPELIN_NOTEBOOK_S3_USER=kbarclay

export ZEPPELIN_NOTEBOOK_STORAGE=org.apache.zeppelin.notebook.repo.S3NotebookRepo

export ZEPPELIN_NOTEBOOK_PUBLIC=true

 

and I changed one property in zeppelin-site.xml, because according to https://zeppelin.apache.org/docs/0.7.1/install/configuration.html, it cannot be changed in zeppelin-env.sh:

 

<property>

  <name>zeppelin.anonymous.allowed</name>

  <value>false</value>

  <description>Anonymous user allowed by default</description>

</property>

 

But after restarting Zeppelin server, it automatically makes me user ‘anonymous’ still, and offers no means of logging out and then back in as another user.

 

Interestingly, if I put in one of our real bucket names for ZEPPELIN_NOTEBOOK_S3_BUCKET, I get an error in the log file  that says it can’t read it and that it’s ignoring the config file and using defaults. So it *is* reading the config file. If I set it back to the default (zeppelin), I don’t get a message about an invalid config file, but I also don’t see any of my configuration changes in the Configuration pull-down (like ZEPPELIN_NOTEBOOK_STORAGE) – everything is set to defaults.

 

I’m going to look at the ‘Notebook Storage in S3’ steps next and set up credentials – maybe I need  ~/.aws/credentials on the master node.

 

If you know anyone who’s set up a notebook environment in some multitenant fashion to interact with EMR, please let me know!

 

Thanks for your help

-Ken

 

 

 

From: Jonathan Kelly <[hidden email]>
Reply-To: "[hidden email]" <[hidden email]>
Date: Friday, May 26, 2017 at 6:56 PM
To: "[hidden email]" <[hidden email]>
Cc: "[hidden email]" <[hidden email]>
Subject: Re: Zeppelin 0.7.1,config file not being read

 

I have not tried using Shiro with Zeppelin yet, but one thing I'll point out is that for Zeppelin on EMR, configuration should be done through zeppelin-env.sh rather than zeppelin-site.xml. All Zeppelin configuration properties can be set via either file, so we (I'm on the EMR team, btw) decided to support just one of them. At least, zeppelin-env.sh is the only one you can change via the Configuration JSON you may supply when creating a cluster. Nothing stops you from ssh'ing to the master node and changing the files yourself.

~ Jonathan

On Fri, May 26, 2017 at 6:15 PM Jaideep Singh <[hidden email]> wrote:

Hello,

 

I am also new to this but you can try changing

> /api/version = authc,roles[admin]

Thanks,

Jaideep Singh

 



On Saturday, May 27, 2017, kebarcla <[hidden email]> wrote:

Hello,

I'm running Zep 0.7.1 on the new EMR 5.5 on AWS: I have all the usual big
data tools deployed. I'd like to set up users to access the cluster.

In following the steps documented in 'Apache Shiro authentication for Apache
Zeppelin' I'm finding that my changes to conf/shiro.ini or zeppelin-site.xml
on the master node seem to be ignored: I don't see them under
'Configuration' on the user drop-down.

After restarting Zeppelin server, I'm still automatically logged in as
'anonymous.'
How does one log out, and log back in as another user?
How to get Zeppelin server to recognize config file changes?

Thanks for any pointers,
Ken

# shiro.ini changes:

< newuser = Password!, admin
< #/api/version = anon
< /api/interpreter/** = authc, roles[admin]
< /api/configurations/** = authc, roles[admin]
< /api/credential/** = authc, roles[admin]
---
> /api/version = anon
> #/api/interpreter/** = authc, roles[admin]
> #/api/configurations/** = authc, roles[admin]
> #/api/credential/** = authc, roles[admin]



--
View this message in context: http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/Zeppelin-0-7-1-config-file-not-being-read-tp5695.html
Sent from the Apache Zeppelin Users (incubating) mailing list mailing list archive at Nabble.com.