Not able to see hive metastore in Zeppelin

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Not able to see hive metastore in Zeppelin

Vadla, Karthik

Hi All,

 

Recently I have imported new databases into my hive meta store on cluster.

When I use %sql interpreter with below commands. I’m not able to see newly imported databases.

 

Command used:

 

%sql

 

show databases

 

It shows only default databases not the newly imported databases. But when I use hive command from command line

 

$hive –e “show databases”

 

It shows all the newly imported databases too.

 

 

Do anyone have clue on this, why I’m not able to see with zeppelin

Appreciate your help.

 

Thanks

Karthik Vadla

Reply | Threaded
Open this post in threaded view
|

Re: Not able to see hive metastore in Zeppelin

IT CTO
Can you access these databases? e.g. use %hive to list the DB and then use %sql to select from them?
I have the same behavior
Eran 

On Sun, Jun 28, 2015 at 8:21 AM Vadla, Karthik <[hidden email]> wrote:

Hi All,

 

Recently I have imported new databases into my hive meta store on cluster.

When I use %sql interpreter with below commands. I’m not able to see newly imported databases.

 

Command used:

 

%sql

 

show databases

 

It shows only default databases not the newly imported databases. But when I use hive command from command line

 

$hive –e “show databases”

 

It shows all the newly imported databases too.

 

 

Do anyone have clue on this, why I’m not able to see with zeppelin

Appreciate your help.

 

Thanks

Karthik Vadla

Reply | Threaded
Open this post in threaded view
|

Re: Not able to see hive metastore in Zeppelin

Brian McDevitt
If you don't have the hive-site.xml in your SPARK_HOME/conf directory, spark will create a new Hive metastore in the current directory.  See https://spark.apache.org/docs/1.3.1/sql-programming-guide.html#hive-tables

You can test this by running the spark-shell and trying to use sqlContext.sql("show databases").  When you quit the spark-shell, if there's a metastore directory created, then you don't have spark configured correctly for Hive.

HTH


Thanks,
Brian McDevitt
Software Engineer
The Nerdery

On Sun, Jun 28, 2015 at 1:22 AM, IT CTO <[hidden email]> wrote:
Can you access these databases? e.g. use %hive to list the DB and then use %sql to select from them?
I have the same behavior
Eran 

On Sun, Jun 28, 2015 at 8:21 AM Vadla, Karthik <[hidden email]> wrote:

Hi All,

 

Recently I have imported new databases into my hive meta store on cluster.

When I use %sql interpreter with below commands. I’m not able to see newly imported databases.

 

Command used:

 

%sql

 

show databases

 

It shows only default databases not the newly imported databases. But when I use hive command from command line

 

$hive –e “show databases”

 

It shows all the newly imported databases too.

 

 

Do anyone have clue on this, why I’m not able to see with zeppelin

Appreciate your help.

 

Thanks

Karthik Vadla


Reply | Threaded
Open this post in threaded view
|

Re: Not able to see hive metastore in Zeppelin

moon
Administrator
Hi,

Try either 
a) place your hive-site.xml in ZEPPELIN_HOME/conf dir, 
b) export HADOOP_CONF_DIR in ZEPPELIN_HOME/conf/zeppelin-env.sh to the directory that contains your hive-site.xml

Thanks,
moon



On Sun, Jun 28, 2015 at 6:12 AM Brian McDevitt <[hidden email]> wrote:
If you don't have the hive-site.xml in your SPARK_HOME/conf directory, spark will create a new Hive metastore in the current directory.  See https://spark.apache.org/docs/1.3.1/sql-programming-guide.html#hive-tables

You can test this by running the spark-shell and trying to use sqlContext.sql("show databases").  When you quit the spark-shell, if there's a metastore directory created, then you don't have spark configured correctly for Hive.

HTH


Thanks,
Brian McDevitt
Software Engineer
The Nerdery

On Sun, Jun 28, 2015 at 1:22 AM, IT CTO <[hidden email]> wrote:
Can you access these databases? e.g. use %hive to list the DB and then use %sql to select from them?
I have the same behavior
Eran 

On Sun, Jun 28, 2015 at 8:21 AM Vadla, Karthik <[hidden email]> wrote:

Hi All,

 

Recently I have imported new databases into my hive meta store on cluster.

When I use %sql interpreter with below commands. I’m not able to see newly imported databases.

 

Command used:

 

%sql

 

show databases

 

It shows only default databases not the newly imported databases. But when I use hive command from command line

 

$hive –e “show databases”

 

It shows all the newly imported databases too.

 

 

Do anyone have clue on this, why I’m not able to see with zeppelin

Appreciate your help.

 

Thanks

Karthik Vadla


Reply | Threaded
Open this post in threaded view
|

RE: Not able to see hive metastore in Zeppelin

Vadla, Karthik
In reply to this post by IT CTO

Thank you Eran, I did help me.

Now I’m able to see all databases.

 

Regards

Karthik

 

From: IT CTO [mailto:[hidden email]]
Sent: Saturday, June 27, 2015 11:22 PM
To: [hidden email]
Subject: Re: Not able to see hive metastore in Zeppelin

 

Can you access these databases? e.g. use %hive to list the DB and then use %sql to select from them?

I have the same behavior

Eran 

 

On Sun, Jun 28, 2015 at 8:21 AM Vadla, Karthik <[hidden email]> wrote:

Hi All,

 

Recently I have imported new databases into my hive meta store on cluster.

When I use %sql interpreter with below commands. I’m not able to see newly imported databases.

 

Command used:

 

%sql

 

show databases

 

It shows only default databases not the newly imported databases. But when I use hive command from command line

 

$hive –e “show databases”

 

It shows all the newly imported databases too.

 

 

Do anyone have clue on this, why I’m not able to see with zeppelin

Appreciate your help.

 

Thanks

Karthik Vadla

Reply | Threaded
Open this post in threaded view
|

Re: Not able to see hive metastore in Zeppelin

IT CTO
I just want to understand what is the expected behavior?
1) show tables works only using hive interpreter
2) %sql allow access directly to hive tables
3) %sql does not show tables when running "show tables"

Is that correct? this is what is happening on my configuration.

BTW, When using %SQL to access a hive table is that runs spark behind the scene? should I expect different performance when using spark code directly on hdfs files ves spark SQL through hive tables?

Eran  

On Mon, Jun 29, 2015 at 12:36 AM Vadla, Karthik <[hidden email]> wrote:

Thank you Eran, I did help me.

Now I’m able to see all databases.

 

Regards

Karthik

 

From: IT CTO [mailto:[hidden email]]
Sent: Saturday, June 27, 2015 11:22 PM
To: [hidden email]
Subject: Re: Not able to see hive metastore in Zeppelin

 

Can you access these databases? e.g. use %hive to list the DB and then use %sql to select from them?

I have the same behavior

Eran 

 

On Sun, Jun 28, 2015 at 8:21 AM Vadla, Karthik <[hidden email]> wrote:

Hi All,

 

Recently I have imported new databases into my hive meta store on cluster.

When I use %sql interpreter with below commands. I’m not able to see newly imported databases.

 

Command used:

 

%sql

 

show databases

 

It shows only default databases not the newly imported databases. But when I use hive command from command line

 

$hive –e “show databases”

 

It shows all the newly imported databases too.

 

 

Do anyone have clue on this, why I’m not able to see with zeppelin

Appreciate your help.

 

Thanks

Karthik Vadla

Reply | Threaded
Open this post in threaded view
|

RE: Not able to see hive metastore in Zeppelin

Vadla, Karthik

Hi Eran,

 

Even I’m looking for a clarify in this regard.

 

1.       In my one-node CDH 5.4 cluster, when I configured zeppelin. I’m able to access hive metastore with all hives commands using %sql (ex: show databases, show tables..etc)

2.       In my multi-node CDH 5.4 cluster, when I configured zeppelin. I’m able to access hive metastore only with %hive interpreter. Not able to access any hive tables using %sql interpreter. I really don’t have clue what is happening at backend.

3.       Yes, it uses spark but in local mode. I did not configure SPARK_HOME in zeppelin but yet it uses local spark.

 

I hope this helps. Please correct me if go wrong anywhere.

 

 

Thanks

Karthik

 

From: IT CTO [mailto:[hidden email]]
Sent: Sunday, June 28, 2015 11:51 PM
To: [hidden email]
Subject: Re: Not able to see hive metastore in Zeppelin

 

I just want to understand what is the expected behavior?

1) show tables works only using hive interpreter

2) %sql allow access directly to hive tables

3) %sql does not show tables when running "show tables"

 

Is that correct? this is what is happening on my configuration.

 

BTW, When using %SQL to access a hive table is that runs spark behind the scene? should I expect different performance when using spark code directly on hdfs files ves spark SQL through hive tables?

 

Eran  

 

On Mon, Jun 29, 2015 at 12:36 AM Vadla, Karthik <[hidden email]> wrote:

Thank you Eran, I did help me.

Now I’m able to see all databases.

 

Regards

Karthik

 

From: IT CTO [mailto:[hidden email]]
Sent: Saturday, June 27, 2015 11:22 PM
To: [hidden email]
Subject: Re: Not able to see hive metastore in Zeppelin

 

Can you access these databases? e.g. use %hive to list the DB and then use %sql to select from them?

I have the same behavior

Eran 

 

On Sun, Jun 28, 2015 at 8:21 AM Vadla, Karthik <[hidden email]> wrote:

Hi All,

 

Recently I have imported new databases into my hive meta store on cluster.

When I use %sql interpreter with below commands. I’m not able to see newly imported databases.

 

Command used:

 

%sql

 

show databases

 

It shows only default databases not the newly imported databases. But when I use hive command from command line

 

$hive –e “show databases”

 

It shows all the newly imported databases too.

 

 

Do anyone have clue on this, why I’m not able to see with zeppelin

Appreciate your help.

 

Thanks

Karthik Vadla