Not able to see registered table records

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Not able to see registered table records

Vadla, Karthik

Hi All.

 

I just registered a tables using below code

 

val eduText = sc.textFile("hdfs://ip.address/user/karthik/education.csv")

 

case class Education(unitid:Integer, instnm:String, addr : String, city : String, stabbr : String, zip : Integer)

 

val education = eduText.map(s=>s.split(",")).filter(s=>s(0)!="UNITID").map(

    s=>Education(s(0).toInt,

            s(1).replaceAll("\"", ""),

            s(2).replaceAll("\"", ""),

            s(3).replaceAll("\"", ""),

            s(4).replaceAll("\"", ""),

            s(5).replaceAll("\"", "").toInt

        )

)

 

// Below line works only in spark 1.3.0.

// For spark 1.1.x and spark 1.2.x,

// use bank.registerTempTable("bank") instead.

 

education.toDF().registerTempTable("education")

 

when I run “%sql show tables”

 

It displays table “education”

 

But when I try to run the command “%sql select count(*) from education”.  It is throwing below error.

 

java.lang.reflect.InvocationTargetException

 

 

 

Can anyone help me with this.

Appreciate your help.

 

And I enclosed .csv file used to register table.

 

Thanks

Karthik


eucation.csv (5M) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Not able to see registered table records

moon
Administrator
Hi,


It'll give you better stacktrace than just displaying "java.lang.reflect.InvocationTargetException"

Thanks,
moon


On Thu, Jul 2, 2015 at 10:34 AM Vadla, Karthik <[hidden email]> wrote:

Hi All.

 

I just registered a tables using below code

 

val eduText = sc.textFile("hdfs://ip.address/user/karthik/education.csv")

 

case class Education(unitid:Integer, instnm:String, addr : String, city : String, stabbr : String, zip : Integer)

 

val education = eduText.map(s=>s.split(",")).filter(s=>s(0)!="UNITID").map(

    s=>Education(s(0).toInt,

            s(1).replaceAll("\"", ""),

            s(2).replaceAll("\"", ""),

            s(3).replaceAll("\"", ""),

            s(4).replaceAll("\"", ""),

            s(5).replaceAll("\"", "").toInt

        )

)

 

// Below line works only in spark 1.3.0.

// For spark 1.1.x and spark 1.2.x,

// use bank.registerTempTable("bank") instead.

 

education.toDF().registerTempTable("education")

 

when I run “%sql show tables”

 

It displays table “education”

 

But when I try to run the command “%sql select count(*) from education”.  It is throwing below error.

 

java.lang.reflect.InvocationTargetException

 

 

 

Can anyone help me with this.

Appreciate your help.

 

And I enclosed .csv file used to register table.

 

Thanks

Karthik