Question on converting pandas dataframe to spark frame

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Question on converting pandas dataframe to spark frame

Balachandar R.A.

Hello

I am new to zeppelin. Quite interested to make some progress in my usecase with the help of zeppelin capabilities. I have posted a question in stack overflow. I guess this is the right forum to ask
Hence sharing the question

http://stackoverflow.com/questions/32966344/converting-pandas-dataframes-to-spark-dataframe-in-zeppelin

Will be more than happy if I could get help

Thanks
Bala

Reply | Threaded
Open this post in threaded view
|

Re: Question on converting pandas dataframe to spark frame

moon
Administrator
Hi Bala,

Thanks for sharing the problem.
I reproduced the same error, the same code works in bin/pyspark but not in Zeppelin.
I'm going to take a look and keep you updated here.

Thanks,
moon

On Wed, Oct 7, 2015 at 6:17 AM Balachandar R.A. <[hidden email]> wrote:

Hello

I am new to zeppelin. Quite interested to make some progress in my usecase with the help of zeppelin capabilities. I have posted a question in stack overflow. I guess this is the right forum to ask
Hence sharing the question

http://stackoverflow.com/questions/32966344/converting-pandas-dataframes-to-spark-dataframe-in-zeppelin

Will be more than happy if I could get help

Thanks
Bala

Reply | Threaded
Open this post in threaded view
|

Re: Question on converting pandas dataframe to spark frame

Balachandar R.A.

Thanks for your response.  Looking forward for any update.

Thanks
Bala

On 13-Oct-2015 1:51 pm, "moon soo Lee" <[hidden email]> wrote:
Hi Bala,

Thanks for sharing the problem.
I reproduced the same error, the same code works in bin/pyspark but not in Zeppelin.
I'm going to take a look and keep you updated here.

Thanks,
moon

On Wed, Oct 7, 2015 at 6:17 AM Balachandar R.A. <[hidden email]> wrote:

Hello

I am new to zeppelin. Quite interested to make some progress in my usecase with the help of zeppelin capabilities. I have posted a question in stack overflow. I guess this is the right forum to ask
Hence sharing the question

http://stackoverflow.com/questions/32966344/converting-pandas-dataframes-to-spark-dataframe-in-zeppelin

Will be more than happy if I could get help

Thanks
Bala

Reply | Threaded
Open this post in threaded view
|

RE: Question on converting pandas dataframe to spark frame

Felix Cheung
In reply to this post by moon
let me know if I could help...

 

From: [hidden email]
Date: Tue, 13 Oct 2015 08:20:52 +0000
Subject: Re: Question on converting pandas dataframe to spark frame
To: [hidden email]

Hi Bala,

Thanks for sharing the problem.
I reproduced the same error, the same code works in bin/pyspark but not in Zeppelin.
I'm going to take a look and keep you updated here.

Thanks,
moon

On Wed, Oct 7, 2015 at 6:17 AM Balachandar R.A. <[hidden email]> wrote:

Hello

I am new to zeppelin. Quite interested to make some progress in my usecase with the help of zeppelin capabilities. I have posted a question in stack overflow. I guess this is the right forum to ask
Hence sharing the question

http://stackoverflow.com/questions/32966344/converting-pandas-dataframes-to-spark-dataframe-in-zeppelin

Will be more than happy if I could get help

Thanks
Bala

Reply | Threaded
Open this post in threaded view
|

Re: Question on converting pandas dataframe to spark frame

moon
Administrator
Somehow, I can not reproduce the problem anymore. 
Can anyone reproduce this problem?

I tried with Spark 1.4.1, 1.5.1 local mode / standalone cluster mode. python 2.6.6, 2.7.6, with latest master branch of Zeppelin.

Thanks,
moon

On Wed, Oct 14, 2015 at 7:01 AM Felix Cheung <[hidden email]> wrote:
let me know if I could help...

 

From: [hidden email]
Date: Tue, 13 Oct 2015 08:20:52 +0000
Subject: Re: Question on converting pandas dataframe to spark frame
To: [hidden email]


Hi Bala,

Thanks for sharing the problem.
I reproduced the same error, the same code works in bin/pyspark but not in Zeppelin.
I'm going to take a look and keep you updated here.

Thanks,
moon

On Wed, Oct 7, 2015 at 6:17 AM Balachandar R.A. <[hidden email]> wrote:

Hello

I am new to zeppelin. Quite interested to make some progress in my usecase with the help of zeppelin capabilities. I have posted a question in stack overflow. I guess this is the right forum to ask
Hence sharing the question

http://stackoverflow.com/questions/32966344/converting-pandas-dataframes-to-spark-dataframe-in-zeppelin

Will be more than happy if I could get help

Thanks
Bala