Zepplin 0.7.3 DEBUG console output, how do I switch it off!!! Spark 2.2.0-bin-hadoop2.7

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Zepplin 0.7.3 DEBUG console output, how do I switch it off!!! Spark 2.2.0-bin-hadoop2.7

amersheik@gmail.com
Hello there

I have installed Spark 2.20 and from the spark-shell, my scala code works perfectly and performs well.

The same scala executed from Zeppelin causes lots of DEBUG statements to be printed in the console (almost non-stop) and the performance is very slow.

I have just installed binaries and haven't changed any configurations, but it seems Zeppelin is running in debug mode...

Help..

I use bin\zeppelin.cmd to start zeppelin. I'm on Windows Server 2012

Please help...


Reply | Threaded
Open this post in threaded view
|

Re: Zepplin 0.7.3 DEBUG console output, how do I switch it off!!! Spark 2.2.0-bin-hadoop2.7

amersheik@gmail.com


On 2017-10-12 10:41, "[hidden email]"<[hidden email]> wrote:

> Hello there
>
> I have installed Spark 2.20 and from the spark-shell, my scala code works perfectly and performs well.
>
> The same scala executed from Zeppelin causes lots of DEBUG statements to be printed in the console (almost non-stop) and the performance is very slow.
>
> I have just installed binaries and haven't changed any configurations, but it seems Zeppelin is running in debug mode...
>
> Help..
>
> I use bin\zeppelin.cmd to start zeppelin. I'm on Windows Server 2012
>
> Please help...
>
>
> PS

The scala works perfectly within Zeppelin... Just very slow performance and lots of Debug output...
Reply | Threaded
Open this post in threaded view
|

Re: Zepplin 0.7.3 DEBUG console output, how do I switch it off!!! Spark 2.2.0-bin-hadoop2.7

Jeff Zhang

What kind of DEBUG log you see ? Can you paste it ?


[hidden email] <[hidden email]>于2017年10月12日周四 下午5:44写道:


On 2017-10-12 10:41, "[hidden email]"<[hidden email]> wrote:
> Hello there
>
> I have installed Spark 2.20 and from the spark-shell, my scala code works perfectly and performs well.
>
> The same scala executed from Zeppelin causes lots of DEBUG statements to be printed in the console (almost non-stop) and the performance is very slow.
>
> I have just installed binaries and haven't changed any configurations, but it seems Zeppelin is running in debug mode...
>
> Help..
>
> I use bin\zeppelin.cmd to start zeppelin. I'm on Windows Server 2012
>
> Please help...
>
>
> PS

The scala works perfectly within Zeppelin... Just very slow performance and lots of Debug output...
Reply | Threaded
Open this post in threaded view
|

Re: Zepplin 0.7.3 DEBUG console output, how do I switch it off!!! Spark 2.2.0-bin-hadoop2.7

amersheik@gmail.com
In reply to this post by amersheik@gmail.com
Hi there

Please see the console output



On Thu, Oct 12, 2017 at 1:25 PM, Jeff Zhang <[hidden email]> wrote:

What kind of DEBUG log you see ? Can you paste it ?


[hidden email] <[hidden email]>于2017年10月12日周四 下午5:44写道:


On 2017-10-12 10:41, "[hidden email]"<[hidden email]> wrote:
> Hello there
>
> I have installed Spark 2.20 and from the spark-shell, my scala code works perfectly and performs well.
>
> The same scala executed from Zeppelin causes lots of DEBUG statements to be printed in the console (almost non-stop) and the performance is very slow.
>
> I have just installed binaries and haven't changed any configurations, but it seems Zeppelin is running in debug mode...
>
> Help..
>
> I use bin\zeppelin.cmd to start zeppelin. I'm on Windows Server 2012
>
> Please help...
>
>
> PS

The scala works perfectly within Zeppelin... Just very slow performance and lots of Debug output...


Screen Shot 2017-10-13 at 5.13.56 PM.png (127K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Zepplin 0.7.3 DEBUG console output, how do I switch it off!!! Spark 2.2.0-bin-hadoop2.7

Jianfeng (Jeff) Zhang

These are zeppelin server log which is controlled by log4.properties under ZEPPELIN_CONF_DIR. And I don’t understand why would you say these log would affect the performance.

Could you explain more ?




Best Regard,
Jeff Zhang


From: Amer Sheikh <[hidden email]>
Reply-To: "[hidden email]" <[hidden email]>
Date: Saturday, October 14, 2017 at 12:15 AM
To: "[hidden email]" <[hidden email]>
Subject: Re: Zepplin 0.7.3 DEBUG console output, how do I switch it off!!! Spark 2.2.0-bin-hadoop2.7

Hi there

Please see the console output



On Thu, Oct 12, 2017 at 1:25 PM, Jeff Zhang <[hidden email]> wrote:

What kind of DEBUG log you see ? Can you paste it ?


[hidden email] <[hidden email]>于2017年10月12日周四 下午5:44写道:


On 2017-10-12 10:41, "[hidden email]"<[hidden email]> wrote:
> Hello there
>
> I have installed Spark 2.20 and from the spark-shell, my scala code works perfectly and performs well.
>
> The same scala executed from Zeppelin causes lots of DEBUG statements to be printed in the console (almost non-stop) and the performance is very slow.
>
> I have just installed binaries and haven't changed any configurations, but it seems Zeppelin is running in debug mode...
>
> Help..
>
> I use bin\zeppelin.cmd to start zeppelin. I'm on Windows Server 2012
>
> Please help...
>
>
> PS

The scala works perfectly within Zeppelin... Just very slow performance and lots of Debug output...

Reply | Threaded
Open this post in threaded view
|

Re: Zepplin 0.7.3 DEBUG console output, how do I switch it off!!! Spark 2.2.0-bin-hadoop2.7

amersheik@gmail.com
Hi Jeff

Thanks for your reply. The zeppelin console output is continuous when my scala code (spark sql) is running.
The same code runs very from a separate spark-shell, so im suspecting that Zeppelin is executing the code in DEBUG mode.

When I installed Zeppelin, I did not change any settings.  My main concern is the very slow performance.

Kind Regards

Amer

On Sat, Oct 14, 2017 at 2:16 AM, Jianfeng (Jeff) Zhang <[hidden email]> wrote:

These are zeppelin server log which is controlled by log4.properties under ZEPPELIN_CONF_DIR. And I don’t understand why would you say these log would affect the performance.

Could you explain more ?




Best Regard,
Jeff Zhang


From: Amer Sheikh <[hidden email]>
Reply-To: "[hidden email]" <[hidden email]>
Date: Saturday, October 14, 2017 at 12:15 AM
To: "[hidden email]" <[hidden email]>
Subject: Re: Zepplin 0.7.3 DEBUG console output, how do I switch it off!!! Spark 2.2.0-bin-hadoop2.7

Hi there

Please see the console output



On Thu, Oct 12, 2017 at 1:25 PM, Jeff Zhang <[hidden email]> wrote:

What kind of DEBUG log you see ? Can you paste it ?


[hidden email] <[hidden email]>于2017年10月12日周四 下午5:44写道:


On 2017-10-12 10:41, "[hidden email]"<[hidden email]> wrote:
> Hello there
>
> I have installed Spark 2.20 and from the spark-shell, my scala code works perfectly and performs well.
>
> The same scala executed from Zeppelin causes lots of DEBUG statements to be printed in the console (almost non-stop) and the performance is very slow.
>
> I have just installed binaries and haven't changed any configurations, but it seems Zeppelin is running in debug mode...
>
> Help..
>
> I use bin\zeppelin.cmd to start zeppelin. I'm on Windows Server 2012
>
> Please help...
>
>
> PS

The scala works perfectly within Zeppelin... Just very slow performance and lots of Debug output...


Reply | Threaded
Open this post in threaded view
|

Re: Zepplin 0.7.3 DEBUG console output, how do I switch it off!!! Spark 2.2.0-bin-hadoop2.7

Jeff Zhang
I just suspect it is due to different spark configuration in zeppelin spark interpreter and spark-shell. Please make sure they are using the same configuration.


Amer Sheikh <[hidden email]>于2017年10月14日周六 下午1:44写道:
Hi Jeff

Thanks for your reply. The zeppelin console output is continuous when my scala code (spark sql) is running.
The same code runs very from a separate spark-shell, so im suspecting that Zeppelin is executing the code in DEBUG mode.

When I installed Zeppelin, I did not change any settings.  My main concern is the very slow performance.

Kind Regards

Amer

On Sat, Oct 14, 2017 at 2:16 AM, Jianfeng (Jeff) Zhang <[hidden email]> wrote:

These are zeppelin server log which is controlled by log4.properties under ZEPPELIN_CONF_DIR. And I don’t understand why would you say these log would affect the performance.

Could you explain more ?




Best Regard,
Jeff Zhang


From: Amer Sheikh <[hidden email]>
Reply-To: "[hidden email]" <[hidden email]>
Date: Saturday, October 14, 2017 at 12:15 AM
To: "[hidden email]" <[hidden email]>
Subject: Re: Zepplin 0.7.3 DEBUG console output, how do I switch it off!!! Spark 2.2.0-bin-hadoop2.7

Hi there

Please see the console output



On Thu, Oct 12, 2017 at 1:25 PM, Jeff Zhang <[hidden email]> wrote:

What kind of DEBUG log you see ? Can you paste it ?


[hidden email] <[hidden email]>于2017年10月12日周四 下午5:44写道:


On 2017-10-12 10:41, "[hidden email]"<[hidden email]> wrote:
> Hello there
>
> I have installed Spark 2.20 and from the spark-shell, my scala code works perfectly and performs well.
>
> The same scala executed from Zeppelin causes lots of DEBUG statements to be printed in the console (almost non-stop) and the performance is very slow.
>
> I have just installed binaries and haven't changed any configurations, but it seems Zeppelin is running in debug mode...
>
> Help..
>
> I use bin\zeppelin.cmd to start zeppelin. I'm on Windows Server 2012
>
> Please help...
>
>
> PS

The scala works perfectly within Zeppelin... Just very slow performance and lots of Debug output...


Reply | Threaded
Open this post in threaded view
|

Re: Zepplin 0.7.3 DEBUG console output, how do I switch it off!!! Spark 2.2.0-bin-hadoop2.7

amersheik@gmail.com


On 2017-10-14 06:48, Jeff Zhang <[hidden email]> wrote:

> I just suspect it is due to different spark configuration in zeppelin spark
> interpreter and spark-shell. Please make sure they are using the same
> configuration.
>
>
> Amer Sheikh <[hidden email]>于2017年10月14日周六 下午1:44写道:
>
> > Hi Jeff
> >
> > Thanks for your reply. The zeppelin console output is continuous when my
> > scala code (spark sql) is running.
> > The same code runs very from a separate spark-shell, so im suspecting that
> > Zeppelin is executing the code in DEBUG mode.
> >
> > When I installed Zeppelin, I did not change any settings.  My main concern
> > is the very slow performance.
> >
> > Kind Regards
> >
> > Amer
> >
> > On Sat, Oct 14, 2017 at 2:16 AM, Jianfeng (Jeff) Zhang <
> > [hidden email]> wrote:
> >
> >>
> >> These are zeppelin server log which is controlled by log4.properties
> >> under ZEPPELIN_CONF_DIR. And I don’t understand why would you say these
> >> log would affect the performance.
> >>
> >> Could you explain more ?
> >>
> >>
> >>
> >>
> >> Best Regard,
> >> Jeff Zhang
> >>
> >>
> >> From: Amer Sheikh <[hidden email]>
> >> Reply-To: "[hidden email]" <[hidden email]>
> >> Date: Saturday, October 14, 2017 at 12:15 AM
> >> To: "[hidden email]" <[hidden email]>
> >> Subject: Re: Zepplin 0.7.3 DEBUG console output, how do I switch it
> >> off!!! Spark 2.2.0-bin-hadoop2.7
> >>
> >> Hi there
> >>
> >> Please see the console output
> >>
> >>
> >>
> >> On Thu, Oct 12, 2017 at 1:25 PM, Jeff Zhang <[hidden email]> wrote:
> >>
> >>>
> >>> What kind of DEBUG log you see ? Can you paste it ?
> >>>
> >>>
> >>> [hidden email] <[hidden email]>于2017年10月12日周四 下午5:44写道:
> >>>
> >>>>
> >>>>
> >>>> On 2017-10-12 10:41, "[hidden email]"<[hidden email]> wrote:
> >>>> > Hello there
> >>>> >
> >>>> > I have installed Spark 2.20 and from the spark-shell, my scala code
> >>>> works perfectly and performs well.
> >>>> >
> >>>> > The same scala executed from Zeppelin causes lots of DEBUG statements
> >>>> to be printed in the console (almost non-stop) and the performance is very
> >>>> slow.
> >>>> >
> >>>> > I have just installed binaries and haven't changed any
> >>>> configurations, but it seems Zeppelin is running in debug mode...
> >>>> >
> >>>> > Help..
> >>>> >
> >>>> > I use bin\zeppelin.cmd to start zeppelin. I'm on Windows Server 2012
> >>>> >
> >>>> > Please help...
> >>>> >
> >>>> >
> >>>> > PS
> >>>>
> >>>> The scala works perfectly within Zeppelin... Just very slow performance
> >>>> and lots of Debug output...
> >>>>
> >>>
> >>
> >
>  Hi there. Since the same spark sql is 3 times SLOWER on ZEPPELIN, how do i ensure that a query through ZEPPELIN uses the same as that in SPARK-SHELL. Please help
Reply | Threaded
Open this post in threaded view
|

Re: Zepplin 0.7.3 DEBUG console output, how do I switch it off!!! Spark 2.2.0-bin-hadoop2.7

Jeff Zhang

Check the spark interpreter setting

[hidden email] <[hidden email]>于2017年10月16日周一 下午5:07写道:


On 2017-10-14 06:48, Jeff Zhang <[hidden email]> wrote:
> I just suspect it is due to different spark configuration in zeppelin spark
> interpreter and spark-shell. Please make sure they are using the same
> configuration.
>
>
> Amer Sheikh <[hidden email]>于2017年10月14日周六 下午1:44写道:
>
> > Hi Jeff
> >
> > Thanks for your reply. The zeppelin console output is continuous when my
> > scala code (spark sql) is running.
> > The same code runs very from a separate spark-shell, so im suspecting that
> > Zeppelin is executing the code in DEBUG mode.
> >
> > When I installed Zeppelin, I did not change any settings.  My main concern
> > is the very slow performance.
> >
> > Kind Regards
> >
> > Amer
> >
> > On Sat, Oct 14, 2017 at 2:16 AM, Jianfeng (Jeff) Zhang <
> > [hidden email]> wrote:
> >
> >>
> >> These are zeppelin server log which is controlled by log4.properties
> >> under ZEPPELIN_CONF_DIR. And I don’t understand why would you say these
> >> log would affect the performance.
> >>
> >> Could you explain more ?
> >>
> >>
> >>
> >>
> >> Best Regard,
> >> Jeff Zhang
> >>
> >>
> >> From: Amer Sheikh <[hidden email]>
> >> Reply-To: "[hidden email]" <[hidden email]>
> >> Date: Saturday, October 14, 2017 at 12:15 AM
> >> To: "[hidden email]" <[hidden email]>
> >> Subject: Re: Zepplin 0.7.3 DEBUG console output, how do I switch it
> >> off!!! Spark 2.2.0-bin-hadoop2.7
> >>
> >> Hi there
> >>
> >> Please see the console output
> >>
> >>
> >>
> >> On Thu, Oct 12, 2017 at 1:25 PM, Jeff Zhang <[hidden email]> wrote:
> >>
> >>>
> >>> What kind of DEBUG log you see ? Can you paste it ?
> >>>
> >>>
> >>> [hidden email] <[hidden email]>于2017年10月12日周四 下午5:44写道:
> >>>
> >>>>
> >>>>
> >>>> On 2017-10-12 10:41, "[hidden email]"<[hidden email]> wrote:
> >>>> > Hello there
> >>>> >
> >>>> > I have installed Spark 2.20 and from the spark-shell, my scala code
> >>>> works perfectly and performs well.
> >>>> >
> >>>> > The same scala executed from Zeppelin causes lots of DEBUG statements
> >>>> to be printed in the console (almost non-stop) and the performance is very
> >>>> slow.
> >>>> >
> >>>> > I have just installed binaries and haven't changed any
> >>>> configurations, but it seems Zeppelin is running in debug mode...
> >>>> >
> >>>> > Help..
> >>>> >
> >>>> > I use bin\zeppelin.cmd to start zeppelin. I'm on Windows Server 2012
> >>>> >
> >>>> > Please help...
> >>>> >
> >>>> >
> >>>> > PS
> >>>>
> >>>> The scala works perfectly within Zeppelin... Just very slow performance
> >>>> and lots of Debug output...
> >>>>
> >>>
> >>
> >
>  Hi there. Since the same spark sql is 3 times SLOWER on ZEPPELIN, how do i ensure that a query through ZEPPELIN uses the same as that in SPARK-SHELL. Please help