is there way to update(push latest data to) Zeppelin dashboard ?

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

is there way to update(push latest data to) Zeppelin dashboard ?

kant kodali
Hi Guys,

is there way to update(push latest data to but not pull) Zeppelin dashboard ? for example I get bunch of data to my spark streaming cluster and as I get more and more data I want to compute and write it somewhere such that my Zeppelin dashboard is updated? If so, where would be the best place to write the latest results?

Thanks,
Kant
Reply | Threaded
Open this post in threaded view
|

Re: is there way to update(push latest data to) Zeppelin dashboard ?

Vinay Shukla
Hey Kant,

For fast data updates via streaming the normal flow is to put the aggregated results in things like HBase. You could then use Zeppelin's Phoenix interpreter to read that data.
Not sure if this fits your requirements.

Thanks,
Vinay 

On Wednesday, August 24, 2016, kant kodali <[hidden email]> wrote:
Hi Guys,

is there way to update(push latest data to but not pull) Zeppelin dashboard ? for example I get bunch of data to my spark streaming cluster and as I get more and more data I want to compute and write it somewhere such that my Zeppelin dashboard is updated? If so, where would be the best place to write the latest results?

Thanks,
Kant
Reply | Threaded
Open this post in threaded view
|

Re: is there way to update(push latest data to) Zeppelin dashboard ?

kant kodali
Hi,

Sure, I can either write it to HDFS or Cassandra but not HBASE since we don't use it but how about constant updates? you know what I mean? I want the updates to be pushed to the dashboard I dont want keep running the query over and over again or do some sort of polling.

Thanks!



On Wed, Aug 24, 2016 3:36 AM, Vinay Shukla [hidden email] wrote:
Hey Kant,

For fast data updates via streaming the normal flow is to put the aggregated results in things like HBase. You could then use Zeppelin's Phoenix interpreter to read that data.
Not sure if this fits your requirements.

Thanks,
Vinay 

On Wednesday, August 24, 2016, kant kodali <[hidden email]> wrote:
Hi Guys,

is there way to update(push latest data to but not pull) Zeppelin dashboard ? for example I get bunch of data to my spark streaming cluster and as I get more and more data I want to compute and write it somewhere such that my Zeppelin dashboard is updated? If so, where would be the best place to write the latest results?

Thanks,
Kant
Reply | Threaded
Open this post in threaded view
|

Re: is there way to update(push latest data to) Zeppelin dashboard ?

DuyHai Doan
"I want the updates to be pushed to the dashboard I dont want keep running the query over and over again or do some sort of polling."

In this case the only solution for you is:

1) use %angular interpreter
2) import a JS library in a paragraph and do some JS coding to open a web-socket channel to the data source so that the new updates can be pushed directly to the paragraph



On Wed, Aug 24, 2016 at 1:15 PM, kant kodali <[hidden email]> wrote:
Hi,

Sure, I can either write it to HDFS or Cassandra but not HBASE since we don't use it but how about constant updates? you know what I mean? I want the updates to be pushed to the dashboard I dont want keep running the query over and over again or do some sort of polling.

Thanks!



On Wed, Aug 24, 2016 3:36 AM, Vinay Shukla [hidden email] wrote:
Hey Kant,

For fast data updates via streaming the normal flow is to put the aggregated results in things like HBase. You could then use Zeppelin's Phoenix interpreter to read that data.
Not sure if this fits your requirements.

Thanks,
Vinay 

On Wednesday, August 24, 2016, kant kodali <[hidden email]> wrote:
Hi Guys,

is there way to update(push latest data to but not pull) Zeppelin dashboard ? for example I get bunch of data to my spark streaming cluster and as I get more and more data I want to compute and write it somewhere such that my Zeppelin dashboard is updated? If so, where would be the best place to write the latest results?

Thanks,
Kant

Reply | Threaded
Open this post in threaded view
|

Re: is there way to update(push latest data to) Zeppelin dashboard ?

Vinay Shukla
Is there a way to make it easier for our Zeppelin users? Perhaps some built in visualization widget that can poll a backend for updates?


On Wednesday, August 24, 2016, DuyHai Doan <[hidden email]> wrote:
"I want the updates to be pushed to the dashboard I dont want keep running the query over and over again or do some sort of polling."

In this case the only solution for you is:

1) use %angular interpreter
2) import a JS library in a paragraph and do some JS coding to open a web-socket channel to the data source so that the new updates can be pushed directly to the paragraph



On Wed, Aug 24, 2016 at 1:15 PM, kant kodali <<a href="javascript:_e(%7B%7D,&#39;cvml&#39;,&#39;kanth909@gmail.com&#39;);" target="_blank">kanth909@...> wrote:
Hi,

Sure, I can either write it to HDFS or Cassandra but not HBASE since we don't use it but how about constant updates? you know what I mean? I want the updates to be pushed to the dashboard I dont want keep running the query over and over again or do some sort of polling.

Thanks!



On Wed, Aug 24, 2016 3:36 AM, Vinay Shukla <a href="javascript:_e(%7B%7D,&#39;cvml&#39;,&#39;vinayshukla@gmail.com&#39;);" target="_blank">vinayshukla@... wrote:
Hey Kant,

For fast data updates via streaming the normal flow is to put the aggregated results in things like HBase. You could then use Zeppelin's Phoenix interpreter to read that data.
Not sure if this fits your requirements.

Thanks,
Vinay 

On Wednesday, August 24, 2016, kant kodali <<a href="javascript:_e(%7B%7D,&#39;cvml&#39;,&#39;kanth909@gmail.com&#39;);" target="_blank">kanth909@...> wrote:
Hi Guys,

is there way to update(push latest data to but not pull) Zeppelin dashboard ? for example I get bunch of data to my spark streaming cluster and as I get more and more data I want to compute and write it somewhere such that my Zeppelin dashboard is updated? If so, where would be the best place to write the latest results?

Thanks,
Kant

Reply | Threaded
Open this post in threaded view
|

Re: is there way to update(push latest data to) Zeppelin dashboard ?

DuyHai Doan
"Perhaps some built in visualization widget that can poll a backend for updates?"

Surely, but then someone need to code it. And a pre-built solution is never really as flexible as a customized dev. I don't see how we can create a generic component. Unless we decouple the data receiver component from the display component and make them pluggable.


On Wed, Aug 24, 2016 at 5:51 PM, Vinay Shukla <[hidden email]> wrote:
Is there a way to make it easier for our Zeppelin users? Perhaps some built in visualization widget that can poll a backend for updates?


On Wednesday, August 24, 2016, DuyHai Doan <[hidden email]> wrote:
"I want the updates to be pushed to the dashboard I dont want keep running the query over and over again or do some sort of polling."

In this case the only solution for you is:

1) use %angular interpreter
2) import a JS library in a paragraph and do some JS coding to open a web-socket channel to the data source so that the new updates can be pushed directly to the paragraph



On Wed, Aug 24, 2016 at 1:15 PM, kant kodali <[hidden email]> wrote:
Hi,

Sure, I can either write it to HDFS or Cassandra but not HBASE since we don't use it but how about constant updates? you know what I mean? I want the updates to be pushed to the dashboard I dont want keep running the query over and over again or do some sort of polling.

Thanks!



On Wed, Aug 24, 2016 3:36 AM, Vinay Shukla [hidden email] wrote:
Hey Kant,

For fast data updates via streaming the normal flow is to put the aggregated results in things like HBase. You could then use Zeppelin's Phoenix interpreter to read that data.
Not sure if this fits your requirements.

Thanks,
Vinay 

On Wednesday, August 24, 2016, kant kodali <[hidden email]> wrote:
Hi Guys,

is there way to update(push latest data to but not pull) Zeppelin dashboard ? for example I get bunch of data to my spark streaming cluster and as I get more and more data I want to compute and write it somewhere such that my Zeppelin dashboard is updated? If so, where would be the best place to write the latest results?

Thanks,
Kant


Reply | Threaded
Open this post in threaded view
|

Re: is there way to update(push latest data to) Zeppelin dashboard ?

kant kodali
I would say this is like a very basic requirement for a dashboard and it is surprising Zeppelin doesn't have an out of the box solution. 



On Wed, Aug 24, 2016 9:06 AM, DuyHai Doan [hidden email] wrote:
"Perhaps some built in visualization widget that can poll a backend for updates?"

Surely, but then someone need to code it. And a pre-built solution is never really as flexible as a customized dev. I don't see how we can create a generic component. Unless we decouple the data receiver component from the display component and make them pluggable.


On Wed, Aug 24, 2016 at 5:51 PM, Vinay Shukla <[hidden email]> wrote:
Is there a way to make it easier for our Zeppelin users? Perhaps some built in visualization widget that can poll a backend for updates?


On Wednesday, August 24, 2016, DuyHai Doan <[hidden email]> wrote:
"I want the updates to be pushed to the dashboard I dont want keep running the query over and over again or do some sort of polling."

In this case the only solution for you is:

1) use %angular interpreter
2) import a JS library in a paragraph and do some JS coding to open a web-socket channel to the data source so that the new updates can be pushed directly to the paragraph



On Wed, Aug 24, 2016 at 1:15 PM, kant kodali <[hidden email]> wrote:
Hi,

Sure, I can either write it to HDFS or Cassandra but not HBASE since we don't use it but how about constant updates? you know what I mean? I want the updates to be pushed to the dashboard I dont want keep running the query over and over again or do some sort of polling.

Thanks!



On Wed, Aug 24, 2016 3:36 AM, Vinay Shukla [hidden email] wrote:
Hey Kant,

For fast data updates via streaming the normal flow is to put the aggregated results in things like HBase. You could then use Zeppelin's Phoenix interpreter to read that data.
Not sure if this fits your requirements.

Thanks,
Vinay 

On Wednesday, August 24, 2016, kant kodali <[hidden email]> wrote:
Hi Guys,

is there way to update(push latest data to but not pull) Zeppelin dashboard ? for example I get bunch of data to my spark streaming cluster and as I get more and more data I want to compute and write it somewhere such that my Zeppelin dashboard is updated? If so, where would be the best place to write the latest results?

Thanks,
Kant


Reply | Threaded
Open this post in threaded view
|

Re: is there way to update(push latest data to) Zeppelin dashboard ?

kant kodali
at very least can we auto refreshing on the Zeppelin dashboard on a certain time interval?



On Wed, Aug 24, 2016 10:56 AM, kant kodali [hidden email] wrote:
I would say this is like a very basic requirement for a dashboard and it is surprising Zeppelin doesn't have an out of the box solution. 



On Wed, Aug 24, 2016 9:06 AM, DuyHai Doan [hidden email] wrote:
"Perhaps some built in visualization widget that can poll a backend for updates?"

Surely, but then someone need to code it. And a pre-built solution is never really as flexible as a customized dev. I don't see how we can create a generic component. Unless we decouple the data receiver component from the display component and make them pluggable.


On Wed, Aug 24, 2016 at 5:51 PM, Vinay Shukla <[hidden email]> wrote:
Is there a way to make it easier for our Zeppelin users? Perhaps some built in visualization widget that can poll a backend for updates?


On Wednesday, August 24, 2016, DuyHai Doan <[hidden email]> wrote:
"I want the updates to be pushed to the dashboard I dont want keep running the query over and over again or do some sort of polling."

In this case the only solution for you is:

1) use %angular interpreter
2) import a JS library in a paragraph and do some JS coding to open a web-socket channel to the data source so that the new updates can be pushed directly to the paragraph



On Wed, Aug 24, 2016 at 1:15 PM, kant kodali <[hidden email]> wrote:
Hi,

Sure, I can either write it to HDFS or Cassandra but not HBASE since we don't use it but how about constant updates? you know what I mean? I want the updates to be pushed to the dashboard I dont want keep running the query over and over again or do some sort of polling.

Thanks!



On Wed, Aug 24, 2016 3:36 AM, Vinay Shukla [hidden email] wrote:
Hey Kant,

For fast data updates via streaming the normal flow is to put the aggregated results in things like HBase. You could then use Zeppelin's Phoenix interpreter to read that data.
Not sure if this fits your requirements.

Thanks,
Vinay 

On Wednesday, August 24, 2016, kant kodali <[hidden email]> wrote:
Hi Guys,

is there way to update(push latest data to but not pull) Zeppelin dashboard ? for example I get bunch of data to my spark streaming cluster and as I get more and more data I want to compute and write it somewhere such that my Zeppelin dashboard is updated? If so, where would be the best place to write the latest results?

Thanks,
Kant


Reply | Threaded
Open this post in threaded view
|

Re: is there way to update(push latest data to) Zeppelin dashboard ?

moon
Administrator
Try find small 'clock' icon on any particular Note next to 'Remove' button.
This scheduler feature will help you auto refresh Note.

Thanks,
moon

On Wed, Aug 24, 2016 at 11:31 AM kant kodali <[hidden email]> wrote:
at very least can we auto refreshing on the Zeppelin dashboard on a certain time interval?



On Wed, Aug 24, 2016 10:56 AM, kant kodali [hidden email] wrote:
I would say this is like a very basic requirement for a dashboard and it is surprising Zeppelin doesn't have an out of the box solution. 



On Wed, Aug 24, 2016 9:06 AM, DuyHai Doan [hidden email] wrote:
"Perhaps some built in visualization widget that can poll a backend for updates?"

Surely, but then someone need to code it. And a pre-built solution is never really as flexible as a customized dev. I don't see how we can create a generic component. Unless we decouple the data receiver component from the display component and make them pluggable.


On Wed, Aug 24, 2016 at 5:51 PM, Vinay Shukla <[hidden email]> wrote:
Is there a way to make it easier for our Zeppelin users? Perhaps some built in visualization widget that can poll a backend for updates?


On Wednesday, August 24, 2016, DuyHai Doan <[hidden email]> wrote:
"I want the updates to be pushed to the dashboard I dont want keep running the query over and over again or do some sort of polling."

In this case the only solution for you is:

1) use %angular interpreter
2) import a JS library in a paragraph and do some JS coding to open a web-socket channel to the data source so that the new updates can be pushed directly to the paragraph



On Wed, Aug 24, 2016 at 1:15 PM, kant kodali <[hidden email]> wrote:
Hi,

Sure, I can either write it to HDFS or Cassandra but not HBASE since we don't use it but how about constant updates? you know what I mean? I want the updates to be pushed to the dashboard I dont want keep running the query over and over again or do some sort of polling.

Thanks!



On Wed, Aug 24, 2016 3:36 AM, Vinay Shukla [hidden email] wrote:
Hey Kant,

For fast data updates via streaming the normal flow is to put the aggregated results in things like HBase. You could then use Zeppelin's Phoenix interpreter to read that data.
Not sure if this fits your requirements.

Thanks,
Vinay 

On Wednesday, August 24, 2016, kant kodali <[hidden email]> wrote:
Hi Guys,

is there way to update(push latest data to but not pull) Zeppelin dashboard ? for example I get bunch of data to my spark streaming cluster and as I get more and more data I want to compute and write it somewhere such that my Zeppelin dashboard is updated? If so, where would be the best place to write the latest results?

Thanks,
Kant


Reply | Threaded
Open this post in threaded view
|

Re: is there way to update(push latest data to) Zeppelin dashboard ?

Corneau Damien
Wouldn't angular bind work in that case?

On Thu, Aug 25, 2016 at 6:13 AM, moon soo Lee <[hidden email]> wrote:
Try find small 'clock' icon on any particular Note next to 'Remove' button.
This scheduler feature will help you auto refresh Note.

Thanks,
moon


On Wed, Aug 24, 2016 at 11:31 AM kant kodali <[hidden email]> wrote:
at very least can we auto refreshing on the Zeppelin dashboard on a certain time interval?



On Wed, Aug 24, 2016 10:56 AM, kant kodali [hidden email] wrote:
I would say this is like a very basic requirement for a dashboard and it is surprising Zeppelin doesn't have an out of the box solution. 



On Wed, Aug 24, 2016 9:06 AM, DuyHai Doan [hidden email] wrote:
"Perhaps some built in visualization widget that can poll a backend for updates?"

Surely, but then someone need to code it. And a pre-built solution is never really as flexible as a customized dev. I don't see how we can create a generic component. Unless we decouple the data receiver component from the display component and make them pluggable.


On Wed, Aug 24, 2016 at 5:51 PM, Vinay Shukla <[hidden email]> wrote:
Is there a way to make it easier for our Zeppelin users? Perhaps some built in visualization widget that can poll a backend for updates?


On Wednesday, August 24, 2016, DuyHai Doan <[hidden email]> wrote:
"I want the updates to be pushed to the dashboard I dont want keep running the query over and over again or do some sort of polling."

In this case the only solution for you is:

1) use %angular interpreter
2) import a JS library in a paragraph and do some JS coding to open a web-socket channel to the data source so that the new updates can be pushed directly to the paragraph



On Wed, Aug 24, 2016 at 1:15 PM, kant kodali <[hidden email]> wrote:
Hi,

Sure, I can either write it to HDFS or Cassandra but not HBASE since we don't use it but how about constant updates? you know what I mean? I want the updates to be pushed to the dashboard I dont want keep running the query over and over again or do some sort of polling.

Thanks!



On Wed, Aug 24, 2016 3:36 AM, Vinay Shukla [hidden email] wrote:
Hey Kant,

For fast data updates via streaming the normal flow is to put the aggregated results in things like HBase. You could then use Zeppelin's Phoenix interpreter to read that data.
Not sure if this fits your requirements.

Thanks,
Vinay 

On Wednesday, August 24, 2016, kant kodali <[hidden email]> wrote:
Hi Guys,

is there way to update(push latest data to but not pull) Zeppelin dashboard ? for example I get bunch of data to my spark streaming cluster and as I get more and more data I want to compute and write it somewhere such that my Zeppelin dashboard is updated? If so, where would be the best place to write the latest results?

Thanks,
Kant



Reply | Threaded
Open this post in threaded view
|

Re: is there way to update(push latest data to) Zeppelin dashboard ?

kant kodali
I dont know how angular works moreover I don't understand why we should be tied to certain framework? 



On Wed, Aug 24, 2016 6:53 PM, Corneau Damien [hidden email] wrote:
Wouldn't angular bind work in that case?

On Thu, Aug 25, 2016 at 6:13 AM, moon soo Lee <[hidden email]> wrote:
Try find small 'clock' icon on any particular Note next to 'Remove' button.
This scheduler feature will help you auto refresh Note.

Thanks,
moon


On Wed, Aug 24, 2016 at 11:31 AM kant kodali <[hidden email]> wrote:
at very least can we auto refreshing on the Zeppelin dashboard on a certain time interval?



On Wed, Aug 24, 2016 10:56 AM, kant kodali [hidden email] wrote:
I would say this is like a very basic requirement for a dashboard and it is surprising Zeppelin doesn't have an out of the box solution. 



On Wed, Aug 24, 2016 9:06 AM, DuyHai Doan [hidden email] wrote:
"Perhaps some built in visualization widget that can poll a backend for updates?"

Surely, but then someone need to code it. And a pre-built solution is never really as flexible as a customized dev. I don't see how we can create a generic component. Unless we decouple the data receiver component from the display component and make them pluggable.


On Wed, Aug 24, 2016 at 5:51 PM, Vinay Shukla <[hidden email]> wrote:
Is there a way to make it easier for our Zeppelin users? Perhaps some built in visualization widget that can poll a backend for updates?


On Wednesday, August 24, 2016, DuyHai Doan <[hidden email]> wrote:
"I want the updates to be pushed to the dashboard I dont want keep running the query over and over again or do some sort of polling."

In this case the only solution for you is:

1) use %angular interpreter
2) import a JS library in a paragraph and do some JS coding to open a web-socket channel to the data source so that the new updates can be pushed directly to the paragraph



On Wed, Aug 24, 2016 at 1:15 PM, kant kodali <[hidden email]> wrote:
Hi,

Sure, I can either write it to HDFS or Cassandra but not HBASE since we don't use it but how about constant updates? you know what I mean? I want the updates to be pushed to the dashboard I dont want keep running the query over and over again or do some sort of polling.

Thanks!



On Wed, Aug 24, 2016 3:36 AM, Vinay Shukla [hidden email] wrote:
Hey Kant,

For fast data updates via streaming the normal flow is to put the aggregated results in things like HBase. You could then use Zeppelin's Phoenix interpreter to read that data.
Not sure if this fits your requirements.

Thanks,
Vinay 

On Wednesday, August 24, 2016, kant kodali <[hidden email]> wrote:
Hi Guys,

is there way to update(push latest data to but not pull) Zeppelin dashboard ? for example I get bunch of data to my spark streaming cluster and as I get more and more data I want to compute and write it somewhere such that my Zeppelin dashboard is updated? If so, where would be the best place to write the latest results?

Thanks,
Kant



Reply | Threaded
Open this post in threaded view
|

Re: is there way to update(push latest data to) Zeppelin dashboard ?

Corneau Damien

On Thu, Aug 25, 2016 at 11:19 AM, kant kodali <[hidden email]> wrote:
I dont know how angular works moreover I don't understand why we should be tied to certain framework? 



On Wed, Aug 24, 2016 6:53 PM, Corneau Damien [hidden email] wrote:
Wouldn't angular bind work in that case?

On Thu, Aug 25, 2016 at 6:13 AM, moon soo Lee <[hidden email]> wrote:
Try find small 'clock' icon on any particular Note next to 'Remove' button.
This scheduler feature will help you auto refresh Note.

Thanks,
moon


On Wed, Aug 24, 2016 at 11:31 AM kant kodali <[hidden email]> wrote:
at very least can we auto refreshing on the Zeppelin dashboard on a certain time interval?



On Wed, Aug 24, 2016 10:56 AM, kant kodali [hidden email] wrote:
I would say this is like a very basic requirement for a dashboard and it is surprising Zeppelin doesn't have an out of the box solution. 



On Wed, Aug 24, 2016 9:06 AM, DuyHai Doan [hidden email] wrote:
"Perhaps some built in visualization widget that can poll a backend for updates?"

Surely, but then someone need to code it. And a pre-built solution is never really as flexible as a customized dev. I don't see how we can create a generic component. Unless we decouple the data receiver component from the display component and make them pluggable.


On Wed, Aug 24, 2016 at 5:51 PM, Vinay Shukla <[hidden email]> wrote:
Is there a way to make it easier for our Zeppelin users? Perhaps some built in visualization widget that can poll a backend for updates?


On Wednesday, August 24, 2016, DuyHai Doan <[hidden email]> wrote:
"I want the updates to be pushed to the dashboard I dont want keep running the query over and over again or do some sort of polling."

In this case the only solution for you is:

1) use %angular interpreter
2) import a JS library in a paragraph and do some JS coding to open a web-socket channel to the data source so that the new updates can be pushed directly to the paragraph



On Wed, Aug 24, 2016 at 1:15 PM, kant kodali <[hidden email]> wrote:
Hi,

Sure, I can either write it to HDFS or Cassandra but not HBASE since we don't use it but how about constant updates? you know what I mean? I want the updates to be pushed to the dashboard I dont want keep running the query over and over again or do some sort of polling.

Thanks!



On Wed, Aug 24, 2016 3:36 AM, Vinay Shukla [hidden email] wrote:
Hey Kant,

For fast data updates via streaming the normal flow is to put the aggregated results in things like HBase. You could then use Zeppelin's Phoenix interpreter to read that data.
Not sure if this fits your requirements.

Thanks,
Vinay 

On Wednesday, August 24, 2016, kant kodali <[hidden email]> wrote:
Hi Guys,

is there way to update(push latest data to but not pull) Zeppelin dashboard ? for example I get bunch of data to my spark streaming cluster and as I get more and more data I want to compute and write it somewhere such that my Zeppelin dashboard is updated? If so, where would be the best place to write the latest results?

Thanks,
Kant




Reply | Threaded
Open this post in threaded view
|

Re: is there way to update(push latest data to) Zeppelin dashboard ?

Vinay Shukla
Kant,

Moon referred to the attached feature. You can have the entire note run on a scheduled basis. See the attached screenshot.

You might say it refreshes the entire note, not just selected cells, that is true.

-Vinay

On Wed, Aug 24, 2016 at 7:30 PM, Corneau Damien <[hidden email]> wrote:

On Thu, Aug 25, 2016 at 11:19 AM, kant kodali <[hidden email]> wrote:
I dont know how angular works moreover I don't understand why we should be tied to certain framework? 



On Wed, Aug 24, 2016 6:53 PM, Corneau Damien [hidden email] wrote:
Wouldn't angular bind work in that case?

On Thu, Aug 25, 2016 at 6:13 AM, moon soo Lee <[hidden email]> wrote:
Try find small 'clock' icon on any particular Note next to 'Remove' button.
This scheduler feature will help you auto refresh Note.

Thanks,
moon


On Wed, Aug 24, 2016 at 11:31 AM kant kodali <[hidden email]> wrote:
at very least can we auto refreshing on the Zeppelin dashboard on a certain time interval?



On Wed, Aug 24, 2016 10:56 AM, kant kodali [hidden email] wrote:
I would say this is like a very basic requirement for a dashboard and it is surprising Zeppelin doesn't have an out of the box solution. 



On Wed, Aug 24, 2016 9:06 AM, DuyHai Doan [hidden email] wrote:
"Perhaps some built in visualization widget that can poll a backend for updates?"

Surely, but then someone need to code it. And a pre-built solution is never really as flexible as a customized dev. I don't see how we can create a generic component. Unless we decouple the data receiver component from the display component and make them pluggable.


On Wed, Aug 24, 2016 at 5:51 PM, Vinay Shukla <[hidden email]> wrote:
Is there a way to make it easier for our Zeppelin users? Perhaps some built in visualization widget that can poll a backend for updates?


On Wednesday, August 24, 2016, DuyHai Doan <[hidden email]> wrote:
"I want the updates to be pushed to the dashboard I dont want keep running the query over and over again or do some sort of polling."

In this case the only solution for you is:

1) use %angular interpreter
2) import a JS library in a paragraph and do some JS coding to open a web-socket channel to the data source so that the new updates can be pushed directly to the paragraph



On Wed, Aug 24, 2016 at 1:15 PM, kant kodali <[hidden email]> wrote:
Hi,

Sure, I can either write it to HDFS or Cassandra but not HBASE since we don't use it but how about constant updates? you know what I mean? I want the updates to be pushed to the dashboard I dont want keep running the query over and over again or do some sort of polling.

Thanks!



On Wed, Aug 24, 2016 3:36 AM, Vinay Shukla [hidden email] wrote:
Hey Kant,

For fast data updates via streaming the normal flow is to put the aggregated results in things like HBase. You could then use Zeppelin's Phoenix interpreter to read that data.
Not sure if this fits your requirements.

Thanks,
Vinay 

On Wednesday, August 24, 2016, kant kodali <[hidden email]> wrote:
Hi Guys,

is there way to update(push latest data to but not pull) Zeppelin dashboard ? for example I get bunch of data to my spark streaming cluster and as I get more and more data I want to compute and write it somewhere such that my Zeppelin dashboard is updated? If so, where would be the best place to write the latest results?

Thanks,
Kant






Screen Shot 2016-08-24 at 7.40.40 PM.png (114K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: is there way to update(push latest data to) Zeppelin dashboard ?

kant kodali
yeah In a real time dashboard we need less than a second latency or even millisecond.



On Wed, Aug 24, 2016 7:42 PM, Vinay Shukla [hidden email] wrote:
Kant,

Moon referred to the attached feature. You can have the entire note run on a scheduled basis. See the attached screenshot.

You might say it refreshes the entire note, not just selected cells, that is true.

-Vinay

On Wed, Aug 24, 2016 at 7:30 PM, Corneau Damien <[hidden email]> wrote:

On Thu, Aug 25, 2016 at 11:19 AM, kant kodali <[hidden email]> wrote:
I dont know how angular works moreover I don't understand why we should be tied to certain framework? 



On Wed, Aug 24, 2016 6:53 PM, Corneau Damien [hidden email] wrote:
Wouldn't angular bind work in that case?

On Thu, Aug 25, 2016 at 6:13 AM, moon soo Lee <[hidden email]> wrote:
Try find small 'clock' icon on any particular Note next to 'Remove' button.
This scheduler feature will help you auto refresh Note.

Thanks,
moon


On Wed, Aug 24, 2016 at 11:31 AM kant kodali <[hidden email]> wrote:
at very least can we auto refreshing on the Zeppelin dashboard on a certain time interval?



On Wed, Aug 24, 2016 10:56 AM, kant kodali [hidden email] wrote:
I would say this is like a very basic requirement for a dashboard and it is surprising Zeppelin doesn't have an out of the box solution. 



On Wed, Aug 24, 2016 9:06 AM, DuyHai Doan [hidden email] wrote:
"Perhaps some built in visualization widget that can poll a backend for updates?"

Surely, but then someone need to code it. And a pre-built solution is never really as flexible as a customized dev. I don't see how we can create a generic component. Unless we decouple the data receiver component from the display component and make them pluggable.


On Wed, Aug 24, 2016 at 5:51 PM, Vinay Shukla <[hidden email]> wrote:
Is there a way to make it easier for our Zeppelin users? Perhaps some built in visualization widget that can poll a backend for updates?


On Wednesday, August 24, 2016, DuyHai Doan <[hidden email]> wrote:
"I want the updates to be pushed to the dashboard I dont want keep running the query over and over again or do some sort of polling."

In this case the only solution for you is:

1) use %angular interpreter
2) import a JS library in a paragraph and do some JS coding to open a web-socket channel to the data source so that the new updates can be pushed directly to the paragraph



On Wed, Aug 24, 2016 at 1:15 PM, kant kodali <[hidden email]> wrote:
Hi,

Sure, I can either write it to HDFS or Cassandra but not HBASE since we don't use it but how about constant updates? you know what I mean? I want the updates to be pushed to the dashboard I dont want keep running the query over and over again or do some sort of polling.

Thanks!



On Wed, Aug 24, 2016 3:36 AM, Vinay Shukla [hidden email] wrote:
Hey Kant,

For fast data updates via streaming the normal flow is to put the aggregated results in things like HBase. You could then use Zeppelin's Phoenix interpreter to read that data.
Not sure if this fits your requirements.

Thanks,
Vinay 

On Wednesday, August 24, 2016, kant kodali <[hidden email]> wrote:
Hi Guys,

is there way to update(push latest data to but not pull) Zeppelin dashboard ? for example I get bunch of data to my spark streaming cluster and as I get more and more data I want to compute and write it somewhere such that my Zeppelin dashboard is updated? If so, where would be the best place to write the latest results?

Thanks,
Kant





Reply | Threaded
Open this post in threaded view
|

Re: is there way to update(push latest data to) Zeppelin dashboard ?

Corneau Damien
Actually there isn't any Dashboard feature to say in Apache Zeppelin

On Thu, Aug 25, 2016 at 11:54 AM, kant kodali <[hidden email]> wrote:
yeah In a real time dashboard we need less than a second latency or even millisecond.



On Wed, Aug 24, 2016 7:42 PM, Vinay Shukla [hidden email] wrote:
Kant,

Moon referred to the attached feature. You can have the entire note run on a scheduled basis. See the attached screenshot.

You might say it refreshes the entire note, not just selected cells, that is true.

-Vinay

On Wed, Aug 24, 2016 at 7:30 PM, Corneau Damien <[hidden email]> wrote:

On Thu, Aug 25, 2016 at 11:19 AM, kant kodali <[hidden email]> wrote:
I dont know how angular works moreover I don't understand why we should be tied to certain framework? 



On Wed, Aug 24, 2016 6:53 PM, Corneau Damien [hidden email] wrote:
Wouldn't angular bind work in that case?

On Thu, Aug 25, 2016 at 6:13 AM, moon soo Lee <[hidden email]> wrote:
Try find small 'clock' icon on any particular Note next to 'Remove' button.
This scheduler feature will help you auto refresh Note.

Thanks,
moon


On Wed, Aug 24, 2016 at 11:31 AM kant kodali <[hidden email]> wrote:
at very least can we auto refreshing on the Zeppelin dashboard on a certain time interval?



On Wed, Aug 24, 2016 10:56 AM, kant kodali [hidden email] wrote:
I would say this is like a very basic requirement for a dashboard and it is surprising Zeppelin doesn't have an out of the box solution. 



On Wed, Aug 24, 2016 9:06 AM, DuyHai Doan [hidden email] wrote:
"Perhaps some built in visualization widget that can poll a backend for updates?"

Surely, but then someone need to code it. And a pre-built solution is never really as flexible as a customized dev. I don't see how we can create a generic component. Unless we decouple the data receiver component from the display component and make them pluggable.


On Wed, Aug 24, 2016 at 5:51 PM, Vinay Shukla <[hidden email]> wrote:
Is there a way to make it easier for our Zeppelin users? Perhaps some built in visualization widget that can poll a backend for updates?


On Wednesday, August 24, 2016, DuyHai Doan <[hidden email]> wrote:
"I want the updates to be pushed to the dashboard I dont want keep running the query over and over again or do some sort of polling."

In this case the only solution for you is:

1) use %angular interpreter
2) import a JS library in a paragraph and do some JS coding to open a web-socket channel to the data source so that the new updates can be pushed directly to the paragraph



On Wed, Aug 24, 2016 at 1:15 PM, kant kodali <[hidden email]> wrote:
Hi,

Sure, I can either write it to HDFS or Cassandra but not HBASE since we don't use it but how about constant updates? you know what I mean? I want the updates to be pushed to the dashboard I dont want keep running the query over and over again or do some sort of polling.

Thanks!



On Wed, Aug 24, 2016 3:36 AM, Vinay Shukla [hidden email] wrote:
Hey Kant,

For fast data updates via streaming the normal flow is to put the aggregated results in things like HBase. You could then use Zeppelin's Phoenix interpreter to read that data.
Not sure if this fits your requirements.

Thanks,
Vinay 

On Wednesday, August 24, 2016, kant kodali <[hidden email]> wrote:
Hi Guys,

is there way to update(push latest data to but not pull) Zeppelin dashboard ? for example I get bunch of data to my spark streaming cluster and as I get more and more data I want to compute and write it somewhere such that my Zeppelin dashboard is updated? If so, where would be the best place to write the latest results?

Thanks,
Kant






Reply | Threaded
Open this post in threaded view
|

Re: is there way to update(push latest data to) Zeppelin dashboard ?

kant kodali
By Dashboard I mean a Visualization backed by a data but the main point here is how can we make real time latency requirements using Zeppelin



On Wed, Aug 24, 2016 7:59 PM, Corneau Damien [hidden email] wrote:
Actually there isn't any Dashboard feature to say in Apache Zeppelin

On Thu, Aug 25, 2016 at 11:54 AM, kant kodali <[hidden email]> wrote:
yeah In a real time dashboard we need less than a second latency or even millisecond.



On Wed, Aug 24, 2016 7:42 PM, Vinay Shukla [hidden email] wrote:
Kant,

Moon referred to the attached feature. You can have the entire note run on a scheduled basis. See the attached screenshot.

You might say it refreshes the entire note, not just selected cells, that is true.

-Vinay

On Wed, Aug 24, 2016 at 7:30 PM, Corneau Damien <[hidden email]> wrote:

On Thu, Aug 25, 2016 at 11:19 AM, kant kodali <[hidden email]> wrote:
I dont know how angular works moreover I don't understand why we should be tied to certain framework? 



On Wed, Aug 24, 2016 6:53 PM, Corneau Damien [hidden email] wrote:
Wouldn't angular bind work in that case?

On Thu, Aug 25, 2016 at 6:13 AM, moon soo Lee <[hidden email]> wrote:
Try find small 'clock' icon on any particular Note next to 'Remove' button.
This scheduler feature will help you auto refresh Note.

Thanks,
moon


On Wed, Aug 24, 2016 at 11:31 AM kant kodali <[hidden email]> wrote:
at very least can we auto refreshing on the Zeppelin dashboard on a certain time interval?



On Wed, Aug 24, 2016 10:56 AM, kant kodali [hidden email] wrote:
I would say this is like a very basic requirement for a dashboard and it is surprising Zeppelin doesn't have an out of the box solution. 



On Wed, Aug 24, 2016 9:06 AM, DuyHai Doan [hidden email] wrote:
"Perhaps some built in visualization widget that can poll a backend for updates?"

Surely, but then someone need to code it. And a pre-built solution is never really as flexible as a customized dev. I don't see how we can create a generic component. Unless we decouple the data receiver component from the display component and make them pluggable.


On Wed, Aug 24, 2016 at 5:51 PM, Vinay Shukla <[hidden email]> wrote:
Is there a way to make it easier for our Zeppelin users? Perhaps some built in visualization widget that can poll a backend for updates?


On Wednesday, August 24, 2016, DuyHai Doan <[hidden email]> wrote:
"I want the updates to be pushed to the dashboard I dont want keep running the query over and over again or do some sort of polling."

In this case the only solution for you is:

1) use %angular interpreter
2) import a JS library in a paragraph and do some JS coding to open a web-socket channel to the data source so that the new updates can be pushed directly to the paragraph



On Wed, Aug 24, 2016 at 1:15 PM, kant kodali <[hidden email]> wrote:
Hi,

Sure, I can either write it to HDFS or Cassandra but not HBASE since we don't use it but how about constant updates? you know what I mean? I want the updates to be pushed to the dashboard I dont want keep running the query over and over again or do some sort of polling.

Thanks!



On Wed, Aug 24, 2016 3:36 AM, Vinay Shukla [hidden email] wrote:
Hey Kant,

For fast data updates via streaming the normal flow is to put the aggregated results in things like HBase. You could then use Zeppelin's Phoenix interpreter to read that data.
Not sure if this fits your requirements.

Thanks,
Vinay 

On Wednesday, August 24, 2016, kant kodali <[hidden email]> wrote:
Hi Guys,

is there way to update(push latest data to but not pull) Zeppelin dashboard ? for example I get bunch of data to my spark streaming cluster and as I get more and more data I want to compute and write it somewhere such that my Zeppelin dashboard is updated? If so, where would be the best place to write the latest results?

Thanks,
Kant