SVG interaction on RRDs graphs

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

SVG interaction on RRDs graphs

nbelorgey
Hi all,

We have a lot of stacked graphs & we would like to add interaction on them.

We managed to have a mouse over interaction on a area and a tinytips by editing manually a svg rrdgraph outpout.


To automaticaly do this stuff :
 - is there a way to add with the "rrdtool graph command" explicit ids to the SVG xml objects in order to easily parse it /add scripts after generation
 - or should we parse it and create an automatic detection of our objects ?

Cheers,
Nico
Reply | Threaded
Open this post in threaded view
|

Re: SVG interaction on RRDs graphs

Vladimir Vuksan
I was wondering if you can share how you hand edited SVG and what kind of
HTML markup you added to have interaction.

Thanks,

Vladimir

On Thu, 1 Sep 2011, nbelorgey wrote:

> Hi all,
>
> We have a lot of stacked graphs & we would like to add interaction on them.
>
> We managed to have a mouse over interaction on a area and a tinytips by
> editing manually a svg rrdgraph outpout.
>
>
> To automaticaly do this stuff :
> - is there a way to add with the "rrdtool graph command" explicit ids to
> the SVG xml objects in order to easily parse it /add scripts after
> generation
> - or should we parse it and create an automatic detection of our objects ?
>
> Cheers,
> Nico
>
>
> --
> View this message in context: http://rrd-mailinglists.937164.n2.nabble.com/SVG-interaction-on-RRDs-graphs-tp6750442p6750442.html
> Sent from the RRDtool Users Mailinglist mailing list archive at Nabble.com.
>
> _______________________________________________
> rrd-users mailing list
> [hidden email]
> https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
>

_______________________________________________
rrd-users mailing list
[hidden email]
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
Reply | Threaded
Open this post in threaded view
|

Re: SVG interaction on RRDs graphs

nbelorgey
Have a look here :

http://www.codedread.com/displayWebStats.svg

This could be really nice to have for rrd graph / interaction.
Reply | Threaded
Open this post in threaded view
|

Re: SVG interaction on RRDs graphs

Richard Wall
In reply to this post by nbelorgey
On 1 September 2011 16:48, nbelorgey <[hidden email]> wrote:
> We have a lot of stacked graphs & we would like to add interaction on them.

I don't think it was ever posted it to this list, but there is a
complete JavaScript port of rrdgraph on github:
 * https://github.com/manuelluis/jsrrdgraph

...and demos here....
 * http://manuelluis.github.com/jsrrdgraph/

Try the second demo in Chrome - drag the graph to move the timeline
and use the mouse wheel to zoom in and out.

It does *everything* on the client side, including the reading of the
rrd files. All you have to do is make the RRDs available via http.

Anyway, I recommend you take a look and note that there were a few
issues in that initial version which I have fixed and am waiting for
the author to merge.
 * https://github.com/manuelluis/jsrrdgraph/pull/2

-RichardW.

_______________________________________________
rrd-users mailing list
[hidden email]
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
Reply | Threaded
Open this post in threaded view
|

Re: SVG interaction on RRDs graphs

nbelorgey
Richard,

The javascript graphing is huge !!!
This is fast, clear and efficient. The zoom is really user friendly & impressive...

But in our case, some graphs will stack between 20-* elements that are stored in ~6 MB RRDs. ( We're stacking VMs performance métrics on VMware Clusters)
And we have around 4-12 graph per page...  This will make for 25 VMs 25*12*6 ~ 1800 MB to download ?

The client canno't download and process so much data
That's why it has to be server site processed...

Is there in your roadmap an ajax version that process on the server side and return only needed information for graphing purpose ?
 
Reply | Threaded
Open this post in threaded view
|

Re: SVG interaction on RRDs graphs

Richard Wall
On 2 September 2011 14:20, nbelorgey <[hidden email]> wrote:
> The javascript graphing is huge !!!
> This is fast, clear and efficient. The zoom is really user friendly &
> impressive...

Yep, it's very cool. Manuel Luis did all the hard work- he originally
posted about it on the Collectd mailing  list a few months ago.

> But in our case, some graphs will stack between 20-* elements that are
> stored in ~6 MB RRDs. ( We're stacking VMs performance métrics on VMware
> Clusters)
> And we have around 4-12 graph per page...  This will make for 25 VMs 25*12*6
> ~ 1800 MB to download ?
>
> The client canno't download and process so much data
> That's why it has to be server site processed...

Understood.

> Is there in your roadmap an ajax version that process on the server side and
> return only needed information for graphing purpose ?

I haven't heard from Manuel Luis since since submitting my merge
request on Github, but there's nothing to stop us modifying jsrrdgraph
as we see fit.

My first goal is to integrate it with my Jarmon graphing
system....which currently uses javascriptRRD and Flot.
 * https://launchpad.net/jarmon

I quite like the fact that all the processing is done client
side....especially when relatively few RRDs are served with HTTP Gzip
compression.

I had thought that it might be possible to create a Javascript RRD
proxy that used XHR and byte range requests to examine the header in a
remote RRD file then download the required portion of data from the
remote RRD file.

-RichardW.

_______________________________________________
rrd-users mailing list
[hidden email]
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
Reply | Threaded
Open this post in threaded view
|

Re: SVG interaction on RRDs graphs

Vladimir Vuksan
In reply to this post by nbelorgey
I have written code for Ganglia (http://ganglia.info) that uses rrdtool
export to export data of interest then "massages" the data into a proper
JSON format. I then use Flot (Javascript graphing library) to draw the
images. You can for instance see how it looks in action if you go here

http://fjrkr5ab.joyent.us/ganglia-2.0/?r=hour&cs=&ce=&m=&c=bx+workservers&h=baker.bx.psu.edu&mc=2&z=medium&metric_group=cpu

Find a graph that has an Enlarge button next to it. Click on it. You can
then hover over the graph value and it will show you current value. This
is very much work in progress.

There was a patch submitted by Erik Kastner to allow rrdtool to export
directly to JSON but I have not seen it included.

Vladimir

On Fri, 2 Sep 2011, nbelorgey wrote:

> Richard,
>
> The javascript graphing is huge !!!
> This is fast, clear and efficient. The zoom is really user friendly &
> impressive...
>
> But in our case, some graphs will stack between 20-* elements that are
> stored in ~6 MB RRDs. ( We're stacking VMs performance métrics on VMware
> Clusters)
> And we have around 4-12 graph per page...  This will make for 25 VMs 25*12*6
> ~ 1800 MB to download ?
>
> The client canno't download and process so much data
> That's why it has to be server site processed...
>
> Is there in your roadmap an ajax version that process on the server side and
> return only needed information for graphing purpose ?
>
>
>
> --
> View this message in context: http://rrd-mailinglists.937164.n2.nabble.com/SVG-interaction-on-RRDs-graphs-tp6750442p6753841.html
> Sent from the RRDtool Users Mailinglist mailing list archive at Nabble.com.
>
> _______________________________________________
> rrd-users mailing list
> [hidden email]
> https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
_______________________________________________
rrd-users mailing list
[hidden email]
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
Reply | Threaded
Open this post in threaded view
|

Re: SVG interaction on RRDs graphs

oetiker
Administrator
In reply to this post by Richard Wall
Hi Richard,

Today Richard Wall wrote:

>
> My first goal is to integrate it with my Jarmon graphing
> system....which currently uses javascriptRRD and Flot.
>  * https://launchpad.net/jarmon
>
> I quite like the fact that all the processing is done client
> side....especially when relatively few RRDs are served with HTTP Gzip
> compression.
>
> I had thought that it might be possible to create a Javascript RRD
> proxy that used XHR and byte range requests to examine the header in a
> remote RRD file then download the required portion of data from the
> remote RRD file.

one problem of this aproach is, that the rrd data format is
different depending on the architecture where rrdtool is running
... if you used rrdtool xport in the backend you would get to a
more portable solution where you can even save bandwith by doing
some of the processing serverside ...

cheers
tobi

>
> -RichardW.
>
> _______________________________________________
> rrd-users mailing list
> [hidden email]
> https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
>
>

--
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
http://it.oetiker.ch [hidden email] ++41 62 775 9902 / sb: -9900

_______________________________________________
rrd-users mailing list
[hidden email]
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
Reply | Threaded
Open this post in threaded view
|

Re: SVG interaction on RRDs graphs

nbelorgey
@ Richard :
Jarmon look & feel is really nice also.
But Server side processing will be more efficient in our case...
The good way to do this for us will be to parse rrds with php after an ajax request & then send gzipped values to specific client side javascript graphing functions...

@Vladimir
The mouse over HRULE & tiny tips is exactly one thing we are looking for. Many thanks !!
.

@ Tobi
Our goals :
Some of our graphs stack / compile ~20-60 AREA & Lines, then we would like to implement interaction with check box / mouse over high lights & tinytips for precisely watching data values on the graphs.

SVG limittations :
Per default my SVG graphs are 3-4 times bigger than png and this generates heavy loads on the client side. ( especially for firefox )

The python scour script divide the size per 2-3 but is really slow... but permit to draw more quickly on the client side. http://www.codedread.com/scour/

It think we will add like Vladimir made in Ganglia, ie a little button that replace the default png file by a interactive SVG / emacs / Javascript / Ajax powered graph.




Reply | Threaded
Open this post in threaded view
|

Re: SVG interaction on RRDs graphs

Vladimir Vuksan
In reply to this post by oetiker
What are the odds of adding JSON output to export ? That would be a lot
more portable. There was a patch submitted some time ago.

http://oss.oetiker.ch/rrdtool/forum.en.html#nabble-td5629054

Thanks,

Vladimir

On Fri, 2 Sep 2011, Tobias Oetiker wrote:

> Hi Richard,
>
> Today Richard Wall wrote:
>
>>
>> My first goal is to integrate it with my Jarmon graphing
>> system....which currently uses javascriptRRD and Flot.
>>  * https://launchpad.net/jarmon
>>
>> I quite like the fact that all the processing is done client
>> side....especially when relatively few RRDs are served with HTTP Gzip
>> compression.
>>
>> I had thought that it might be possible to create a Javascript RRD
>> proxy that used XHR and byte range requests to examine the header in a
>> remote RRD file then download the required portion of data from the
>> remote RRD file.
>
> one problem of this aproach is, that the rrd data format is
> different depending on the architecture where rrdtool is running
> ... if you used rrdtool xport in the backend you would get to a
> more portable solution where you can even save bandwith by doing
> some of the processing serverside ...
>
> cheers
> tobi
>
>>
>> -RichardW.
>>
>> _______________________________________________
>> rrd-users mailing list
>> [hidden email]
>> https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
>>
>>
>
> --
> Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
> http://it.oetiker.ch [hidden email] ++41 62 775 9902 / sb: -9900
>
> _______________________________________________
> rrd-users mailing list
> [hidden email]
> https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
>

_______________________________________________
rrd-users mailing list
[hidden email]
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
Reply | Threaded
Open this post in threaded view
|

Re: SVG interaction on RRDs graphs

oetiker
Administrator
Hi Vladimir,

the 'problem' with the patch is two-fold

a) xport xml part is implemented in the commandline FE only, so for
   all real tools, (using the bindings) you get access to the raw data
   anyway, so there is no problem generating json directly.

b) the patch to the frontend adds another 'command' which has no
   representation in the backend ... it should be an option to xport
   instead to keep at least some sort of consistancy ... if you
   care to update the patch accordingly ?

cheers
tobi

Yesterday Vladimir Vuksan wrote:

> What are the odds of adding JSON output to export ? That would be a lot
> more portable. There was a patch submitted some time ago.
>
> http://oss.oetiker.ch/rrdtool/forum.en.html#nabble-td5629054
>
> Thanks,
>
> Vladimir
>
> On Fri, 2 Sep 2011, Tobias Oetiker wrote:
>
> > Hi Richard,
> >
> > Today Richard Wall wrote:
> >
> >>
> >> My first goal is to integrate it with my Jarmon graphing
> >> system....which currently uses javascriptRRD and Flot.
> >>  * https://launchpad.net/jarmon
> >>
> >> I quite like the fact that all the processing is done client
> >> side....especially when relatively few RRDs are served with HTTP Gzip
> >> compression.
> >>
> >> I had thought that it might be possible to create a Javascript RRD
> >> proxy that used XHR and byte range requests to examine the header in a
> >> remote RRD file then download the required portion of data from the
> >> remote RRD file.
> >
> > one problem of this aproach is, that the rrd data format is
> > different depending on the architecture where rrdtool is running
> > ... if you used rrdtool xport in the backend you would get to a
> > more portable solution where you can even save bandwith by doing
> > some of the processing serverside ...
> >
> > cheers
> > tobi
> >
> >>
> >> -RichardW.
> >>
> >> _______________________________________________
> >> rrd-users mailing list
> >> [hidden email]
> >> https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
> >>
> >>
> >
> > --
> > Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
> > http://it.oetiker.ch [hidden email] ++41 62 775 9902 / sb: -9900
> >
> > _______________________________________________
> > rrd-users mailing list
> > [hidden email]
> > https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
> >
>
> _______________________________________________
> rrd-users mailing list
> [hidden email]
> https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
>
>

--
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
http://it.oetiker.ch [hidden email] ++41 62 775 9902 / sb: -9900

_______________________________________________
rrd-users mailing list
[hidden email]
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
Reply | Threaded
Open this post in threaded view
|

Re: SVG interaction on RRDs graphs

Richard Wall
In reply to this post by oetiker
On 2 September 2011 15:42, Tobias Oetiker <[hidden email]> wrote:
<snip>
> one problem of this aproach is, that the rrd data format is
> different depending on the architecture where rrdtool is running

Tobi,

Yeah, I know this ought to be a problem, but actually it hasn't bitten
me so far.

All my RRDs are generated on Linux x86 (32 and 64bit) and these seem
to be parsed fine by the javascriptRRD library on Windows 32 / 64,
Linux x86 32 / 64 and MacOSX.

Maybe its just luck.

Do you think the idea of parsing the RRD files in javascript is
completely doomed, or are there combinations of RRD source
architecture and client arch that will generally work?

I read your portable RRD wiki page which sounds promising.

 * http://oss.oetiker.ch/rrdtool-trac/wiki/PortableRrdFormat

> ... if you used rrdtool xport in the backend you would get to a
> more portable solution where you can even save bandwith by doing
> some of the processing serverside ...

Yes, I'll certainly give that a try.

Thanks.

-RichardW.

_______________________________________________
rrd-users mailing list
[hidden email]
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
Reply | Threaded
Open this post in threaded view
|

Re: SVG interaction on RRDs graphs

oetiker
Administrator
Today Richard Wall wrote:

> On 2 September 2011 15:42, Tobias Oetiker <[hidden email]> wrote:
> <snip>
> > one problem of this aproach is, that the rrd data format is
> > different depending on the architecture where rrdtool is running
>
> Tobi,
>
> Yeah, I know this ought to be a problem, but actually it hasn't bitten
> me so far.
>
> All my RRDs are generated on Linux x86 (32 and 64bit) and these seem
> to be parsed fine by the javascriptRRD library on Windows 32 / 64,
> Linux x86 32 / 64 and MacOSX.
>
> Maybe its just luck.
>
> Do you think the idea of parsing the RRD files in javascript is
> completely doomed, or are there combinations of RRD source
> architecture and client arch that will generally work?

well it is just work ... not particularly difficult, but in the
end, you always will

* track any changes in the rrdtool format happening in the backend
* deal with the fact that rrd files can be realy big


> I read your portable RRD wiki page which sounds promising.
>
>  * http://oss.oetiker.ch/rrdtool-trac/wiki/PortableRrdFormat

yes ... unfortunately noone seems willing to sponsor such work

> > ... if you used rrdtool xport in the backend you would get to a
> > more portable solution where you can even save bandwith by doing
> > some of the processing serverside ...
>
> Yes, I'll certainly give that a try.

:-)

cheers
tobi
>
> Thanks.
>
> -RichardW.
>
>

--
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
http://it.oetiker.ch [hidden email] ++41 62 775 9902 / sb: -9900

_______________________________________________
rrd-users mailing list
[hidden email]
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
Reply | Threaded
Open this post in threaded view
|

Re: SVG interaction on RRDs graphs

Kevin P. Foote
In reply to this post by Vladimir Vuksan

Vladimir

very cool!

Any chance you can point to a code repo or post location where that data
"massage" / manipulation code is?

I'd like to take a go at something similar (JSON/Flot) outside of ganglia or other
front ends.

------
thanks
  kevin.foote

On Fri, 2 Sep 2011, Vladimir Vuksan wrote:

-> I have written code for Ganglia (http://ganglia.info) that uses rrdtool
-> export to export data of interest then "massages" the data into a proper JSON
-> format. I then use Flot (Javascript graphing library) to draw the images. You
-> can for instance see how it looks in action if you go here
->
-> http://fjrkr5ab.joyent.us/ganglia-2.0/?r=hour&cs=&ce=&m=&c=bx+workservers&h=baker.bx.psu.edu&mc=2&z=medium&metric_group=cpu
->
-> Find a graph that has an Enlarge button next to it. Click on it. You can then
-> hover over the graph value and it will show you current value. This is very
-> much work in progress.
->
-> There was a patch submitted by Erik Kastner to allow rrdtool to export
-> directly to JSON but I have not seen it included.
->
-> Vladimir
->
-> On Fri, 2 Sep 2011, nbelorgey wrote:
->
-> > Richard,
-> > The javascript graphing is huge !!!
-> > This is fast, clear and efficient. The zoom is really user friendly &
-> > impressive...
-> >
-> > But in our case, some graphs will stack between 20-* elements that are
-> > stored in ~6 MB RRDs. ( We're stacking VMs performance métrics on VMware
-> > Clusters)
-> > And we have around 4-12 graph per page...  This will make for 25 VMs
-> > 25*12*6
-> > ~ 1800 MB to download ?
-> >
-> > The client canno't download and process so much data That's why it has to
-> > be server site processed...
-> >
-> > Is there in your roadmap an ajax version that process on the server side
-> > and
-> > return only needed information for graphing purpose ?
-> >
-> >
-> > --
-> > View this message in context:
-> > http://rrd-mailinglists.937164.n2.nabble.com/SVG-interaction-on-RRDs-graphs-tp6750442p6753841.html
-> > Sent from the RRDtool Users Mailinglist mailing list archive at Nabble.com.
-> >
-> > _______________________________________________
-> > rrd-users mailing list
-> > [hidden email]
-> > https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
_______________________________________________
rrd-users mailing list
[hidden email]
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
Reply | Threaded
Open this post in threaded view
|

Re: SVG interaction on RRDs graphs

Vladimir Vuksan
Here is the code that does it

https://github.com/vvuksan/ganglia-misc/blob/master/ganglia-web/graph.php#L494

Basically it looks for definitions then does rrdtool xport for period of
interest. Then uses SimpleXML to parse it. After that you just output in
the format you want e.g. this is for flot

https://github.com/vvuksan/ganglia-misc/blob/master/ganglia-web/graph.php#L559


On Fri, 16 Sep 2011, Kevin P. Foote wrote:

>
> Vladimir
>
> very cool!
>
> Any chance you can point to a code repo or post location where that data
> "massage" / manipulation code is?
>
> I'd like to take a go at something similar (JSON/Flot) outside of ganglia or other
> front ends.
>
> ------
> thanks
>  kevin.foote
>
> On Fri, 2 Sep 2011, Vladimir Vuksan wrote:
>
> -> I have written code for Ganglia (http://ganglia.info) that uses rrdtool
> -> export to export data of interest then "massages" the data into a proper JSON
> -> format. I then use Flot (Javascript graphing library) to draw the images. You
> -> can for instance see how it looks in action if you go here
> ->
> -> http://fjrkr5ab.joyent.us/ganglia-2.0/?r=hour&cs=&ce=&m=&c=bx+workservers&h=baker.bx.psu.edu&mc=2&z=medium&metric_group=cpu
> ->
> -> Find a graph that has an Enlarge button next to it. Click on it. You can then
> -> hover over the graph value and it will show you current value. This is very
> -> much work in progress.
> ->
> -> There was a patch submitted by Erik Kastner to allow rrdtool to export
> -> directly to JSON but I have not seen it included.
> ->
> -> Vladimir
> ->
> -> On Fri, 2 Sep 2011, nbelorgey wrote:
> ->
> -> > Richard,
> -> > The javascript graphing is huge !!!
> -> > This is fast, clear and efficient. The zoom is really user friendly &
> -> > impressive...
> -> >
> -> > But in our case, some graphs will stack between 20-* elements that are
> -> > stored in ~6 MB RRDs. ( We're stacking VMs performance métrics on VMware
> -> > Clusters)
> -> > And we have around 4-12 graph per page...  This will make for 25 VMs
> -> > 25*12*6
> -> > ~ 1800 MB to download ?
> -> >
> -> > The client canno't download and process so much data That's why it has to
> -> > be server site processed...
> -> >
> -> > Is there in your roadmap an ajax version that process on the server side
> -> > and
> -> > return only needed information for graphing purpose ?
> -> >
> -> >
> -> > --
> -> > View this message in context:
> -> > http://rrd-mailinglists.937164.n2.nabble.com/SVG-interaction-on-RRDs-graphs-tp6750442p6753841.html
> -> > Sent from the RRDtool Users Mailinglist mailing list archive at Nabble.com.
> -> >
> -> > _______________________________________________
> -> > rrd-users mailing list
> -> > [hidden email]
> -> > https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
_______________________________________________
rrd-users mailing list
[hidden email]
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
Reply | Threaded
Open this post in threaded view
|

Re: SVG interaction on RRDs graphs

Doug Leith
In reply to this post by Richard Wall
> one problem of this aproach is, that the rrd data format is
> different depending on the architecture where rrdtool is running
...
> ... if you used rrdtool xport in the backend you would get to a
> more portable solution where you can even save bandwith by doing
> some of the processing serverside ...

perhaps also worth taking a look at rrd::editor.   this is a standalone perl module that aims to provide a platform-portable server backend for javascript client-server setups.  the portable format used is just the intel 32 bit platform rrdtool binary file format, so should be compatible with your javascript client front end (it certainly works with the javascriptRRD client).  

doug