reduce graph generation times

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

reduce graph generation times

synthnassizer
hi all,
I am using rrdtool in an ARM A9 Cortex processor (Freescale iMX6q, 1GB RAM).
There I create png graphs for a web server that runs on the same computer module.

I am using a ramdrive , to minimize access to the rrd db and saving graphs times.
But it still takes about 3 seconds to generate 12 graphs and I would like to reduce that time.

What options do I have?

For example, can I save in uncompressed format (to avoid the compression time ) ? I did not find such an option in the documentation, but I hope there may be something I have overlooked.
Are there other options?

Thank you in advance for your help.

Reply | Threaded
Open this post in threaded view
|

Re: reduce graph generation times

Alex van den Bogaerdt-5
> hi all,
> I am using rrdtool in an ARM A9 Cortex processor (Freescale iMX6q, 1GB
> RAM).
> There I create png graphs for a web server that runs on the same computer
> module.
>
> I am using a ramdrive , to minimize access to the rrd db and saving graphs
> times.
> But it still takes about 3 seconds to generate 12 graphs and I would like
> to reduce that time.

Try to display data which is already in the RRAs, without needing further
processing.  What do I mean by this: if your RRA stores data in a 5-minute
resolution, make sure your graph is also 5 minutes per pixel column, and
the RRA contains enough data to display what you ask for.

End time should be the current time, rounded down to exactly 300 seconds.
Start time should be end time minus the amount of columns times 300
seconds. The RRA needs to contain enough rows to cover start to end.

Example numbers: unix time while writing this was 1446967168. Rounded down
to the nearest n*300 = int(1446967168/300)*300 = 4823223*300 = 1446966900.
Assuming a graph width of 400 pixels, the amount of time is 400*300=120000
seconds.
graph parameters: --end 1446966900 --start end-120000 --width 400

This way RRDtool does not have to do 'on the fly' consolidation at graph
time.

Of course you may want to have other step sizes, the same principle applies.
* desired amount of time to display / desired number of pixels = step size
of each RRA row
* desired number of pixels wide == minimum amount of rows in your RRA
* selecting an end time in the past => need more rows in your RRA

One more example: you want to display exactly one month's worth of data.
31 days * 24 hours = 744 hours. That graph would be too wide, so you want
to display 2 hours per pixel.  This means you would want a RRA containing
7200 seconds per row. At the very minimum you need to keep 372 rows in
this RRA. 745 if you always want to be able to display last month.
At graph time you select an appropriate end time, round down to the next
whole hour, and so on similar to the previous example. If instead you
would not have this RRA containing 7200 seconds per row, then RRDtool
would have to need to compute the average of two 1-hour rows at graph
time, and do this 372 times. Or, worse, compute the average of 24 5-minute
rows, 372 times.


Another thing you may want to try is to run rrdtool in pipe mode. It needs
to load one time only, processing all 12 graphs.

HTH
Alex


_______________________________________________
rrd-users mailing list
[hidden email]
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
Reply | Threaded
Open this post in threaded view
|

Re: reduce graph generation times

Simon Hobson
Alex van den Bogaerdt <[hidden email]> wrote: some very good advice.

The only thing I can add is to, if you have the storage, the --lazy option. That way, while each graph will take the same time to create, it only needs to be created once.
So if (for example) you have a page with several graphs of the same thing over different timescales (eg day, week, month, year) then you really only need to create the day graph the first time the page is shown in any day - and after that it won't change again until tomorrow.
Using --lazy means it'll keep that first image on disk and just serve it up rather than re-render it.

It really depends on your workload, but can significantly reduce the number of graphs you need to draw.

_______________________________________________
rrd-users mailing list
[hidden email]
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
Reply | Threaded
Open this post in threaded view
|

Re: reduce graph generation times

oetiker
Administrator
image compression does indeed take significant time. unfortunately, cairo does not allow to configure the compression level. so in order to tune this, the png generation process would have to be moved into rrdtool. not all that difficult ... just look at the relevant code from cairo and copy it over

cheers

Tobias Oetiker
[hidden email]
062 775 9902

> On 08.11.2015, at 14:02, Simon Hobson <[hidden email]> wrote:
>
> Alex van den Bogaerdt <[hidden email]> wrote: some very good advice.
>
> The only thing I can add is to, if you have the storage, the --lazy option. That way, while each graph will take the same time to create, it only needs to be created once.
> So if (for example) you have a page with several graphs of the same thing over different timescales (eg day, week, month, year) then you really only need to create the day graph the first time the page is shown in any day - and after that it won't change again until tomorrow.
> Using --lazy means it'll keep that first image on disk and just serve it up rather than re-render it.
>
> It really depends on your workload, but can significantly reduce the number of graphs you need to draw.
>
> _______________________________________________
> rrd-users mailing list
> [hidden email]
> https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
>

_______________________________________________
rrd-users mailing list
[hidden email]
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users