RRDcached performance issues (from Users mailinglist)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RRDcached performance issues (from Users mailinglist)

Gaby
Dear developers, could you take a look at my post in the Users forum?

http://rrd-mailinglists.937164.n2.nabble.com/RRDcached-performance-issues-tp7582783.html;cid=1426345040002-291

It has no replies, and I'm still very interested in shedding some light on it.

Gaby
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: RRDcached performance issues (from Users mailinglist)

oetiker
Administrator
Hi Gaby,

I guess that very few people have rrdtool setup at your scale ...
so chances for someone coming forward with useful hints may be
rather slim.

Things to look at:

a) try rrdtool 1.5rc2 since this is the latest and greates code

b) go into the cached code and add some debugging output

c) strace/truss may provide some insight in to what rrdcached when
   it seems busy

d) attach gdb and break processing to see where in the source this
   is happening.

cheers
tobi

Yesterday Gaby wrote:

> Dear developers, could you take a look at my post in the Users forum?
>
> http://rrd-mailinglists.937164.n2.nabble.com/RRDcached-performance-issues-tp7582783.html;cid=1426345040002-291
>
> It has no replies, and I'm still very interested in shedding some light on
> it.
>
> Gaby
>
>
>
> --
> View this message in context: http://rrd-mailinglists.937164.n2.nabble.com/RRDcached-performance-issues-from-Users-mailinglist-tp7582860.html
> Sent from the RRDtool Developers Mailinglist mailing list archive at Nabble.com.
>
> _______________________________________________
> rrd-developers mailing list
> [hidden email]
> https://lists.oetiker.ch/cgi-bin/listinfo/rrd-developers
>
>

--
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
www.oetiker.ch [hidden email] +41 62 775 9902

_______________________________________________
rrd-developers mailing list
[hidden email]
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-developers
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: RRDcached performance issues (from Users mailinglist)

Steve Schnepp

Hi,

I'm a dev guy from the Munin project(*). We have some heavy RRD among our users.

Some hints from the trenches:

* I discovered that having multiple rrdcached daemon is actually very bad for performance. The only way to shard them efficiently is to have them handling a different FS with different disk subsystem. (I didn't try with SSD btw)
* For some workloads tmpfs+sync (note that a simple cp might be more efficient than a rsync) is simply the best.
* You have to avoid doing reads on rrds, as it flushes. (1.4.x, don't know about 1.5)
* A very big -w and -z setting (think 3600) helps, as it give rrdcached the most flexibility to reorder writes
* Always flush manually before closing (FLUSHALL on the socket), to avoid too long restart times. The restarts journal read is way too slow, a you noticed.

Hopes it helps.

(*) http://mm0.eu/

On 17:59, Sun, Mar 15, 2015 Tobias Oetiker <[hidden email]> wrote:

Hi Gaby,

I guess that very few people have rrdtool setup at your scale ...
so chances for someone coming forward with useful hints may be
rather slim.

Things to look at:

a) try rrdtool 1.5rc2 since this is the latest and greates code

b) go into the cached code and add some debugging output

c) strace/truss may provide some insight in to what rrdcached when
   it seems busy

d) attach gdb and break processing to see where in the source this
   is happening.

cheers
tobi

Yesterday Gaby wrote:

> Dear developers, could you take a look at my post in the Users forum?
>
> http://rrd-mailinglists.937164.n2.nabble.com/RRDcached-performance-issues-tp7582783.html;cid=1426345040002-291
>
> It has no replies, and I'm still very interested in shedding some light on
> it.
>
> Gaby
>
>
>
> --
> View this message in context: http://rrd-mailinglists.937164.n2.nabble.com/RRDcached-performance-issues-from-Users-mailinglist-tp7582860.html
> Sent from the RRDtool Developers Mailinglist mailing list archive at Nabble.com.
>
> _______________________________________________
> rrd-developers mailing list
> [hidden email]
> https://lists.oetiker.ch/cgi-bin/listinfo/rrd-developers
>
>

--
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
www.oetiker.ch [hidden email] +41 62 775 9902

_______________________________________________
rrd-developers mailing list
[hidden email]
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-developers



_______________________________________________
rrd-developers mailing list
[hidden email]
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-developers
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: RRDcached performance issues (from Users mailinglist)

Steve Schnepp
... Looking closely at your post shows that you are already implementing all my hints :-)

You can therefore ignore my email and focus on what Tobi said about "strace/truss".

Also could you setup the same system on a linux host (It would be useful to do differential analysis ) ?

Sorry about the noise ...



_______________________________________________
rrd-developers mailing list
[hidden email]
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-developers
Loading...