I'm not really a newbie to rrdtool, but this seems like a newbie question,
and it probably has been asked since the beginning of time, but I've never seen a good explanation of what's going on with the graph/consolidation functions. we have this problem and it seems obvious, but we can't seem to figure out how to fix it. We have a data set, a script collects data every two minutes and puts data in the database. we defined the data set to be 3 months of daata as follows: 93 days (3 months x 31  worst case) x 24 hours x 60 minutes x 60 seconds. so that's 3 months of data points for every second, even though we only use once every 2 minutes. now technically we have enough data points that creating a graph for 1 day or 1 week or 1 month, should have the ability to plot all the values. but it doesn't. for example, we do a 1 day graph and the "maximum" is 90 we do the same graph for 1 week and the "maximum" has magically changed to 30 do it again for 1 month, and it gets even worse. when we query the rrd database directly, we see that rrd gives us these values, so it's not the graphing function doing it, it's rrd. but when we check the raw data, we can see all the data points are there. if we have so many data points, it shouldn't be consolidating the data, but it is. and it's driving me crazy trying to figure out why. we thought of using separate data files for min, max and average, but triple the data storatge and triple the overhead is killing our server. can someone tell us what we're doing wrong or give us a suggestion so that the data will be reported properly? It seems I recall this issue, and there some some setting we could use, like ABS, but that doesn't seem to fit properly. thanks _________________________________________________________________ Show Your Messenger Buddies How You Really Feel http://www.freemessengeremoticons.ca/?icid=EMENCA122 
On Fri, Aug 31, 2007 at 01:07:44PM 0400, Dan Gahlinger wrote:
> for example, we do a 1 day graph and the "maximum" is 90 > we do the same graph for 1 week and the "maximum" has magically changed to > 30 > do it again for 1 month, and it gets even worse. An RRA stores average, minimum or maximum rates. GPRINT writes the average, minimum or maximum on the graph. If you are graphing data from an AVERAGE RRA, you can print the maximum of those averages, not the maximum out of another RRA. 1 step per row: 1,2,3,4,5,6,7,8,9,10,11,12 MAX(1,2,3,4,5,6,7,8,9,10,11,12) > 12 consolidation function average, 3 steps per entry: avg(1,2,3),avg(4,5,6),avg(7,8,9),avg(10,11,12) > 2,5,8,11 MAX(2,5,8,11) > 11, not 12. consolidation function average, 6 steps per entry: avg(1,2,3,4,5,6),avg(7,8,9,10,11,12) > 3.5 , 9.5 MAX(3.5,9.5) > 9.5, not 12 and so on. > but when we check the raw data, we can see all the data points are there. > if we have so many data points, it shouldn't be consolidating the data, but > it is. It shouldn't? Or perhaps you ask it to do so but don't want to? > and it's driving me crazy trying to figure out why. > we thought of using separate data files for min, max and average, but triple > the data storatge and triple the overhead is killing our server. > > can someone tell us what we're doing wrong or give us a suggestion so that > the data will be reported properly? Post the command you use to create your RRD file. Also post the relevant parts of your graph command. Keep it simple. We don't need to see all fancy formatting and such, a simplified example would be best, as long as it shows what you are talking about. HTH  Alex van den Bogaerdt http://www.vandenbogaerdt.nl/rrdtool/ 
I'll have to do this tomorrow or tuesday (monday is a holiday here),
and I don't have access to it at the moment. you may be correct that we are asking it to consolidate, and we don't want to. we simply don't know. it just seems, totally wrong. we used to have the RRD file setup for a years worth of data points, and it was still doing this. what's the point of defining so many data points if its going to consolidate it anyhow? we have learned it is NOT the graphing function doing it, though. if we look at the raw data it is correct, but if we query the data, we get the consolidated results. it shouldn't be consolidating. there is no reason that an RRD file created with 93 days of data points should start consolidating data after only one week  or less. there has to be something we're doing wrong. so i'll post it either later this weekend or at latest tuesday. >From: Alex van den Bogaerdt <[hidden email]> >To: [hidden email] >Subject: Re: [rrdusers] newbie rrd question  probably a very old issue >Date: Fri, 31 Aug 2007 19:35:57 +0200 > >On Fri, Aug 31, 2007 at 01:07:44PM 0400, Dan Gahlinger wrote: > > > for example, we do a 1 day graph and the "maximum" is 90 > > we do the same graph for 1 week and the "maximum" has magically changed >to > > 30 > > do it again for 1 month, and it gets even worse. > >An RRA stores average, minimum or maximum rates. >GPRINT writes the average, minimum or maximum on the graph. > >If you are graphing data from an AVERAGE RRA, you can print the >maximum of those averages, not the maximum out of another RRA. > > >1 step per row: 1,2,3,4,5,6,7,8,9,10,11,12 > >MAX(1,2,3,4,5,6,7,8,9,10,11,12) > 12 > >consolidation function average, 3 steps per entry: > >avg(1,2,3),avg(4,5,6),avg(7,8,9),avg(10,11,12) > 2,5,8,11 >MAX(2,5,8,11) > 11, not 12. > >consolidation function average, 6 steps per entry: >avg(1,2,3,4,5,6),avg(7,8,9,10,11,12) > 3.5 , 9.5 >MAX(3.5,9.5) > 9.5, not 12 > >and so on. > > > but when we check the raw data, we can see all the data points are >there. > > if we have so many data points, it shouldn't be consolidating the data, >but > > it is. > >It shouldn't? Or perhaps you ask it to do so but don't want to? > > > and it's driving me crazy trying to figure out why. > > we thought of using separate data files for min, max and average, but >triple > > the data storatge and triple the overhead is killing our server. > > > > can someone tell us what we're doing wrong or give us a suggestion so >that > > the data will be reported properly? > >Post the command you use to create your RRD file. Also post the >relevant parts of your graph command. > >Keep it simple. We don't need to see all fancy formatting and such, >a simplified example would be best, as long as it shows what you are >talking about. > >HTH > >Alex van den Bogaerdt >http://www.vandenbogaerdt.nl/rrdtool/ > >_______________________________________________ >rrdusers mailing list >[hidden email] >https://lists.oetiker.ch/cgibin/listinfo/rrdusers _________________________________________________________________ Windows Live Hotmail. Even hotter than before. Get a better look now. www.newhotmail.ca?icid=WLHMENCA148 
On Fri, Aug 31, 2007 at 10:06:24PM 0400, Dan Gahlinger wrote:
> you may be correct that we are asking it to consolidate, and we don't want > to. we simply don't know. > it just seems, totally wrong. If you have 400 pixels on the graph, and you have 4000 averages to display, it won't fit. Somehow you will have to combine 10 intervals into one. If you don't, then rrdtool will. > what's the point of defining so many data points if its going to > consolidate it anyhow? With many datarows you can look back in time at the highest possible resolution. You won't be using all of the data, just a portion of it. Consolidated data can be used to quickly report on a longer interval, without having to wait for rrdtool to do "consolidation" on the fly. No need to reduce 4000 rows into 400, if you already have the data available in a more appropriate resolution. They both serve a purpose. > we have learned it is NOT the graphing function doing it, though. then it has to be another available RRA. > if we look at the raw data it is correct, but if we query the data, we get > the consolidated results. > it shouldn't be consolidating. Then you shouldn't have defined that RRA. More likely you should have created another RRA, or if you did, you should be using it. If you want to see maxima, then you should use the MAX RRA, not the AVERAGE RRA. (capitals not for shouting, but because that's how it's written!)  Alex van den Bogaerdt http://www.vandenbogaerdt.nl/rrdtool/ 
I think you're missing the point.
true, the graph only has so much space, but I call "BS" on your overall explanation for 3 reasons. #1 we don't have that MANY points to plot  so there's more than enough space on the graph to show everything #2  even so it shouldn't be changing the actual points it does show. a peak of 90 should be a peak of 90 regardless, it shouldn't shift it to a peak of 30, no matter what the numbers are.  if you can see the point on the graph, it should be the same value. #3  using "max" instead of, or in addition to "avg" for RRA is a bogus answer. if we use "max" instead of "Avg" we lose the "min", and the "avg" is skewed. if we use it in addition, we have to define, "min", "avg" and "max"  three RRA's for one data set, which kills the load on the server. imagine 1000 service checks having to write 3x the data instead of one. this isn't the answer. the answer is to find out why RRD/RRA is doing what its doing and make it stop. in our case, we only had 2 days worth of data, actually less than 1000 bits of data. so on the weekly graph there are enough points. on the graph we can clearly see the "90" spike on the daily graph, but the weekly graph, that days spike has been changed to "30" and is plotted as 30. this is very bad. it shouldn't happen. but again, it's not the graph function that's doing it. with the command line query, it produces the same results. the graph is graphing it that way because RRD is giving it the data that way. RRD command line query for the week retrieves the max value as "30" even though the raw data shows the "90" is there. this is definitely broken. and badly. I'm not sure what's wrong with it. again, it could be our fault in how the RRA is setup, but I don't see how. I'll post that as soon as I can. >From: Alex van den Bogaerdt <[hidden email]> >To: [hidden email] >Subject: Re: [rrdusers] newbie rrd question  probably a very old issue >Date: Sat, 1 Sep 2007 13:10:30 +0200 > >On Fri, Aug 31, 2007 at 10:06:24PM 0400, Dan Gahlinger wrote: > > > you may be correct that we are asking it to consolidate, and we don't >want > > to. we simply don't know. > > it just seems, totally wrong. > >If you have 400 pixels on the graph, and you have 4000 averages to display, >it won't fit. Somehow you will have to combine 10 intervals into one. >If you don't, then rrdtool will. > > > what's the point of defining so many data points if its going to > > consolidate it anyhow? > >With many datarows you can look back in time at the highest possible >resolution. You won't be using all of the data, just a portion of it. > >Consolidated data can be used to quickly report on a longer interval, >without having to wait for rrdtool to do "consolidation" on the fly. >No need to reduce 4000 rows into 400, if you already have the data >available in a more appropriate resolution. > >They both serve a purpose. > > > > we have learned it is NOT the graphing function doing it, though. > >then it has to be another available RRA. > > > if we look at the raw data it is correct, but if we query the data, we >get > > the consolidated results. > > it shouldn't be consolidating. > >Then you shouldn't have defined that RRA. > >More likely you should have created another RRA, or if you did, you should >be using it. If you want to see maxima, then you should use the MAX RRA, >not the AVERAGE RRA. >(capitals not for shouting, but because that's how it's written!) > > >Alex van den Bogaerdt >http://www.vandenbogaerdt.nl/rrdtool/ > >_______________________________________________ >rrdusers mailing list >[hidden email] >https://lists.oetiker.ch/cgibin/listinfo/rrdusers _________________________________________________________________ Enter to win a night a VIP night out at TIFF http://redcarpet.sympatico.msn.ca/ 
Administrator

Dear Dan,
> I think you're missing the point. just take a look at the number of posts alex makes. he does understand how rrdtool works, he know the code that makes it tick ... so if you tell him he is missing the point, chances are that he will not bother anymore help you understand how rrdtool works ... to your question. RRDtool tries hard to work correctly independant of the number of points it has to work with. So your graph, should turn out the same results, regardles of the number of samples that are available (withing reason). This means for example, that if you input data from an RRA that is using AVERAGE consolidation, that the graph will also use AVERAGE consolidation to reduce the data until it fits in the graph. If you use a MAX function over this, the MAX will return what you see in the graph. If you are interested in keeping the MAX of the data you have originally fed into RRDtool, you must store it in an RRA that has a MAX function attached. A little example: You have to 2 points in your RRA a: 10 b: 90 On the graph they map onto the same pixel. Depending on the consolidation function of the RRA, rrdtool will draw the pixle at the following positions avg: 50 min: 10 max: 90 on top of this you can use VDEF to calculate, but obviously the results will depend on the first round of consolidation that has already happened. >From a performance point of view, given your box has enough memory, you should be able to get about * 200400 RRA updates a second per spindle Note that you can cheat this by increassing the amount of dirty buffers the OS tolerates as well as the ammount of time the dirty buffers can stay dirty. Especially if you run frequent updates this can be a neat strategy to increasse throughput dramatically (in cache updates, rrdtool 1.3 can do about 20k per second on a current box.) cheers tobi  Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH4600 Olten http://it.oetiker.ch [hidden email] ++41 62 213 9902 
Tobias,
yes, we are just getting rather frustrated with why this is happening. I hope you and Alex can understand. and I have said in every post it is likely something we're doing, we just don't know what. we can't see why with what we've done  how it's doing this. it's going to be some small stupid thing we missed, that's my gut feeling. we've eliminated the graph, which is good. we are trying to understand how the raw data shows one value and query gives us another  using the command line functions. I used RRD years ago, and became quite adept at it, I seem to recall we had this exact same issue and fixed it, but then I didn't use RRD for many years, and I forgot these details. so hopefully Alex can understand the frustration, and I'll definitely post the details when I can i appreciate the help, in the meantime i'll look into recreating this on my local system, if I can do that with a smaller data set we have a much better chance of finding the issue. >From: Tobias Oetiker <[hidden email]> >To: Dan Gahlinger <[hidden email]> >CC: [hidden email], [hidden email] >Subject: Re: [rrdusers] newbie rrd question  probably a very old issue >Date: Sat, 1 Sep 2007 17:14:33 +0200 (CEST) > >Dear Dan, > > > I think you're missing the point. > >just take a look at the number of posts alex makes. he does >understand how rrdtool works, he know the code that makes it tick >... so if you tell him he is missing the point, chances are that he >will not bother anymore help you understand how rrdtool works ... > >to your question. > >RRDtool tries hard to work correctly independant of the number of >points it has to work with. So your graph, should turn out the same >results, regardles of the number of samples that are available >(withing reason). > >This means for example, that if you input data from an RRA that is >using AVERAGE consolidation, that the graph will also use AVERAGE >consolidation to reduce the data until it fits in the graph. If you >use a MAX function over this, the MAX will return what you see in >the graph. > >If you are interested in keeping the MAX of the data you have >originally fed into RRDtool, you must store it in an RRA that has a >MAX function attached. > >A little example: > >You have to 2 points in your RRA > >a: 10 >b: 90 > >On the graph they map onto the same pixel. Depending on the >consolidation function of the RRA, rrdtool will draw the pixle at >the following positions > >avg: 50 >min: 10 >max: 90 > >on top of this you can use VDEF to calculate, but obviously the >results will depend on the first round of consolidation that has >already happened. > >From a performance point of view, given your box has enough memory, >you should be able to get about > > * 200400 RRA updates a second per spindle > >Note that you can cheat this by increassing the amount of dirty >buffers the OS tolerates as well as the ammount of time the dirty >buffers can stay dirty. > >Especially if you run frequent updates this can be a neat strategy >to increasse throughput dramatically (in cache updates, rrdtool 1.3 >can do about 20k per second on a current box.) > >cheers >tobi > > >Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH4600 Olten >http://it.oetiker.ch [hidden email] ++41 62 213 9902 _________________________________________________________________ Show Your Messenger Buddies How You Really Feel http://www.freemessengeremoticons.ca/?icid=EMENCA122 
In reply to this post by Dan Gahlinger3
> Original Message > From: [hidden email] [mailto:rrdusers > [hidden email]] On Behalf Of Dan Gahlinger > Sent: Saturday, September 01, 2007 9:31 AM > To: [hidden email]; [hidden email] > Subject: Re: [rrdusers] newbie rrd question  probably a very old issue > > > in our case, we only had 2 days worth of data, actually less than 1000 > bits > of data. > so on the weekly graph there are enough points. > > on the graph we can clearly see the "90" spike on the daily graph, but the > weekly graph, that days spike has been changed to "30" and is plotted as > 30. > this is very bad. it shouldn't happen. Look at it this way. Assuming 5 minute samples, there are 288 data points in 1 day. On a 300 pixel wide graph, you have enough space to graph each single data point and then some. Now you ask rrdtool to graph a weeks worth of data points in a 300 pixel wide graph. There are 2016 possible data points from which it must choose only 300 to display. How should it do this? RRDTool uses the best fitting RRA and the consolidation function you specify (likely AVERAGE). The fact that you only have two days of data doesn't mean RRDTool is going to change the way it creates a weekly graph. You've told it to graph a weeks worth of data so it'll do the best it can to show you a weeks worth of data, NAN or not. You only had two days worth of data but still asked rrdtool to show 7 days worth. It will, properly, consolidate what you have to fit the two days and show 5 days worth of NAN data.  Marc 
what about the commandline query function?
there is no "space limitation" with that. why is that consolidating? and I still disagree with the analysis. even as you say, it has to shrink it to fit, that doesn't mean it should be averaging out the maximum (or the minimum) and adjusting the graph. you say there isn't enough space, I disagree. take a look at a "Nasdaq" graph for comparison. whether you do a day, week, month, 6 months, or a year, the graph is the same size with the same amount of available space, yet it shows every minimum, and every maximum correctly. it never averages those out. and all on one graph. I'll post a link to some examples if you would like to compare  not sure it can relate directly though. the point is, it is possible to do, so rrd should also be doing this. but even more to the point, the commandline tools are consolidating  and those have infinite space, so they definitely should not be consolidating. can you explain why the command line tool is doing the same thing? I'd really like an explanation for that. Dan. >From: "Marc Powell" <[hidden email]> >To: <[hidden email]> >Subject: Re: [rrdusers] newbie rrd question  probably a very old issue >Date: Sat, 1 Sep 2007 10:36:20 0500 > > > > > Original Message > > From: [hidden email] [mailto:rrdusers > > [hidden email]] On Behalf Of Dan Gahlinger > > Sent: Saturday, September 01, 2007 9:31 AM > > To: [hidden email]; [hidden email] > > Subject: Re: [rrdusers] newbie rrd question  probably a very old >issue > > > > > > > > in our case, we only had 2 days worth of data, actually less than 1000 > > bits > > of data. > > so on the weekly graph there are enough points. > > > > on the graph we can clearly see the "90" spike on the daily graph, but >the > > weekly graph, that days spike has been changed to "30" and is plotted >as > > 30. > > this is very bad. it shouldn't happen. > >Look at it this way. Assuming 5 minute samples, there are 288 data >points in 1 day. On a 300 pixel wide graph, you have enough space to >graph each single data point and then some. Now you ask rrdtool to graph >a weeks worth of data points in a 300 pixel wide graph. There are 2016 >possible data points from which it must choose only 300 to display. How >should it do this? RRDTool uses the best fitting RRA and the >consolidation function you specify (likely AVERAGE). > >The fact that you only have two days of data doesn't mean RRDTool is >going to change the way it creates a weekly graph. You've told it to >graph a weeks worth of data so it'll do the best it can to show you a >weeks worth of data, NAN or not. You only had two days worth of data but >still asked rrdtool to show 7 days worth. It will, properly, consolidate >what you have to fit the two days and show 5 days worth of NAN data. > > >Marc > >_______________________________________________ >rrdusers mailing list >[hidden email] >https://lists.oetiker.ch/cgibin/listinfo/rrdusers _________________________________________________________________ Get the device you want, with the HotmailÂ® you love. http://www.windowsmobile.com/hotmailmobile?ocid=MobileHMTagline_1 
In reply to this post by oetiker
Well here it is, I managed to get our definition  from the 1 year of data
version create rrd_file start 0 step 120 "DS:a:GAUGE:360:U:U","RRA:AVERAGE:0.5:1:12*31*24*3600/120" sorry its cut from the perl script that does it, but that's the definition the 3 months version is just 3*31*24*3600/120 instead of the above. again in a sense, the graph being "wrong" makes sense, because the command line query gives the same results. if we can solve why the command line query gives the result it does, I think we'll have our answer. _________________________________________________________________ Share More On Messenger with a Windows Live Space http://spaces.live.com/?mkt=enca 
Based on a side conversation with Dan, this is what I'm understanding:
RRD will graph exactly one consolidated point in time, if there are more points than pixels on the X axis. Dan wants *all* value points represented, not consolidated. RRD has no option to do that. I'm thinking that *maybe* it can be faked, with some stacked graphs or something, *if* MIN and MAX were also included in the rrd file; do a fill between min and max. But it isn't going to happen with a single graph statement, and I'm having problems figuring out how to sanely show min/average/max if desired. :) 
that's not exactly what I want. to be honest, I'd be happy if rrd did one simple thing (or two depending on your point of view). I'd recommend the "min" and "max" of the average function be done a bit differently. for an average graph, the "maximum" (shown at the bottom) should always be kept, and the "minimum" should always be kept also the points for those on the graph should always be kept, even if it means the verticals in the graph are stretched a bit. the graph doesnt have to show ever portion of the scale either. by this I mean if there are no values between 10 and 40, you dont need to show the portions from 1139 essentially as long as the maximum, minimum and average are represented properly it should be simple enough to keep track of the minimum and maximum over time. my logic: if you have a spike of 100 one day and that is you maximum, then a week or a month or a year later, if that is still your maximum, then it is always your maximum, and the spike should be shown today it should be the max, tomorrow it should be the max, and until a higher value occurs, it should always be the max, same with minimum, if we got a zero, that will always be the minimum. and the overall average should be the average between those two values. otherwise the data is incorrect and quickly becomes meaningless. yesterday it spiked to 90, but today it shows the max is only 30, so did it spike? not according to the graph it didnt. and I still dont understand why the command line (rrdfetch) is consolidating, it has no graphing limitations I don't know a better way to do it. Dan. >From: Jason Fesler <[hidden email]> >To: Dan Gahlinger <[hidden email]> >CC: [hidden email], [hidden email] >Subject: Re: [rrdusers] newbie rrd question  probably a very old issue >Date: Sat, 1 Sep 2007 10:09:32 0700 (PDT) > >Based on a side conversation with Dan, this is what I'm understanding: > >RRD will graph exactly one consolidated point in time, if there are more >points than pixels on the X axis. Dan wants *all* value points >represented, not consolidated. RRD has no option to do that. > >I'm thinking that *maybe* it can be faked, with some stacked graphs or >something, *if* MIN and MAX were also included in the rrd file; do a fill >between min and max. > >But it isn't going to happen with a single graph statement, and I'm having >problems figuring out how to sanely show min/average/max if desired. :) > _________________________________________________________________ Show Your Messenger Buddies How You Really Feel http://www.freemessengeremoticons.ca/?icid=EMENCA122 
Administrator

In reply to this post by Jason Fesler
Jason,
if you want to have this, you create TWO rras, one with MIN and one with MAX consolidation. in the graph you draw MIN and stack MAXMIN on top of it and you get that 'magic' graph dans seems to be looking for ... it will look odd, since it will change its 'width' depending on the distance between min and max, but I can't see how he expects this to work any other way ... I wonder what his nsdaq graph does if in a single day there is an extreem high and and extreem low, but there is only a single pixel in the graph to represent that day ... cheers tobi Today Jason Fesler wrote: > Based on a side conversation with Dan, this is what I'm understanding: > > RRD will graph exactly one consolidated point in time, if there are more > points than pixels on the X axis. Dan wants *all* value points represented, > not consolidated. RRD has no option to do that. > > I'm thinking that *maybe* it can be faked, with some stacked graphs or > something, *if* MIN and MAX were also included in the rrd file; do a fill > between min and max. > > But it isn't going to happen with a single graph statement, and I'm having > problems figuring out how to sanely show min/average/max if desired. :) > >  Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH4600 Olten http://it.oetiker.ch [hidden email] ++41 62 213 9902 
Administrator

In reply to this post by Dan Gahlinger3
Dan,
note that rrdfetch is NOT consolidating anything, it shows you what is stored in the rrd ... if you want to see what is actually stored, use rrdtool dump ... you may also want to read through the mailinglist archives, since we get someone like you every few months how needs many days of patient explaining until he understand why rrdtool is designed the way it is designed and why it works the way it works, and why this is the right thing todo ... (it has been like this for the last 8 years actually and tousands have used and understood the logic) ... you will get there ... cheers tobi > > that's not exactly what I want. > > to be honest, I'd be happy if rrd did one simple thing (or two depending on > your point of view). > > I'd recommend the "min" and "max" of the average function be done a bit > differently. > for an average graph, the "maximum" (shown at the bottom) should always be > kept, > and the "minimum" should always be kept > > also the points for those on the graph should always be kept, even if it means > the verticals in the graph are stretched a bit. > > the graph doesnt have to show ever portion of the scale either. by this I mean > if there are no values between 10 and 40, you dont need to show the portions > from 1139 essentially > as long as the maximum, minimum and average are represented properly > > it should be simple enough to keep track of the minimum and maximum over time. > my logic: > if you have a spike of 100 one day and that is you maximum, then a week or a > month or a year later, if that is still your maximum, then it is always your > maximum, and the spike should be shown > today it should be the max, tomorrow it should be the max, and until a higher > value occurs, it should always be the max, same with minimum, if we got a > zero, that will always be the minimum. and the overall average should be the > average between those two values. > > otherwise the data is incorrect and quickly becomes meaningless. > yesterday it spiked to 90, but today it shows the max is only 30, so did it > spike? not according to the graph it didnt. > > and I still dont understand why the command line (rrdfetch) is consolidating, > it has no graphing limitations > > I don't know a better way to do it. > Dan. > > From: Jason Fesler <[hidden email]> > > To: Dan Gahlinger <[hidden email]> > > CC: [hidden email], [hidden email] > > Subject: Re: [rrdusers] newbie rrd question  probably a very old issue > > Date: Sat, 1 Sep 2007 10:09:32 0700 (PDT) > > > > Based on a side conversation with Dan, this is what I'm understanding: > > > > RRD will graph exactly one consolidated point in time, if there are more > > points than pixels on the X axis. Dan wants *all* value points represented, > > not consolidated. RRD has no option to do that. > > > > I'm thinking that *maybe* it can be faked, with some stacked graphs or > > something, *if* MIN and MAX were also included in the rrd file; do a fill > > between min and max. > > > > But it isn't going to happen with a single graph statement, and I'm having > > problems figuring out how to sanely show min/average/max if desired. :) > > > > _________________________________________________________________ > Show Your Messenger Buddies How You Really Feel > http://www.freemessengeremoticons.ca/?icid=EMENCA122 > >  Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH4600 Olten http://it.oetiker.ch [hidden email] ++41 62 213 9902 
In reply to this post by oetiker
check it out for yourself.
a company everyone hates as the reference here's a year NASDAQ graph showing highs and lows with points between and no loss of accuracy http://charting.nasdaq.com/ext/charts.dll?21140051203NA000000SCOX&SF:45WD=484HT=395 if that doesnt work (for whatever reason) use this: http://quotes.nasdaq.com/quote.dll?page=charting&mode=basics&symbol=SCOX%60&selected=SCOX%60 >From: Tobias Oetiker <[hidden email]> >To: Jason Fesler <[hidden email]> >CC: Dan Gahlinger <[hidden email]>, [hidden email] >Subject: Re: [rrdusers] newbie rrd question  probably a very old issue >Date: Sat, 1 Sep 2007 20:45:19 +0200 (CEST) > >Jason, > >if you want to have this, you create TWO rras, one with MIN and one >with MAX consolidation. > >in the graph you draw > >MIN and stack MAXMIN on top of it and you get that 'magic' graph >dans seems to be looking for ... it will look odd, since it will >change its 'width' depending on the distance between min and max, >but I can't see how he expects this to work any other way ... > >I wonder what his nsdaq graph does if in a single day there is an >extreem high and and extreem low, but there is only a single pixel >in the graph to represent that day ... > >cheers >tobi > >Today Jason Fesler wrote: > > > Based on a side conversation with Dan, this is what I'm understanding: > > > > RRD will graph exactly one consolidated point in time, if there are more > > points than pixels on the X axis. Dan wants *all* value points >represented, > > not consolidated. RRD has no option to do that. > > > > I'm thinking that *maybe* it can be faked, with some stacked graphs or > > something, *if* MIN and MAX were also included in the rrd file; do a >fill > > between min and max. > > > > But it isn't going to happen with a single graph statement, and I'm >having > > problems figuring out how to sanely show min/average/max if desired. :) > > > > > > >Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH4600 Olten >http://it.oetiker.ch [hidden email] ++41 62 213 9902 _________________________________________________________________ See Fireworks On Live Image Search http://search.live.com/images/results.aspx?q=Fireworks&mkt=enca&FORM=SERNEP 
In reply to this post by oetiker
Sounds like this is in dire need of a FAQ! :)
so why does rrd dump and rrd fetch show different results? I'll check the "resolution" option before continuing that question. and I could swear I "fixed" this problem with a simple modification to the RRA definition. I'm almost positive it had something to do with "ABS", but that was 5 years ago. >From: Tobias Oetiker <[hidden email]> >To: Dan Gahlinger <[hidden email]> >CC: [hidden email], [hidden email] >Subject: Re: [rrdusers] newbie rrd question  probably a very old issue >Date: Sat, 1 Sep 2007 20:49:26 +0200 (CEST) > >Dan, > >note that rrdfetch is NOT consolidating anything, it shows you what >is stored in the rrd ... if you want to see what is actually >stored, use rrdtool dump ... > >you may also want to read through the mailinglist archives, since >we get someone like you every few months how needs many days of >patient explaining until he understand why rrdtool is designed the >way it is designed and why it works the way it works, and why this >is the right thing todo ... (it has been like this for the last 8 >years actually and tousands have used and understood the logic) ... >you will get there ... > >cheers >tobi > > > > > > that's not exactly what I want. > > > > to be honest, I'd be happy if rrd did one simple thing (or two depending >on > > your point of view). > > > > I'd recommend the "min" and "max" of the average function be done a bit > > differently. > > for an average graph, the "maximum" (shown at the bottom) should always >be > > kept, > > and the "minimum" should always be kept > > > > also the points for those on the graph should always be kept, even if it >means > > the verticals in the graph are stretched a bit. > > > > the graph doesnt have to show ever portion of the scale either. by this >I mean > > if there are no values between 10 and 40, you dont need to show the >portions > > from 1139 essentially > > as long as the maximum, minimum and average are represented properly > > > > it should be simple enough to keep track of the minimum and maximum over >time. > > my logic: > > if you have a spike of 100 one day and that is you maximum, then a week >or a > > month or a year later, if that is still your maximum, then it is always >your > > maximum, and the spike should be shown > > today it should be the max, tomorrow it should be the max, and until a >higher > > value occurs, it should always be the max, same with minimum, if we got >a > > zero, that will always be the minimum. and the overall average should be >the > > average between those two values. > > > > otherwise the data is incorrect and quickly becomes meaningless. > > yesterday it spiked to 90, but today it shows the max is only 30, so did >it > > spike? not according to the graph it didnt. > > > > and I still dont understand why the command line (rrdfetch) is >consolidating, > > it has no graphing limitations > > > > I don't know a better way to do it. > > Dan. > > > From: Jason Fesler <[hidden email]> > > > To: Dan Gahlinger <[hidden email]> > > > CC: [hidden email], [hidden email] > > > Subject: Re: [rrdusers] newbie rrd question  probably a very old >issue > > > Date: Sat, 1 Sep 2007 10:09:32 0700 (PDT) > > > > > > Based on a side conversation with Dan, this is what I'm understanding: > > > > > > RRD will graph exactly one consolidated point in time, if there are >more > > > points than pixels on the X axis. Dan wants *all* value points >represented, > > > not consolidated. RRD has no option to do that. > > > > > > I'm thinking that *maybe* it can be faked, with some stacked graphs or > > > something, *if* MIN and MAX were also included in the rrd file; do a >fill > > > between min and max. > > > > > > But it isn't going to happen with a single graph statement, and I'm >having > > > problems figuring out how to sanely show min/average/max if desired. >:) > > > > > > > _________________________________________________________________ > > Show Your Messenger Buddies How You Really Feel > > http://www.freemessengeremoticons.ca/?icid=EMENCA122 > > > > > > >Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH4600 Olten >http://it.oetiker.ch [hidden email] ++41 62 213 9902 _________________________________________________________________ Kick back and relax with hot games and cool activities at the Messenger CafĂ©. http://www.cafemessenger.com?ocid=TXT_TAGHM_SeptHMtagline1 
Administrator

Dan,
> Sounds like this is in dire need of a FAQ! :) yes, unfortunately eveyone seems to come from a diffferent angle ... but once you got it down, it would be great if you could write up what you understood ... > so why does rrd dump and rrd fetch show different results? > I'll check the "resolution" option before continuing that question. rrdtool dump shows the whole rrd file rrdtool fetch picks one RRA and shows you the data from that RRA cheers tobi > and I could swear I "fixed" this problem with a simple modification to the RRA > definition. > I'm almost positive it had something to do with "ABS", but that was 5 years > ago. > > > > From: Tobias Oetiker <[hidden email]> > > To: Dan Gahlinger <[hidden email]> > > CC: [hidden email], [hidden email] > > Subject: Re: [rrdusers] newbie rrd question  probably a very old issue > > Date: Sat, 1 Sep 2007 20:49:26 +0200 (CEST) > > > > Dan, > > > > note that rrdfetch is NOT consolidating anything, it shows you what > > is stored in the rrd ... if you want to see what is actually > > stored, use rrdtool dump ... > > > > you may also want to read through the mailinglist archives, since > > we get someone like you every few months how needs many days of > > patient explaining until he understand why rrdtool is designed the > > way it is designed and why it works the way it works, and why this > > is the right thing todo ... (it has been like this for the last 8 > > years actually and tousands have used and understood the logic) ... > > you will get there ... > > > > cheers > > tobi > > > > > > > > > > that's not exactly what I want. > > > > > > to be honest, I'd be happy if rrd did one simple thing (or two depending > > on > > > your point of view). > > > > > > I'd recommend the "min" and "max" of the average function be done a bit > > > differently. > > > for an average graph, the "maximum" (shown at the bottom) should always be > > > kept, > > > and the "minimum" should always be kept > > > > > > also the points for those on the graph should always be kept, even if it > > means > > > the verticals in the graph are stretched a bit. > > > > > > the graph doesnt have to show ever portion of the scale either. by this I > > mean > > > if there are no values between 10 and 40, you dont need to show the > > portions > > > from 1139 essentially > > > as long as the maximum, minimum and average are represented properly > > > > > > it should be simple enough to keep track of the minimum and maximum over > > time. > > > my logic: > > > if you have a spike of 100 one day and that is you maximum, then a week or > > a > > > month or a year later, if that is still your maximum, then it is always > > your > > > maximum, and the spike should be shown > > > today it should be the max, tomorrow it should be the max, and until a > > higher > > > value occurs, it should always be the max, same with minimum, if we got a > > > zero, that will always be the minimum. and the overall average should be > > the > > > average between those two values. > > > > > > otherwise the data is incorrect and quickly becomes meaningless. > > > yesterday it spiked to 90, but today it shows the max is only 30, so did > > it > > > spike? not according to the graph it didnt. > > > > > > and I still dont understand why the command line (rrdfetch) is > > consolidating, > > > it has no graphing limitations > > > > > > I don't know a better way to do it. > > > Dan. > > > > From: Jason Fesler <[hidden email]> > > > > To: Dan Gahlinger <[hidden email]> > > > > CC: [hidden email], [hidden email] > > > > Subject: Re: [rrdusers] newbie rrd question  probably a very old issue > > > > Date: Sat, 1 Sep 2007 10:09:32 0700 (PDT) > > > > > > > > Based on a side conversation with Dan, this is what I'm understanding: > > > > > > > > RRD will graph exactly one consolidated point in time, if there are more > > > > points than pixels on the X axis. Dan wants *all* value points > > represented, > > > > not consolidated. RRD has no option to do that. > > > > > > > > I'm thinking that *maybe* it can be faked, with some stacked graphs or > > > > something, *if* MIN and MAX were also included in the rrd file; do a > > fill > > > > between min and max. > > > > > > > > But it isn't going to happen with a single graph statement, and I'm > > having > > > > problems figuring out how to sanely show min/average/max if desired. :) > > > > > > > > > > _________________________________________________________________ > > > Show Your Messenger Buddies How You Really Feel > > > http://www.freemessengeremoticons.ca/?icid=EMENCA122 > > > > > > > > > >  > > Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH4600 Olten > > http://it.oetiker.ch [hidden email] ++41 62 213 9902 > > _________________________________________________________________ > Kick back and relax with hot games and cool activities at the Messenger Caf?? > http://www.cafemessenger.com?ocid=TXT_TAGHM_SeptHMtagline1 > >  Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH4600 Olten http://it.oetiker.ch [hidden email] ++41 62 213 9902 
Free forum by Nabble  Edit this page 