Quantisation issues - how do i get the math right?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Quantisation issues - how do i get the math right?

Andreas Schuldei
drraw.png
This is a graph i get with my current intake of sun and my setup, consisting of thermo sensors on in- and outgoing water pipes, as well as a water meter, emitting S0 impulses, indicating that 10l of fluid were pumped. 
My Issue is with dealing with the impulses in a sensible way. The interval of these impulses varies, depending on the sun intake. 

currently my CDEF for the solar power is
d,c,-,a,*,10,*,4.190,*
where c and d are my temperatures, and a is my counter, indicating 10 liters. (4.190 is the specific heat constant of water.)

Now what is calculated by that CDEF is not really the solar power, its the momentary value, only, calculated for the duration of the impulse, which is the width of my sampling rate (which is 10sec). it would have been more accurate to take the time between the impulses and calculate the incoming energy during that period, divided by the duration of that time and graph that for the previous (or following) interval. 

How can I do that? 

_______________________________________________
rrd-users mailing list
[hidden email]
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
Reply | Threaded
Open this post in threaded view
|

Re: Quantisation issues - how do i get the math right?

oetiker
Administrator
if you have 'pulses' use the ABSOLUTE DS type ... 

and the multiply the result by whatever a pulse is worth

cheers
tobi


----- On Mar 10, 2017, at 2:46 PM, Andreas Schuldei <[hidden email]> wrote:
drraw.png
This is a graph i get with my current intake of sun and my setup, consisting of thermo sensors on in- and outgoing water pipes, as well as a water meter, emitting S0 impulses, indicating that 10l of fluid were pumped. 
My Issue is with dealing with the impulses in a sensible way. The interval of these impulses varies, depending on the sun intake. 

currently my CDEF for the solar power is
d,c,-,a,*,10,*,4.190,*
where c and d are my temperatures, and a is my counter, indicating 10 liters. (4.190 is the specific heat constant of water.)

Now what is calculated by that CDEF is not really the solar power, its the momentary value, only, calculated for the duration of the impulse, which is the width of my sampling rate (which is 10sec). it would have been more accurate to take the time between the impulses and calculate the incoming energy during that period, divided by the duration of that time and graph that for the previous (or following) interval. 

How can I do that? 

_______________________________________________
rrd-users mailing list
[hidden email]
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

--
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
www.oetiker.ch [hidden email] +41 62 775 9902

_______________________________________________
rrd-users mailing list
[hidden email]
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
Reply | Threaded
Open this post in threaded view
|

Re: Quantisation issues - how do i get the math right?

oetiker
Administrator
Hi Andreas

ah it is a counter ... then you use COUNTER ... I thought you had something detecting events ... then you could use update to 'load' and event into the rrd.

cheers
tobi

----- On Mar 10, 2017, at 4:37 PM, Andreas Schuldei <[hidden email]> wrote:
I changed that and am puzzled. 
how do i deal with the huge offset? In that setup, the value the device returns is ~8000, and its a 64bit counter...

in an other thread (https://lists.oetiker.ch/pipermail/rrd-users/2014-September/020015.html) that seems to deal with a similar issue,  someone recommend:

"Alternatively, you could use Absolute and reset the counter each time - then you don't need to worry about storing the total." 

I can not reset the counter in the device. Or is that some rrdtool counter that i can reset with some clever CDEF magic?

And how will the data, which "absolute" would give me, be usable in my case? from the documentation (https://oss.oetiker.ch/rrdtool/tut/rrd-beginners.en.html) i understand i would get an slowly ascending staircase. But in what way can i use that to emulate my incoming power? ever increasing power from my solar panels would be awesome, but that is not what I experienced before using rrdtool :-)


On Fri, Mar 10, 2017 at 3:10 PM Tobias Oetiker <[hidden email]> wrote:
if you have 'pulses' use the ABSOLUTE DS type ... 

and the multiply the result by whatever a pulse is worth

cheers
tobi


----- On Mar 10, 2017, at 2:46 PM, Andreas Schuldei <[hidden email]> wrote:

This is a graph i get with my current intake of sun and my setup, consisting of thermo sensors on in- and outgoing water pipes, as well as a water meter, emitting S0 impulses, indicating that 10l of fluid were pumped. 
My Issue is with dealing with the impulses in a sensible way. The interval of these impulses varies, depending on the sun intake. 

currently my CDEF for the solar power is
d,c,-,a,*,10,*,4.190,*
where c and d are my temperatures, and a is my counter, indicating 10 liters. (4.190 is the specific heat constant of water.)

Now what is calculated by that CDEF is not really the solar power, its the momentary value, only, calculated for the duration of the impulse, which is the width of my sampling rate (which is 10sec). it would have been more accurate to take the time between the impulses and calculate the incoming energy during that period, divided by the duration of that time and graph that for the previous (or following) interval. 

How can I do that? 

_______________________________________________
rrd-users mailing list
[hidden email]
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

--
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
www.oetiker.ch [hidden email] +41 62 775 9902


--
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
www.oetiker.ch [hidden email] +41 62 775 9902

_______________________________________________
rrd-users mailing list
[hidden email]
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
Reply | Threaded
Open this post in threaded view
|

Re: Quantisation issues - how do i get the math right?

Andreas Schuldei
In reply to this post by oetiker

I changed that and am puzzled. 
how do i deal with the huge offset? In that setup, the value the device returns is ~8000, and its a 64bit ever increasing counter...

In an other thread (https://lists.oetiker.ch/pipermail/rrd-users/2014-September/020015.html) that seems to deal with a similar issue,  someone recommend:

"Alternatively, you could use Absolute and reset the counter each time - then you don't need to worry about storing the total." 

I can not reset the counter in the device. Or does that person refer to some rrdtool counter that I can reset with some clever CDEF magic?

And how will the data, which "absolute" would give me, be usable in my case? From the documentation (https://oss.oetiker.ch/rrdtool/tut/rrd-beginners.en.html) I understand that I would get an slowly ascending staircase. But in what way can i use that to emulate my incoming power? Ever increasing power from my solar panels would be awesome, but that is not what I experienced before using rrdtool :-)

I just got an idea that there might be some confusion about the word "Power". 

I am interested in both the engergy (Energie) and the power (Leistung) output. Currently I am dealing with the power graph. Energy is the sum (or integral over time) of the power. So ABSOLUTE (implemented as an ever increasing staircase) would emulate the absorbed energy, but not the power.  



_______________________________________________
rrd-users mailing list
[hidden email]
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
Reply | Threaded
Open this post in threaded view
|

Re: Quantisation issues - how do i get the math right?

Simon Hobson
Andreas Schuldei <[hidden email]> wrote:

> I just got an idea that there might be some confusion about the word "Power".
>
> I am interested in both the engergy (Energie) and the power (Leistung) output. Currently I am dealing with the power graph. Energy is the sum (or integral over time) of the power. So ABSOLUTE (implemented as an ever increasing staircase) would emulate the absorbed energy, but not the power.  

Just to be clear, RRD **ONLY* does rates - so power flow. To get totals (ie accumulated energy) you multiply the rate (where appropriate, an averaged value) by time. SO whatever you feed in, you need to arrange that it's a rate.

In your case, I think you have no option other than to use the counter as it is, and either multiply or divide by an appropriate factor to get the power instead of some arbitrary number of "somethings".
In your case, I suspect the best you'll get is to store : flow rate (f) and treat the values you read as a counter, t1 and t2, and use derived/computed DSs for "t2-t1" (temperature difference) if that might be useful in future, and "f*(t2-t1)*k" (where f is the flow rate and k is the "fudge factor" to store actual power (energy flow rate). It's going to be very uneven due to the chunkiness of the data updates you can feed in - there isn't really anything you can do about that. Once you've done some aggregation (eg to daily figures) then this lumpiness should reduce. However, you will get false data for MAX values.

Simple answer - if you only get some data in big chunks, there's not much you can do to make it "not chunky". IIRC it's come up before, possibly in relation to a weather station and a coarse reading from a rain gauge.
The only other thing you could do is simply not look at fine grained data. If (say) on a typical day the counter might step (say) once in half an hour - then have your step at (say) 2 hours. You'll never be able to look at data finer grained than 2 hours. But your normalised data will vary between (say) 3 per 2 hours when flow is down, to (say) 6 per 2 hours when it's higher. Better than it doing zero, zero, zero, <some large number>, zero, zero, ...

But you will still want to feed in data fairly regularly, because you'll want the average temperature difference over each 2 hour period - assuming it varies significantly over that sort of timescale (eg clouds appearing/disappearing) then you don't want just a snapshot each 2 hours.


The other option is to change the flow meter to one giving a useful output !

_______________________________________________
rrd-users mailing list
[hidden email]
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users