The answer to this question is probably obvious, but I couldn't figure out what it means nor find the documentation for it. When I run rrdtool info, I get ds[source].unknown_sec = 0, and I am not sure what it exactly means...Any help or pointers would be appreciated!
The unknown_sec is the number of seconds for which the value of the DS is Unknown. This could be because the supplied value was Unknown, or was outside the specified range, or because the time since the last sample exceeds the heartbeat time (which marks everything since then as Unknown).
The amount of Unknown time in an interval is then used in combination with the xff fraction in the RRA definitions to determine if a consolodated data point stored in the RRA is Unknown.
Actually, I think I figured out what it means. If two data samples exceed the heartbeat value, then the entire interval between the two data samples are marked as unknown. So unknown_sec = 0 means there hasn't been two data samples that exceed the heartbeat value.
Related
According to rrdcreate manual, the RRA definition has the following syntax:
RRA:{AVERAGE | MIN | MAX | LAST}:xff:steps:rows
XFF(xfiles factor) defines what part of a consolidation interval may be made up from UNKNOWN data while the consolidated value is still regarded as known. Am I correct, that if the step is 1, then XFF value doesn't matter because single PDP is used to build the RRA entry, i.e this single PDP is either present or not? For example, both RRA:AVERAGE:0:1:12 and RRA:AVERAGE:0.9999:1:12 are equal?
Short answer is, yes, sort of. When you have 1cdp=1pdp then the amount of data present in a CDP can logically only be 100% or 0%, and so all xff values in the range 100>xff>=0 are identical. Note that we exclude 100 from this range! An xff of 100 would not make sense unless your computer has a usb-attached crystal ball...
However this parameter becomes much more important once lower-resolution RRAs start to consolidate data over large windows, which is the primary function of the RRA.
I create a standard RRDTool database with a default step of 5mn (300s).
I have different types of values in it, some gauges which are easily processed, but I have other values I would have in COUNTER but here is my problem :
I read the data in a program, and get the difference between values over two steps is good but the counter increment less than time (It can increment by less than 300 during a step), so my out value is wrong.
Is it possible to change the COUNTER for not be a number by second but by step or something like that, if it's not I suppose I have to calculate the difference in my program ?
Thank you for helping.
RRDTool is capable of handling fractional values, so there is no problem if the counter increments by less than the seconds interval since the last update.
RRDTool stores everything as a Rate. If your DS is of type GAUGE, then RRDTool assumes that the incoming value is alreayd a rate, and only applies Data Normalisation (more on this later). If the type is COUNTER or DERIVE, then the value/timepoint you are updating with is compared to the previous value/timepoint to obtain a rate thus: r=(x2 - x1)/(t2 - t1). The rate obtained is then Normalised. The other DS type is ABSOLUTE, which assumes the counter was reset on the last read, giving r=x2/(t2 - t1).
The Normalisation step adjusts the data point based on assuming a linear progression from the last data point so that it lies exactly on an interval boundary. For example, if your step is 5min, and you update at 12:06, the data point is adjusted back to what it would have been at 12:05, and stored against 12:05. However the last unadjusted DP is still preserved for use at the next update, so that overall rates are correct.
So, if you have a 300s (5min) interval, and the value increased by 150, the rate stored will be 0.5.
If the value you are graphing is something small, e.g. 'number of pages printed', this might seem counterintuitive, but it works well for large rates such as network traffic counters (which is what RRDTool was designed for).
If you really do not want to display fractional values in the generated graphs or output, then you can use a format string such as %.0f to enforce no decimal places and the displayed number will be rounded to the nearest integer.
I'm pretty sure this question has been asked several times, but either I did not find the correct answer or I didn't understand the solution.
To my current problem:
I have a sensor which measures the time a motor is running.
The sensor is reset after reading.
I'm not interested in the time the motor was running the last five minutes.
I'm more interested in how long the motor was running from the very beginning (or from the last reset).
When storing the values in an rrd, depending on the aggregate function, several values are recorded.
When working with GAUGE, the value read is 3000 (10th seconds) every five minutes.
When working with ABSOLUTE, the value is 10 every five minutes.
But what I would like to get is something like:
3000 after the first 5 minutes
6000 after the next 5 minutes (last value + 3000)
9000 after another 5 minutes (last value + 3000)
The accuracy of the older values (and slopes) is not so important, but the last value should reflect the time in seconds since the beginning as accurate as possible.
Is there a way to accomplish this?
I dont know if it is useful for ur need or not but maybe using TREND/TRENDNAN CDEF function is what u want, look at here:
TREND CDEF function
I now created a small SQLite database with one table and one column in that tabe.
The table has one row. I update that row every time my cron job runs and add the current value to the current value. So the current value of the one row and column is the cumualted value of my sensor. This is then fed into the rrd.
Any other (better) ideas?
The way that I'd tackle this (in Linux) is to write the value to a plain-text file and then use the value from that file for the RRDTool graph. I think that maybe using SQLite (or any other SQL server) just to keep track of this would be unnecessarily hard on a system just to keep track of something like this.
First of all, sorry for my poor English. I am writting video streaming server in C++. I have multiple mpeg2-ts files (movies and advertisements) which I need to stream via HTTP as one single TS-FILE. The problem is that every mpeg-ts file has its own timestamps (PCR, PTS, DTS). And, as I understand, to make a continuous streaming flow, every new PCR (PTS, DTS) value should continue from the last PCR (PTS, DTS) value.
Here is a picture for better understanding of what I am saying about: http://i.stack.imgur.com/vL1m6.png (I can't include my picture directly in the message. Sorry)
I need to replace pcr`1, pcr`2, pcr`3 timestamps with the new ones. For example, I sent ts-packet containing the pcr3 timestamp and after a few more ts packets (not containing any value of PCR) I want to insert my advertisement. And my question is: how do I calculate the new values for pcr`1, pcr`2, pcr`3 and so on?
Is it correct to calculate the bitrate of the current video and then divide the amount of bits that the program have sent since the last PCR timestamp (in our case, it's pcr3) by this bitrate? I mean the following: (new timestamp) = (previous timestamp) + (the amount of bits) / (bitrate). Or is there a more efficient way to do it?
As for PTS and DTS timestamps, I read here that these timestamps can be non-linear. Is it will be correct to calculate it relative to the last original PCR that I received? I mean:
pts_new = (original_pts - last_original_pcr) + pcr_new.
dts_new = (original_dts - last_original_pcr) + pcr_new.
(original_pts - last_original_pcr) is the difference between pts and pcr values
pcr_new is the last modified pcr value
My program can read and edit these timestamps in mpeg-ts stream. Fortunately, there's a lot of literature on how to do it. But how do I calculate the new values for these timestamps?
I have just started to learn the specification of mpeg2-ts, and please correct my mistakes if I am wrong in something.
Thanks in advance. Hope you understood me.
Mpeg2 "Splicing" is an art-form, and is much more complicated than concatenating two streams. It requires manipulations which many companies have patented (http://www.google.com/patents/US6380991, http://www.google.com/patents/US6806909, http://www.google.com/patents/US6993081)
to answer some of your questions:
your calculation of the next pcr looks ok, although you need take into account many compliancy issues (etr290 for example)
for DTS/PTS you have much more work to do. the most basic splice will just restamp the ad's pts/dts in such a way that they continue from the last timestamp of the first TS.
ad first timestamp = last timestamp + frame interval
the trick lies within making sure that you have no "holes" in either the presentation time stamps or the decoding timestamps. This is the difficult part and requires deep understanding in MPEG2 buffers (tstd, eb, mb).
Good luck.
I mainly write because I'm using the rrdtool holt-winters feature, but sadly it does not work as I would, starting I'll write for you the rrd file command line creation:
`/usr/bin/rrdtool create /home/spread/httphw/rrd/httpr.rrd --start $_[7]-60 --step 60 DS:200:GAUGE:120:U:U RRA:AVERAGE:0.5:1:1440 RRA:HWPREDICT:1440:0.1:0.0035:288 RRA:AVERAGE:0.5:6:700 RRA:AVERAGE:0.5:24:775 RRA:AVERAGE:0.5:288:797`;
After that I basically insert data and then I draw the graph like that:
`/usr/bin/rrdtool graph file.png --start $start --end $time --width 600 --height 200 --imgformat PNG DEF:doscents=$rrd:200:AVERAGE DEF:pred=$rrd:200:HWPREDICT DEF:dev=$rrd:200:DEVPREDICT DEF:fail=$rrd:200:FAILURES TICK:fail#ffffa0:1.0:"Failures Average" CDEF:scale200=doscents,8,* CDEF:upper=pred,dev,2,*,+ CDEF:lower=pred,dev,2,*,- CDEF:scaledupper=upper,8,* CDEF:scaledlower=lower,8,* LINE1:scale200#0000ff:"Average" LINE1:scaledupper#ff0000:"Upper Bound Average" LINE1:scaledlower#ff0000:"Lower Bound Average"`;
Here's the image RRDTOOL IMAGE
The I get a graph like that, but as you can see there's yellow lines that indicates that there has been an error when that's not true, I mean, the activity line at that point is slightly out from the red area but it does not an error, I basically need to understand the values I gotta set up and based on what, I tried it out but I don't really understand the system really well.
Any sugestion from an rrdtool expert?
Many thanks in advance
Being outside the expected range is an error, as far as Holt-Winters is concerned.
The Holt-Winters FAILURES RRA is a slightly more complex than just 'outside the range HWPREDICT+-2*DEVPREDICT'. In fact, there are the additional threshold and window parameters, which (if not specified, as in your case) default to 7 and 9 respectively.
These cause a smoothing of the samples over window samples before comparison, and only trigger a FAILURE flag when there is a sequence of threshold consecutive errors.
As a result, you see a FAILURE trigger where you do, and not in the larger area to the left (which averages down within the range). This results in a better indicator of consistently our of range behaviour, rather than a slope slightly too early or a temporary spike.
If you want to avoid this, and have a FAILURE flag every time the data goes outside of the predicted bounds, then set the FAILURE parameters to 1 and 1. To do this, you would need to explicitly define the additional HW RRAs rather than having them defined implicitly as you do now.
On a separate note, is is bad practice to have a DS with a purely numerical name. It can cause confusion in the RPN calculations. Always have a DS name start with a lowercase letter.