I have a water flowmeter connected to a RPi which is writing data to a simple RRD
RRDs::create ($rrdfile, "--start", 1572829200,
"--step", 60,
"DS:FLOW1:GAUGE:90:U:U",
"RRA:MAX:0.5:1:10512000",);
From this I generate a graph for the last 24 hours and some statistics for the last few days. A simplified version follows
RRDs::graph "temps.png",
"--start=now-1d",
"--end=now",
"--width=1000",
"--base=1000",
"--height=240",
"--title=Flow Data - ",
"--slope-mode",
"--vertical-label=Volume of Water",
"DEF:flow-now=flow.rrd:FLOW1:AVERAGE", #Used to generate the graph
"DEF:flow-1d=flow.rrd:FLOW1:AVERAGE:end=midnight:start=end-1d", #Data for yesterday
"CDEF:flow-1d-1=flow-1d,25440,/", #Convert raw data to litres
"VDEF:flow-1dtotal=flow-1d-1,TOTAL", #Get total litres
"GPRINT:flow-1dtotal:Total Volume last 1 day = %.2lf L", #Print total for yesterday
I would like to add an arbitrary value to flow-1dtotal but can't work out how. Something along the lines of the psuedo code below is what I need
flow-1dtotal = flow-1dtotal + 1000
Thanks for reading and for any suggestions
Related
I am using rrdtool to record data off of a Morningstar Solar Charge controller. The data is obtained through SNMP. One of the datapoints being recorded is the "Charge Current" produced by the Solar array. I'm using PHP's "rrd_graph" to generate the graph. I have no problem generating a graph to show the power generated by the solar array over time, but I also want a summary of the AmpHours generated for the past "x" time displayed by the graph. The data is recorded into the rrd database every 60 seconds. This is the PHP code to display the desired graph:
// Solar Array
$opts= array("--start", $start, "--end", $timestamp,
"-h", "250",
"-w", "800",
"-E",
"-v", "Watts",
"--title", "Array Power for $node",
"DEF:arraymaxpower=/home/anr/data/solar/$node.rrd:arraymaxpower:AVERAGE",
"DEF:arraypower=/home/anr/data/solar/$node.rrd:arraypower:AVERAGE",
"DEF:chargecurrent=/home/anr/data/solar/$node.rrd:chargecurrent:AVERAGE",
"CDEF:amphours=chargecurrent,60,/",
"VDEF:amphourstot=amphours,TOTAL",
"AREA:arraypower#ffaf8f:Array Power",
"LINE1:arraypower#852600",
"LINE2:arraymaxpower#336600:Array Max Power",
"GPRINT:amphourstot:Amp Hours\: %5.2lf "
);
$ret = rrd_graph("/var/www/html/admin/solar/graphs/$node-arraypower.png", $opts);
if (! $ret) {
echo "<b>Graph error: </b>" . rrd_error() . "\n";
}
echo "<img src='/admin/solar/graphs/$node-arraypower.png' alt='Generated RRD image'><br />";
While I'm not displaying the charge. current in the graph, the datapoint generated by the DEF statement is needed to calculate the AmpHours value that I need. Since the data is stored every 60 seconds, I assumed that if I simply divided the data by 60 in a code, it would change the stored value from "AmpMinutes" to "AmpHours" and then I could use a VDEF to Total that value over the displayed range in the graph. However, I am ending up with numbers that are way too high. Any idea on what I am doing wrong?
I am working on a real estate cash-flow simulation.
What I want in the end is a time series where everyday I report if the property is vacant, leased and if I collected rent.
In my present code, I create first a profit array with values of "Leased", "Vacant" or "Today you collected rent of $1000", so I used this to create my time series:
rng=pd.date_range('6/1/2016', periods=len(profit), freq='D')
ts=pd.Series(profit, index=rng)
To simplify, I assumed I collected rent every 30 days. Now I want to be more specific and collect it every 5th day of the month (for example) and be flexible on the day the next tenant will move in.
Do you know commands or a good source where I can learn how to iterate from month to month?
Any help would be appreciated
You can build a sequence of dates using date_range and .shift() (freq='M' is for month-end frequencies) with pd.datetools.day like so:
date_sequence = pd.date_range(start, end, freq='M').shift(num_of_days, freq=pd.datetools.day)
and then use this sequence to select dates from the DateTimeIndex using
df.loc[date_sequence, 'column_name'] = value
Alternatively, you can use pd.DateOffset() like so:
ts = pd.date_range(start=date(2015, 6, 1), end=date(2015, 12, 1), freq='MS')
DatetimeIndex(['2015-06-01', '2015-07-01', '2015-08-01', '2015-09-01',
'2015-10-01', '2015-11-01', '2015-12-01'],
dtype='datetime64[ns]', freq='MS')
Now add 5 days:
ts + pd.DateOffset(days=5)
to get:
DatetimeIndex(['2015-06-06', '2015-07-06', '2015-08-06', '2015-09-06',
'2015-10-06', '2015-11-06', '2015-12-06'],
dtype='datetime64[ns]', freq=None)
Since I didn't find anywhere else this topic I will ask it here. I am getting data from CSV file, I have written datetime format in one of columns. I get that column with pandas module and then I need to count occurrences in specific time slots and plot that with matplotlib. Bellow you can see example of column.
Time and Date
0 2015-08-21 10:51:06.398000
1 2015-08-21 10:51:00.017000
2 2015-08-21 10:52:06.402000
3 2015-08-21 10:54:06.407000
...
I know how I can split time like so:
pd.date_range("10:50", "12:30", freq="1min").time
But how can I assign occurrences of my read values from CSV and then plot it? Any advice or direction would help.
It's hard to tell what you want as you haven't posted desired output but if I understand you correctly you want to count the number of rows in time intervals of certain length. You can do this by combining resample and len. To use resample, first set the index to 'Time and Date:
df.set_index('Date and Time', drop=False)
Note that drop=False is only necessary if the data frame has no other columns.
Then to get the number of rows in each 1-minute interval do
counts = df.resample('1min', len).astype(int)
If there are multiple dates and you want to sum the counts for each time interval over dates do
counts.groupby(lambda ts: ts.time()).sum()
I want to generate fake values in RRD DB for a period of 1 month and with 5 seconds as a frequency for data collection. Is there any tool which would fill RRD DB with fake data for given time duration.
I Googled a lot but did not find any such tool.
Please help.
I would recommend the following one liner:
perl -e 'my $start = time - 30 * 24 * 3600; print join " ","update","my.rrd",(map { ($start+$_*5).":".rand} 0..(30*24*3600/5))' | rrdtool -
this assumes you have an rrd file called my.rrd and that is contains just one data source expecting GAUGE type data.
I have written a python 2.7 script to retrieve all my historical data from Xively.
Originally I wrote it in C#, and it works perfectly.
I am limiting the request to 6 hour blocks, to retrieve all stored data.
My version in Python is as follows:
requestString = 'http://api.xively.com/v2/feeds/41189/datastreams/0001.csv?key=YcfzZVxtXxxxxxxxxxxORnVu_dMQ&start=' + requestDate + '&duration=6hours&interval=0&per_page=1000' response = urllib2.urlopen(requestString).read()
The request date is in the correct format, I compared the full c# requestString version and the python one.
Using the above request, I only get 101 lines of data, which equates to a few minutes of results.
My suspicion is that it is the .read() function, it returns about 34k of characters which is far less than the c# version. I tried adding 100000 as an argument to the ad function, but no change in result.
Left another solution wrote in Python 2.7 too.
In my case, got data each 30 minutes because many sensors sent values every minute and Xively API has limited half hour of data to this sent frequency.
It's general module:
for day in datespan(start_datetime, end_datetime, deltatime): # loop increasing deltatime to star_datetime until finish
while(True): # assurance correct retrieval data
try:
response = urllib2.urlopen('https://api.xively.com/v2/feeds/'+str(feed)+'.csv?key='+apikey_xively+'&start='+ day.strftime("%Y-%m-%dT%H:%M:%SZ")+'&interval='+str(interval)+'&duration='+duration) # get data
break
except:
time.sleep(0.3)
raise # try again
cr = csv.reader(response) # return data in columns
print '.'
for row in cr:
if row[0] in id: # choose desired data
f.write(row[0]+","+row[1]+","+row[2]+"\n") # write "id,timestamp,value"
The full script you can find it here: https://github.com/CarlosRufo/scripts/blob/master/python/retrievalDataXively.py
Hope you might help, delighted to answer any questions :)