RRDB Contains Only NaN - rrdtool

I'm getting all NaN's in my rrdb. Why?
rrdtool create temps.rrd --step 120 \
DS:temp:GAUGE:250:-10:212 \
DS:rate:DERIVE:250:-10:212 \
DS:setpoint:GAUGE:250:-10:212 \
RRA:AVERAGE:0.3:1:43200
After a night of collecting data & updating every 2 minutes something like this:
/usr/bin/rrdupdate temps.rrd N:30.8:30.8:9.6
The DS is changing, but the RRA doesn't have any entries.
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE rrd SYSTEM "http://oss.oetiker.ch/rrdtool/rrdtool.dtd">
<!-- Round Robin Database Dump -->
<rrd>
<version>0003</version>
<step>120</step> <!-- Seconds -->
<lastupdate>1375360140</lastupdate> <!-- 2013-08-01 07:29:00 CDT -->
<ds>
<name> temp </name>
<type> GAUGE </type>
<minimal_heartbeat>250</minimal_heartbeat>
<min>-1.0000000000e+01</min>
<max>2.1200000000e+02</max>
<!-- PDP Status -->
<last_ds>60.1</last_ds>
<value>NaN</value>
<unknown_sec> 60 </unknown_sec>
</ds>
<ds>
<name> rate </name>
<type> DERIVE </type>
<minimal_heartbeat>250</minimal_heartbeat>
<min>-1.0000000000e+01</min>
<max>2.1200000000e+02</max>
<!-- PDP Status -->
<last_ds>59.9</last_ds>
<value>NaN</value>
<unknown_sec> 60 </unknown_sec>
</ds>
<ds>
<name> setpoint </name>
<type> GAUGE </type>
<minimal_heartbeat>250</minimal_heartbeat>
<min>-1.0000000000e+01</min>
<max>2.1200000000e+02</max>
<!-- PDP Status -->
<last_ds>60.0</last_ds>
<value>NaN</value>
<unknown_sec> 60 </unknown_sec>
</ds>
<!-- Round Robin Archives -->
<rra>
<cf>AVERAGE</cf>
<pdp_per_row>1</pdp_per_row> <!-- 120 seconds -->
<params>
<xff>3.0000000000e-01</xff>
</params>
<cdp_prep>
<ds>
<primary_value>NaN</primary_value>
<secondary_value>NaN</secondary_value>
<value>NaN</value>
<unknown_datapoints>0</unknown_datapoints>
</ds>
<ds>
<primary_value>NaN</primary_value>
<secondary_value>NaN</secondary_value>
<value>NaN</value>
<unknown_datapoints>0</unknown_datapoints>
</ds>
<ds>
<primary_value>NaN</primary_value>
<secondary_value>NaN</secondary_value>
<value>NaN</value>
<unknown_datapoints>0</unknown_datapoints>
</ds>
</cdp_prep>
<database>
<!-- 2013-06-02 07:30:00 CDT / 1370176200 --> <row><v>NaN</v><v>NaN</v><v>NaN</v></row>
<!-- 2013-06-02 07:32:00 CDT / 1370176320 --> <row><v>NaN</v><v>NaN</v><v>NaN</v></row>
<!-- 2013-06-02 07:34:00 CDT / 1370176440 --> <row><v>NaN</v><v>NaN</v><v>NaN</v></row>
many lines, all NaN
<!-- 2013-08-01 07:26:00 CDT / 1375359960 --> <row><v>NaN</v><v>NaN</v><v>NaN</v></row>
<!-- 2013-08-01 07:28:00 CDT / 1375360080 --> <row><v>NaN</v><v>NaN</v><v>NaN</v></row>
</database>
</rra>
</rrd>

Your problem is that you are not actually storing any data; your rrdupdate calls are failing.
If you run the rrdupdate command from the commandline, you will immediately see the error:
$ rrdtool update temps.rrd N:30.8:31.2:9.1
ERROR: temps.rrd: not a simple signed integer: '31.2'
The reason for this is that, while the first and third DS are of type GAUGE, the second is of type DERIVE. It is a (poorly documented) fact that you can only use a non-integer value when the data type is GAUGE.
So, you have four options -
Make sure your second value (for 'rate') is always an integer
Change it to type GAUGE and pass the rate rather than the value
Remove the second DS entirely, and calculate this on the fly when displaying the data
Replace the rate DS with a COMPUTE DS type to calculate at storage time
The last may be the best option but would need some work.

Related

Group nodes together based on condition and validate using Schematron

I am working on writing rules using Schematron to validate data below. The requirement is to verify whether a patient has at least one encounter in the past 12 months. If there are multiple encounters per patient, use the last encounter.
<root>
<entry>
<resource>
<resourceType>Encounter</resourceType>
<subject>
<id>Patient/12345</id>
</subject>
<encounterDate>2018-04-10T10:00:00</encounterDate>
</resource>
</entry>
<entry>
<resource>
<resourceType>Encounter</resourceType>
<subject>
<id>Patient/abcde</id>
</subject>
<encounterDate>2020-04-10T10:00:00</encounterDate>
</resource>
</entry>
<entry>
<resource>
<resourceType>Encounter</resourceType>
<subject>
<id>Patient/abcde</id>
</subject>
<encounterDate>2019-05-10T10:00:00</encounterDate>
</resource>
</entry>
</root>
The above data should pass the validation because the latest encounter is less than a year ago.
What I want to know is, if I write a template that groups encounters together by patient id, is there a way to pass that template to the rule context? If not, is there any other way of doing it?
I am completely new to both xslt and Schematron and here is what I have so far:
<schema xmlns="http://purl.oclc.org/dsdl/schematron" >
<pattern>
<key name="patientId" match="entry" use="/resouce/subject/id/text()"/>
<template name="dateByPatient" match="entry">
<root>
<for-each select="resource/subject/id">
<patient >
<for-each select="key('patientId',text())">
<effectiveDateTime><value-of select="./resource/encounterDate"/></effectiveDateTime>
</for-each>
</patient>
</for-each>
</root>
</template>
<let name="template">
<dateByPatient/>
</let>
<let name="latest">
<root>
<for-each select="$template/root/patient">
<patient >
<sort select="effectiveDateTime" order="descending" />
<if test="position() = 1">
<effectiveDateTime><value-of select="effectiveDateTime" /></effectiveDateTime>
</if>
</patient>
</for-each>
</root>
</let>
<rule context="$latest/root/patient/effectiveDateTime">
<let name="days" value="days-from-duration(fn:current-dateTime() - xs:dateTime(text()))" />
<assert test="days-from-duration(fn:current-dateTime() - xs:dateTime(text())) < 365">
Encounter date more than a year : <value-of select="$days" /> days
</assert>
</rule>
</pattern>
</schema>
With XSLT 3 underlying you could use
<?xml version="1.0" encoding="UTF-8"?>
<sch:schema xmlns:sch="http://purl.oclc.org/dsdl/schematron" queryBinding="xslt3"
xmlns:sqf="http://www.schematron-quickfix.com/validator/process">
<sch:ns prefix="map" uri="http://www.w3.org/2005/xpath-functions/map"/>
<sch:pattern>
<sch:rule context="root">
<sch:let name="groups"
value="let $encounter-resources := entry/resource[resourceType = 'Encounter']
return map:merge(
$encounter-resources
!
map {
data(subject/id) : xs:dateTime(encounterDate)
},
map { 'duplicates' : 'combine' }
)"/>
<sch:assert
test="every $patient in map:keys($groups)
satisfies
(current-dateTime() - max($groups($patient)))
lt xs:dayTimeDuration('P365D')">At least one patient with latest encounter more than a year ago.</sch:assert>
</sch:rule>
</sch:pattern>
</sch:schema>
Or to output more detailed information and to only process resources with type Encounter:
<?xml version="1.0" encoding="UTF-8"?>
<sch:schema xmlns:sch="http://purl.oclc.org/dsdl/schematron" queryBinding="xslt3"
xmlns:sqf="http://www.schematron-quickfix.com/validator/process">
<sch:ns prefix="map" uri="http://www.w3.org/2005/xpath-functions/map"/>
<sch:pattern>
<sch:rule context="root">
<sch:let name="groups"
value="let $encounter-resources := entry/resource[resourceType = 'Encounter']
return map:merge(
$encounter-resources
!
map {
data(subject/id) : xs:dateTime(encounterDate)
},
map { 'duplicates' : 'combine' }
)"/>
<sch:let name="failing-patients"
value="map:keys($groups)[(current-dateTime() - max($groups(.))) gt xs:dayTimeDuration('P365D')]"/>
<sch:report
test="exists($failing-patients)">Patients <sch:value-of select="$failing-patients"/> with latest encounter more than a year ago.</sch:report>
</sch:rule>
</sch:pattern>
</sch:schema>
I don't think you can mix Schematron and XSLT as freely as your code tries, you would need to set up an XProc pipeline to use p:xslt to group the original input and then a validation step to validate with Schematron.
As for your problems to run the second sample with node-schematron, it uses an XPath implementation that doesn't support the XPath 3.1 sort function it seems, node-schematron also fails to handle maps as intermediary results of a Schematron variable, so only stuffing all into one variable expression seems to do; two examples work:
<sch:schema xmlns:sch="http://purl.oclc.org/dsdl/schematron" queryBinding="xslt3"
xmlns:sqf="http://www.schematron-quickfix.com/validator/process">
<sch:ns prefix="map" uri="http://www.w3.org/2005/xpath-functions/map"/>
<sch:pattern>
<sch:rule context="root">
<sch:let name="failing-patients"
value="let $encounter-resources := entry/resource[resourceType = 'Encounter'],
$groups := map:merge(
$encounter-resources
!
map {
data(subject/id) : xs:dateTime(encounterDate)
},
map { 'duplicates' : 'combine' }
)
return map:keys($groups)[(current-dateTime() - max($groups(.))) gt xs:dayTimeDuration('P365D')]"/>
<sch:report
test="exists($failing-patients)">Patients <sch:value-of select="$failing-patients"/> with latest encounter more than a year ago.</sch:report>
</sch:rule>
</sch:pattern>
</sch:schema>
or
<sch:schema xmlns:sch="http://purl.oclc.org/dsdl/schematron" queryBinding="xslt3"
xmlns:sqf="http://www.schematron-quickfix.com/validator/process">
<sch:ns prefix="map" uri="http://www.w3.org/2005/xpath-functions/map"/>
<sch:pattern>
<sch:rule context="root">
<sch:let name="failing-patients"
value="let
$encounter-resources := entry/resource[resourceType = 'Encounter'],
$groups := fold-left(
$encounter-resources,
map{},
function($m, $e) {
map:put(
$m,
data($e/subject/id),
max((xs:dateTime($e/encounterDate), map:get($m, data($e/subject/id))))
)
})
return map:keys($groups)[(current-dateTime() - $groups(.)) gt xs:dayTimeDuration('P365D')]"/>
<sch:report test="exists($failing-patients)">Patients <sch:value-of
select="$failing-patients"/> with latest encounter more than a year
ago.</sch:report>
</sch:rule>
</sch:pattern>
</sch:schema>
If you need an assertion that fails then replace the sch:report with
<sch:assert
test="empty($failing-patients)">Patients <sch:value-of select="$failing-patients"/> with latest encounter more than a year ago.</sch:assert>

How to exclude method in coverlet coverage report?

How can I exclude a method from code coverage reporting using coverlet and reportgenerator. Excluding entire namespaces in .runsettings works as expected but using [ExcludeFromCodeCoverage] attribute excludes the entire file instead of only the targeted method. See Comments below for what I've tried in .runsettings.
relevant .runsettings lines:
<DataCollectionRunSettings>
<DataCollectors>
<DataCollector>
<Configuration>
<Format>lcov</Format>
<Include>[*]*</Include>
<Exclude>
<!-- excluded namespaces -->
</Exclude>
<!-- excludes entire file from coverage -->
<ExcludeByAttribute>Obsolete, GeneratedCodeAttribute, CompilerGeneratedAttribute,ExcludeFromCodeCoverage</ExcludeByAttribute>
<!-- included & reported as uncovered -->
<ExcludeByAttribute> ExcludeFromCodeCoverageAttribute </ExcludeByAttribute>
<SingleHit>true</SingleHit>
<UseSourceLink>true</UseSourceLink>
<CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies>
<!-- included and reported as uncovered -->
<CodeCoverage>
<Attributes>
<Exclude>
<Attribute> ^System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverageAttribute$</Attribute>
</Exclude>
</Attributes>
</CodeCoverage>
</Configuration>
</DataCollector>
</DataCollectors>
</DataCollectionRunSettings>
UPDATE: It seems to have been a recently resolved issue with coverlet. Updating resolved the issue. https://github.com/coverlet-coverage/coverlet/issues/809
Just apply [ExcludeFromCodeCoverage] on the method instead of the class.

How to list the two most recent folders inside a directory using their timestamp

I have a parent folder and inside that I have a few folders. For an automation, I want to take the latest of the two folders according to timestamp.
I have tried to take the latest folder by using timstampselector.
<timestampselector property="latest.modified">
<path>
<dirset dir="MyDirectoryPath">
<include name="*" />
</dirset>
</path>
</timestampselector>
Inside my parent folder, I have the following folders:
test (Last modified on 07/04/2019 10:30 AM)
check (Last modified on 08/04/2019 05:00 PM)
integrate (Last modified on 08/04/2019 12:30 PM)
slave (Last modified on 09/04/2019 05:00 PM)
Our script should take the latest two modified folders, which is in the above case it should be integrate & slave.
How can I achieve that?
Generally speaking, it's a good idea to stay away from ant-contrib whenever possible. This particular problem can be quickly solved with native Ant's resource collections:
<last count="2" id="latest.two.files">
<sort>
<date />
<fileset dir="MyDirectoryPath" />
</sort>
</last>
Full example target:
<target name="select-latest">
<delete dir="testdir" />
<mkdir dir="testdir" />
<touch file="testdir/test" datetime="07/04/2019 10:30 AM" />
<touch file="testdir/check" datetime="08/04/2019 05:00 PM" />
<touch file="testdir/integrate" datetime="08/04/2019 12:30 PM" />
<touch file="testdir/slave" datetime="09/04/2019 05:00 PM" />
<last count="2" id="latest.two.files">
<sort>
<date />
<fileset dir="testdir" />
</sort>
</last>
<echo message="${toString:latest.two.files}" />
</target>
The task you are using is part of Ant-Contrib rather than core Ant. The documentation says you can use the count attribute to say how many items you want to select. In your case, set it to two:
<timestampselector property="latest.modified" count="2">
<path>
<dirset dir="MyDirectoryPath">
<include name="*" />
</dirset>
</path>
</timestampselector>
This appeared to work fine for me: the property was set to a comma-separated list of two directories.

MapForce - Add dayTimeDuration to dayTimeDuration

I am trying to use mapforce to generate an xslt 2.0 file. The mapping is adding 2 dayTimeDuration elements, doing so results in the following error;
No match for core.add(xs:dayTimeDuration, xs:dayTimeDuration). Check argument types.
Supported: +(xs:double, xs:double) -> xs:double
I thought that xslt 2.0 supported adding 2 dayTimeDurations. Is there a way of doing this using mapforce?
Cheers
Stew
Had almost the same problem, first tried to add functx-library but saw it creates absolute path in the generated xslt2-code, which isn't very good.
Well, turns out you can implement that function, but first you have to do some modifications...
Find your Mapforce installation directory, and MapForceLibraries -subdirectory. From that open the "core.mff", and find
<group name="math functions">
<component name="add" growable="true" growablebasename="value">
<sources>
<datapoint name="value1" type="xs:decimal"/>
<datapoint name="value2" type="xs:decimal"/>
</sources>
<targets>
<datapoint name="result" type="xs:decimal"/>
</targets>
As you can seem the "sources" and "targets" elements seems to define the in- and out data types. As it is, they have only implemented "add"-function for "xs:decimal". You can copy/paste this component, then rename it and give new in- out- data types, in your case they are both "xs:dayTimeDuration". Note that there are implementations for each supported language, but you can omit those that are not needed. Here's
what should work:
<component name="addDayTimeDuration" growable="true" growablebasename="value">
<sources>
<datapoint name="value1" type="xs:dayTimeDuration"/>
<datapoint name="value2" type="xs:dayTimeDuration"/>
</sources>
<targets>
<datapoint name="result" type="xs:dayTimeDuration"/>
</targets>
<implementations>
<implementation language="xslt">
<operator value="+"/>
</implementation>
<implementation language="xslt2">
<operator value="+"/>
</implementation>
<implementation language="builtin">
<function name="Core_Add"/>
</implementation>
</implementations>
<description>
<short>result = value1 + value2</short>
<long>Result is the dayTimeDuration value of adding value1 and value2.</long>
</description>
</component>
Your new function should now appear in the "math functions" and should be good to use.
After contacting Altova (the makers of MapForce);
While XPath 2 does offer a subtract-dayTimeDurations operation, this is not presently offered as a function inside MapForce.

SyncML bug with my custom server for the Replace command to the client

I develop a syncml server and I do not synchronize from the server to the client (nokia e71) for modified contacts. All the rest works except when I make a command replace towards the client for an existing localuid. The client returns me then the status 415 for this command (type or format of the datum is not corresponding) while the customer accepts the same datum for an addition (by a command add or replaces).
Has anybody already met this problem ?
Here are messages sent between the client and the server:
Server message with Replace command:
<?xml version="1.0" ?>
<!DOCTYPE SyncML
PUBLIC "-//SYNCML//DTD SyncML 1.2//EN"
"http://www.openmobilealliance.org/tech/DTD/OMA-TS-SyncML_RepPro_DTD-V1_2.dtd">
<SyncML xmlns="SYNCML:SYNCML1.2">
<SyncHdr>
<VerDTD>1.2</VerDTD><VerProto>SyncML/1.2</VerProto><SessionID>235</SessionID><MsgID>3</MsgID> <Target><LocURI>IMEI:358240030276208</LocURI></Target> <Source><LocURI>http://192.168.8.20:50000</LocURI></Source> <Meta>
<MaxMsgSize xmlns="syncml:metinf">1000000</MaxMsgSize><MaxObjSize xmlns="syncml:metinf">4000000</MaxObjSize>
</Meta>
</SyncHdr>
<SyncBody>
<Status>
<CmdID>1</CmdID><MsgRef>3</MsgRef><CmdRef>0</CmdRef><Cmd>SyncHdr</Cmd> <TargetRef>http://192.168.8.20:50000</TargetRef> <SourceRef>IMEI:358240030276208</SourceRef> <Data>200</Data>
</Status> <Status>
<CmdID>2</CmdID><MsgRef>3</MsgRef><CmdRef>3</CmdRef><Cmd>Sync</Cmd> <SourceRef>./C:Contacts.cdb</SourceRef> <TargetRef>./card</TargetRef> <Data>200</Data>
</Status> <Sync>
<CmdID>3</CmdID> <Source><LocURI>./card</LocURI></Source> <Target><LocURI>./C:Contacts.cdb</LocURI></Target><NumberOfChanges>1</NumberOfChanges> <Replace>
<CmdID>4</CmdID> <Meta><Type xmlns="syncml:metinf">text/x-vcard</Type> </Meta> <Item> <Target><LocURI>69</LocURI></Target> <Data>
<![CDATA[BEGIN:VCARD VERSION:2.1 N:Smith;Change;;; FN:Change Smith END:VCARD]]>
</Data> </Item>
</Replace>
</Sync> <Final/>
</SyncBody>
</SyncML>
Client Message with a replace command status:
<?xml version="1.0" ?>
<!DOCTYPE SyncML
PUBLIC '-//SYNCML//DTD SyncML 1.2//EN'
'http://www.openmobilealliance.org/tech/DTD/OMA-TS-SyncML_RepPro_DTD-V1_2.dtd'>
<SyncML xmlns="SYNCML:SYNCML1.2">
<SyncHdr>
<VerDTD>
1.2
</VerDTD>
<VerProto>
SyncML/1.2
</VerProto>
<SessionID>
235
</SessionID>
<MsgID>
4
</MsgID>
<Target>
<LocURI>
http://192.168.8.20:50000
</LocURI>
</Target>
<Source>
<LocURI>
IMEI:358240030276208
</LocURI>
<LocName>
test1
</LocName>
</Source>
<Meta>
<MaxMsgSize xmlns="syncml:metinf">
65535
</MaxMsgSize>
</Meta>
</SyncHdr>
<SyncBody>
<Status>
<CmdID>
1
</CmdID>
<MsgRef>
3
</MsgRef>
<CmdRef>
0
</CmdRef>
<Cmd>
SyncHdr
</Cmd>
<TargetRef>
IMEI:358240030276208
</TargetRef>
<SourceRef>
http://192.168.8.20:50000
</SourceRef>
<Data>
200
</Data>
</Status>
<Status>
<CmdID>
2
</CmdID>
<MsgRef>
3
</MsgRef>
<CmdRef>
3
</CmdRef>
<Cmd>
Sync
</Cmd>
<TargetRef>
./C:Contacts.cdb
</TargetRef>
<SourceRef>
./card
</SourceRef>
<Data>
200
</Data>
</Status>
<Status>
<CmdID>
3
</CmdID>
<MsgRef>
3
</MsgRef>
<CmdRef>
4
</CmdRef>
<Cmd>
Replace
</Cmd>
<TargetRef>
69
</TargetRef>
<Data>
415
</Data>
</Status>
<Final/>
</SyncBody>
</SyncML>