NTP timestamps using std::chrono - c++

I'm trying to represent NTP timestamps (including the NTP epoch) in C++ using std::chrono. Therefore, I decided to use a 64-bit unsigned int (unsigned long long) for the ticks and divide it such that the lowest 28-bit represent the fraction of a second (accepting trunction of 4 bits in comparison to the original standard timestamps), the next 32-bit represent the seconds of an epoch and the highest 4-bit represent the epoch. This means that every tick takes 1 / (2^28 - 1) seconds.
I now have the following simple implementation:
#include <chrono>
/**
* Implements a custom C++11 clock starting at 1 Jan 1900 UTC with a tick duration of 2^(-28) seconds.
*/
class NTPClock
{
public:
static constexpr bool is_steady = false;
static constexpr unsigned int era_bits = 4; // epoch uses 4 bits
static constexpr unsigned int fractional_bits = 32-era_bits; // fraction uses 28 bits
static constexpr unsigned int seconds_bits = 32; // second uses 32 bits
using duration = std::chrono::duration<unsigned long long, std::ratio<1, (1<<fractional_bits)-1>>;
using rep = typename duration::rep;
using period = typename duration::period;
using time_point = std::chrono::time_point<NTPClock>;
/**
* Return the current time of this. Note that the implementation is based on the assumption
* that the system clock starts at 1 Jan 1970, which is not defined with C++11 but seems to be a
* standard in most compilers.
*
* #return The current time as represented by an NTP timestamp
*/
static time_point now() noexcept
{
return time_point
(
std::chrono::duration_cast<duration>(std::chrono::system_clock::now().time_since_epoch())
+ std::chrono::duration_cast<duration>(std::chrono::hours(24*25567)) // 25567 days have passed between 1 Jan 1900 and 1 Jan 1970
);
};
}
Unfortunately, a simple test reveals this does not work as expected:
#include <chrono>
#include <iostream>
#include <catch2/catch.hpp>
#include "NTPClock.h"
using namespace std::chrono;
TEST_CASE("NTPClock_now")
{
auto ntp_dur = NTPClock::now().time_since_epoch();
auto sys_dur = system_clock::now().time_since_epoch();
std::cout << duration_cast<hours>(ntp_dur) << std::endl;
std::cout << ntp_dur << std::endl;
std::cout << duration_cast<hours>(sys_dur) << std::endl;
std::cout << sys_dur << std::endl;
REQUIRE(duration_cast<hours>(ntp_dur)-duration_cast<hours>(sys_dur) == hours(24*25567));
}
Output:
613612h
592974797620267184[1/268435455]s
457599h
16473577714886015[1/10000000]s
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
PackageTest.exe is a Catch v2.11.1 host application.
Run with -? for options
-------------------------------------------------------------------------------
NTPClock_now
-------------------------------------------------------------------------------
D:\Repos\...\TestNTPClock.cpp(10)
...............................................................................
D:\Repos\...\TestNTPClock.cpp(18): FAILED:
REQUIRE( duration_cast<hours>(ntp_dur)-duration_cast<hours>(sys_dur) == hours(24*25567) )
with expansion:
156013h == 613608h
===============================================================================
test cases: 1 | 1 failed
assertions: 1 | 1 failed
I also removed the offset of 25567 days in NTPClock::now asserting equality without success. I'm not sure what is going wrong here. Can anybody help?

Your tick period: 1/268'435'455 is unfortunately both extremely fine and also doesn't lend itself to much of a reduced fraction when your desired conversions are used (i.e. between system_clock::duration and NTPClock::duration. This is leading to internal overflow of your unsigned long long NTPClock::rep.
For example, on Windows the system_clock tick period is 1/10,000,000 seconds. The current value of now() is around 1.6 x 1016. To convert this to NTPClock::duration you have to compute 1.6 x 1016 times 53,687,091/2,000,000. The first step in that is the value times the numerator of the conversion factor which is about 8 x 1023, which overflows unsigned long long.
There's a couple of ways to overcome this overflow, and both involve using at least an intermediate representation with a larger range. One could use a 128 bit integral type, but I don't believe that is available on Windows, except perhaps by a 3rd party library. long double is another option. This might look like:
static time_point now() noexcept
{
using imd = std::chrono::duration<long double, period>;
return time_point
(
std::chrono::duration_cast<duration>(imd{std::chrono::system_clock::now().time_since_epoch()
+ std::chrono::hours(24*25567)})
);
};
That is, perform the offset shift with no conversion (system_clock::duration units), then convert that to the intermediate representation imd which has a long double rep, and the same period as NTPClock. This will use long double to compute 1.6 x 1016 times 53,687,091/2,000,000. Then finally duration_cast that to NTPClock::duration. This final duration_cast will end up doing nothing but casting long double to unsigned long long as the conversion factor is simply 1/1.
Another way to accomplish the same thing is:
static time_point now() noexcept
{
return time_point
(
std::chrono::duration_cast<duration>(std::chrono::system_clock::now().time_since_epoch()
+ std::chrono::hours(24*25567)*1.0L)
);
};
This takes advantage of the fact that you can multiply any duration by 1, but with alternate units and the result will have a rep with the common_type of the two arguments, but otherwise have the same value. I.e. std::chrono::hours(24*25567)*1.0L is a long double-based hours. And that long double carries through the rest of the computation until the duration_cast brings it back to NTPClock::duration.
This second way is simpler to write, but code reviewers may not understand the significance of the *1.0L, at least until it becomes a more common idiom.

Related

How to convert unsigned int 64 into time in C++?

I am converting CLI C++ code to standard C++, and i have a piece of code that gets a UINT64 number (from a remote server - so i can't change to format/precision of the time i get) and converts it into DateTime object and later outputs the following value: myDatetime.ToString("dd/MM/yyyy hh:mm:ss.fffffff tt").
I haven't found a way to convert unsigned int 64 into time in C++.
The following code does nothing for numbers so big (that's the 64bit number i get from the server).
time_t rawtime=131274907755873979
localtime_s(&timeinfo, &rawtime);
I need some help :)
My question wan't answered in the thread Convert Epoch Time string to Time since it doesn't work for numbers as large as i need. For example the number 131274907755873979 which is what i get from the server. The function ctime for that value simply returns NULL.
I need a way to convert between the time i get as a unsigned int64 into standard C++ time object.
std::string LongToString(int64_t longDate) {
char buff[128];
std::chrono::duration<int64_t, std::milli> dur(longDate);
auto tp = std::chrono::system_clock::time_point(
std::chrono::duration_cast<std::chrono::system_clock::duration>(dur));
std::time_t in_time_t = std::chrono::system_clock::to_time_t(tp);
strftime(buff, 128, "%Y-%m-%d %H:%M:%S", localtime(&in_time_t));
std::string resDate(buff);
return resDate;
}
This is a case with bsoncxx::types::b_date get_date().to_int64() MongoDB.
The DateTime saved with int64_t.
You have not told us how the existing code converts that number into a DateTime. Let us suppose that it does so by invoking this constructor: DateTime( long long ticks ).
According to the documentation of that constructor of DateTime,
long long ticks A date and time expressed in the number of 100-nanosecond intervals that have elapsed since January 1, 0001 at 00:00:00.000 in the Gregorian calendar.
On the other hand, according to the documentation of localtime_s and the documentation of time_t, localtime_s() requires
the number of seconds (not counting leap seconds) since 00:00, Jan 1 1970 UTC.
So, you first need to convert 100-nanosecond intervals to seconds, and then convert from January 1, 0001 to January 1, 1970.
Using Howard Hinnant's datetime library this computation can be done quite easily. It works with VS 2013 and later.
#include "tz.h"
#include <cstdint>
#include <string>
#include <iostream>
std::string
FILETIME_to_string(std::uint64_t i)
{
using namespace std;
using namespace std::chrono;
using namespace date;
using FileTime = duration<int64_t, ratio<1, 10000000>>;
auto const offset = sys_days{jan/1/1970} - sys_days{jan/1/1601};
auto tp = sys_days{jan/1/1970} + (FileTime{static_cast<int64_t>(i)} - offset);
return format("%d/%m/%Y %I:%M:%S %p", make_zoned("Etc/GMT-2", tp));
}
int
main()
{
std::cout << FILETIME_to_string(131274907755873979) << '\n';
}
This skips DateTime and goes straight to the string. I wasn't sure what you are wanting with tt in the format. But whatever it is, it can be handled.
This library builds on the C++11 <chrono> library. So the first thing to do is to create a duration to represent the windows tick size (100 ns). Then just compute the offset between the two epochs and subtract it from the input, and form a std::chrono::time_point. Now you can format that time_point however you want.
The program above outputs:
29/12/2016 03:12:55.5873979 PM
If you use VS 2017 you'll be able to make offset constexpr, making the conversion more efficient.

Why does std::chrono::time_point not behave as expected?

#include <chrono>
int main()
{
using clock = std::chrono::system_clock;
using time_point = std::chrono::time_point<clock>;
auto tp_now = clock::now();
auto tp_min = time_point::min();
bool b1 = tp_now > tp_min;
bool b2 = (tp_now - tp_min) > std::chrono::seconds{ 0 };
cout << boolalpha << b1 << endl << b2 << endl;
}
The expected output is:
true
true
But the actual output is:
true
false
Why does std::chrono::time_point not behave as expected?
With:
using clock = std::chrono::system_clock;
using time_point = std::chrono::time_point<clock>;
time_point is implemented as if it stores a value of type Duration indicating the time interval from the start of the Clock's epoch. (See std::chrono::time_point)
The duration member type of clock (and of time_point) is capable of representing negative durations.
Thus duration in your implementation may be implemented with a back-end signed integer, (it can be implemented with unsigned integer but with a complicated comparison).
In that particular implementation,
time_point::min();
time_point t(clock::duration::min());
time_point t(clock::duration(std::numeric_limits<Rep>::lowest()));
and tp_now is greater than zero, thus when you subtract them, you get an integer overflow because the result is larger than std::numeric_limits<Rep>::max(). In implementation with signed back-end, it's undefined behavior, in implementation with unsigned back-end, I don't know about it, but I guess its special comparison will make its false.
In this example, tp_min is -9223372036854775808 ticks from its epoch, that number is the same with std::numeric_limits<duration::rep>::lowest()
TL;DR; It's integer overflow. Don't use
(tp1 - tp2) > std::chrono::duration<whatever_rep>::zero
Instead, use
tp1 > tp2

Creating a boost::posix_time::ptime object from a 64 bit integer second count

I have a 32 bit Linux system in which I have to record data that is timestamped with a UINT32 second offset from an epoch of 1901-01-01 00:00:00.
Calculating the timestamp is ok for me as I can use the 64 bit ticks() counter and ticks_per_second() functions to generate the seconds since epoch as follows (I only require second level resolution)
const ptime ptime_origin(time_from_string("1901-01-01 00:00:00"));
time_duration my_utc = microsec_clock::universal_time() - ptime_origin;
boost::int64_t tick_per_sec = my_utc.ticks_per_second();
boost::int64_t tick_count = my_utc.ticks();
boost::int64_t sec_since_epoch = tick_count/tick_per_sec;
This works for me since I know that as an unsigned integer, the seconds count will not exceed the maximum UINT32 value (well not for many years anyway).
The problem I have is that my application can receive a modbus message containing a UINT32 value for which I have to set the hardware and system clock with an ioctl call using RTC_SET_TIME. This UINT32 is again the offset in seconds since my epoch 1901-01-01 00:00:00.
My problem now is that I have no way to create a ptime object using 64 bit integers - the ticks part of the time_duration objects is private and I am restricted to using long which on my 32 bit system is just a 4-byte signed integer not large enough to store the seconds offset from my epoch.
I have no control over the value of the epoch and so I am really stumped as to how I can create my required boost::posix_time::ptime object from the data I have.
I can probably obtain a dirty solution by calculating hard second counts to particular time intervals and using an additional epoch to make a bridge to allow this but I was wondering if there is something in the boost code that will allow me to solve the problem entirely using the boost datetime library.
I have read all the documentation I can find but I cannot see any obvious way to do this.
EDIT: I found this related question Convert int64_t to time_duration but the accepted answer there does NOT work for my epoch
Although boost::posix_time::seconds cannot be used if the seconds represent a number greater than 32 bits (as of Oct 2014), it turns out that boost::posix_time::milliseconds can easily be used (without workarounds), as follows:
inline std::string convertMsSinceEpochToString(std::int64_t const ms)
{
boost::posix_time::ptime time_epoch(boost::gregorian::date(1970, 1, 1));
boost::posix_time::ptime t = time_epoch + boost::posix_time::milliseconds(ms);
return boost::posix_time::to_simple_string(t);
}
So, just convert your 64-bit seconds to (64-bit) milliseconds, and you're good to go!
Note Be /very/ aware of compiler dependent behaviour with the capacity of builting integral types:
uint64_t offset = 113ul*365ul*24ul*60ul*60ul*1000ul; // 113 years give or take some leap seconds/days etc.?
would work on GCC or Clang, but it would simply overflow the calculations in MSVC2013. You'd need to explicitly coerce the calulation to 64 bits:
uint64_t offset = uint64_t(113ul)*365*24*60*60*1000;
You could apply time_durations in the maximum allowable increments (which is std::numeric_limits<long>::max()) since the total_seconds field is limited to long (signed).
Note: I worded it as int32_t below so that it will still work correctly if compiled on a 64-bit platform.
Here's a small demonstration:
#include "boost/date_time.hpp"
#include <iostream>
using namespace boost::gregorian;
using namespace boost::posix_time;
int main()
{
uint64_t offset = 113ul*365ul*24ul*60ul*60ul; // 113 years give or take some leap seconds/days etc.?
static const ptime time_t_epoch(date(1901,1,1));
static const uint32_t max_long = std::numeric_limits<int32_t>::max();
std::cout << "epoch: " << time_t_epoch << "\n";
ptime accum = time_t_epoch;
while (offset > max_long)
{
accum += seconds(max_long);
offset -= max_long;
std::cout << "accumulating: " << accum << "\n";
}
accum += seconds(offset);
std::cout << "final: " << accum << "\n";
}
Prints:
epoch: 1901-Jan-01 00:00:00
accumulating: 1969-Jan-19 03:14:07
final: 2013-Dec-04 00:00:00
See it Live on Coliru

Checking if time_t is between other time_t with some margin of error

I have two time_t variables: timeA and timeB.
What I want to do is check if timeA is the same as timeB. However, I know that in some cases they won't be exactly the same and there may be a 1 or 2 seconds difference between the two of them, so what I really want to check is:
if (timeB - 2sec) <= timeA <= (timeB + 2sec)
Is it possible to do so?
I suppose one option is not to use time_t but instead keep timeB as a tm struct, and just before the comparison, subtract two seconds and create a time_t timeBminus and then add four seconds and create time_t timeBplus. The problem is that I will be comparing several millions of timeA - timeB pairs and want to keep it as simple and fast as possible.
How can I do it?
Something like -
if (std::fabs(std::difftime(timeA, timeB)) < 2.0)
(Not able to check the exact type etc from here but I think the idea is correct)
Even though time_t usually represents seconds, it is in fact not a standard behavior. The details of time_t are left to implementation (According to this)
Instead of using the time_ts directly, it is best to use struct timespec and use the tv_sec member. To check whether they are in 2s distance from each other:
static inline bool close_enough(struct timespec &ts1, struct timespec &ts2)
{
const int64_t sec_in_nsec = 1000000000ll;
int64_t ts1_nsec = ts1.tv_sec * sec_in_nsec + ts1.tv_nsec;
int64_t ts2_nsec = ts2.tv_sec * sec_in_nsec + ts2.tv_nsec;
return ts1_nsec >= ts2_nsec - 2 * sec_in_nsec && ts1_nsec <= ts2_nsec + 2 * sec_in_nsec;
}
or use the same expression in an if. The function call will be optimized away, so no worries about that. Using abs as suggested in other answers is fine, even though it converts your integers into floating points and performs floating point operations.
Update
Using time_t, you can use difftime to get the difference between two times. Again:
static inline bool close_enough(time_t &t1, time_t &t2)
{
double d = difftime(t1, t2);
return d >= -2 && d <= 2;
}
if(abs(timeA - timeB) <= 2) {
// yay!
}
If your compiler supports c++11 you should use chrono instead of ctime:
#include <iostream>
#include <chrono>
#include <tuple>
int main() {
std::chrono::system_clock::time_point a,b;
std::tie(a,b) = std::minmax(a,b);
if ((b-a) < std::chrono::seconds(2)) {
}
}
This way you know the difference between the times is actually less than 2 seconds rather than simply 2 ticks of the clock, where ticks aren't necessarily seconds at all.
Unfortunately there's no std::abs for clock durations, or you would just write if (std::abs(b-a) < std::chrono::seconds(2)), although it's easy enough to write your own:
template<typename Rep,typename Period>
std::chrono::duration<Rep,Period> abs(std::chrono::duration<Rep,Period> d)
{
return std::chrono::duration<Rep,Period>(std::abs(d.count()));
}

Convert Windows Filetime to second in Unix/Linux

I have a trace file that each transaction time represented in Windows filetime format. These time numbers are something like this:
128166372003061629
128166372016382155
128166372026382245
Would you please let me know if there are any C/C++ library in Unix/Linux to extract actual time (specially second) from these numbers ? May I write my own extraction function ?
it's quite simple: the windows epoch starts 1601-01-01T00:00:00Z. It's 11644473600 seconds before the UNIX/Linux epoch (1970-01-01T00:00:00Z). The Windows ticks are in 100 nanoseconds. Thus, a function to get seconds from the UNIX epoch will be as follows:
#define WINDOWS_TICK 10000000
#define SEC_TO_UNIX_EPOCH 11644473600LL
unsigned WindowsTickToUnixSeconds(long long windowsTicks)
{
return (unsigned)(windowsTicks / WINDOWS_TICK - SEC_TO_UNIX_EPOCH);
}
FILETIME type is is the number 100 ns increments since January 1 1601.
To convert this into a unix time_t you can use the following.
#define TICKS_PER_SECOND 10000000
#define EPOCH_DIFFERENCE 11644473600LL
time_t convertWindowsTimeToUnixTime(long long int input){
long long int temp;
temp = input / TICKS_PER_SECOND; //convert from 100ns intervals to seconds;
temp = temp - EPOCH_DIFFERENCE; //subtract number of seconds between epochs
return (time_t) temp;
}
you may then use the ctime functions to manipulate it.
(I discovered I can't enter readable code in a comment, so...)
Note that Windows can represent times outside the range of POSIX epoch times, and thus a conversion routine should return an "out-of-range" indication as appropriate. The simplest method is:
... (as above)
long long secs;
time_t t;
secs = (windowsTicks / WINDOWS_TICK - SEC_TO_UNIX_EPOCH);
t = (time_t) secs;
if (secs != (long long) t) // checks for truncation/overflow/underflow
return (time_t) -1; // value not representable as a POSIX time
return t;
New answer for old question.
Using C++11's <chrono> plus this free, open-source library:
https://github.com/HowardHinnant/date
One can very easily convert these timestamps to std::chrono::system_clock::time_point, and also convert these timestamps to human-readable format in the Gregorian calendar:
#include "date.h"
#include <iostream>
std::chrono::system_clock::time_point
from_windows_filetime(long long t)
{
using namespace std::chrono;
using namespace date;
using wfs = duration<long long, std::ratio<1, 10'000'000>>;
return system_clock::time_point{floor<system_clock::duration>(wfs{t} -
(sys_days{1970_y/jan/1} - sys_days{1601_y/jan/1}))};
}
int
main()
{
using namespace date;
std::cout << from_windows_filetime(128166372003061629) << '\n';
std::cout << from_windows_filetime(128166372016382155) << '\n';
std::cout << from_windows_filetime(128166372026382245) << '\n';
}
For me this outputs:
2007-02-22 17:00:00.306162
2007-02-22 17:00:01.638215
2007-02-22 17:00:02.638224
On Windows, you can actually skip the floor, and get that last decimal digit of precision:
return system_clock::time_point{wfs{t} -
(sys_days{1970_y/jan/1} - sys_days{1601_y/jan/1})};
2007-02-22 17:00:00.3061629
2007-02-22 17:00:01.6382155
2007-02-22 17:00:02.6382245
With optimizations on, the sub-expression (sys_days{1970_y/jan/1} - sys_days{1601_y/jan/1}) will translate at compile time to days{134774} which will further compile-time-convert to whatever units the full-expression requires (seconds, 100-nanoseconds, whatever). Bottom line: This is both very readable and very efficient.
The solution that divides and adds will not work correctly with daylight savings.
Here is a snippet that works, but it is for windows.
time_t FileTime_to_POSIX(FILETIME ft)
{
FILETIME localFileTime;
FileTimeToLocalFileTime(&ft,&localFileTime);
SYSTEMTIME sysTime;
FileTimeToSystemTime(&localFileTime,&sysTime);
struct tm tmtime = {0};
tmtime.tm_year = sysTime.wYear - 1900;
tmtime.tm_mon = sysTime.wMonth - 1;
tmtime.tm_mday = sysTime.wDay;
tmtime.tm_hour = sysTime.wHour;
tmtime.tm_min = sysTime.wMinute;
tmtime.tm_sec = sysTime.wSecond;
tmtime.tm_wday = 0;
tmtime.tm_yday = 0;
tmtime.tm_isdst = -1;
time_t ret = mktime(&tmtime);
return ret;
}
Assuming you are asking about the FILETIME Structure, then FileTimeToSystemTime does what you want, you can get the seconds from the SYSTEMTIME structure it produces.
Here's essentially the same solution except this one encodes negative numbers from Ldap properly and lops off the last 7 digits before conversion.
public static int LdapValueAsUnixTimestamp(SearchResult searchResult, string fieldName)
{
var strValue = LdapValue(searchResult, fieldName);
if (strValue == "0") return 0;
if (strValue == "9223372036854775807") return -1;
return (int)(long.Parse(strValue.Substring(0, strValue.Length - 7)) - 11644473600);
}
If somebody need convert it in MySQL
SELECT timestamp,
FROM_UNIXTIME(ROUND((((timestamp) / CAST(10000000 AS UNSIGNED INTEGER)))
- CAST(11644473600 AS UNSIGNED INTEGER),0))
AS Converted FROM events LIMIT 100
Also here's a pure C#ian way to do it.
(Int32)(DateTime.FromFileTimeUtc(129477880901875000).Subtract(new DateTime(1970, 1, 1))).TotalSeconds;
Here's the result of both methods in my immediate window:
(Int32)(DateTime.FromFileTimeUtc(long.Parse(strValue)).Subtract(new DateTime(1970, 1, 1))).TotalSeconds;
1303314490
(int)(long.Parse(strValue.Substring(0, strValue.Length - 7)) - 11644473600)
1303314490
DateTime.FromFileTimeUtc(long.Parse(strValue))
{2011-04-20 3:48:10 PM}
Date: {2011-04-20 12:00:00 AM}
Day: 20
DayOfWeek: Wednesday
DayOfYear: 110
Hour: 15
InternalKind: 4611686018427387904
InternalTicks: 634389112901875000
Kind: Utc
Millisecond: 187
Minute: 48
Month: 4
Second: 10
Ticks: 634389112901875000
TimeOfDay: {System.TimeSpan}
Year: 2011
dateData: 5246075131329262904