Would you rather use a version number (that will increment) or a timestamp to check for concurrency problems?
Version number is better because no matter what format of time you use, it can still be thrown off if the server's clock is wrong or becomes wrong over time.
I have to admit, I've used a time stamp in many databases because it can serve a dual purpose of concurrency checking as well as readability for when the data was last changed. However, using a version number is really the better way to go.
I'd use a version number, particularly if the resource may ever be updated more than the resolution of your timestamps (e.g. if you're storing timestamps to a resolution of seconds, if you have multiple updates within a single second, your versioning will break).
Lieven,
I know you didn't specify SQL Server, but IF you are talking about SQL Server then the datatype TimeStamp would be the best method to use. Despite its description it doesn't really have anything to do the date and time. It's actually just a binary number which changes every time the row has been modified. Thus, if any amendments are made to the row then you kno the timestamp column will change. This has the advantage over version numbers because you, the programmer, do not have to "maintain" the version number.
Actual Date/Time timestamps need to be used more carefully as another poster referred to earlier - time differences etc.
Version number. Or if I'm using timestamp, I'll make sure it is UTC - so there is no confusion with the timezone.
If you are on Windows, I recommend to use a globally unique identifier (GUID) as the version identifier.
A timestamp (even if UTC) can make problems if the clock is set by the user. An incrementing number (if hold in memory) can make problems if the server application is restarted, or if it overflows (if it's only a 16-bit or 32-bit integer).
Version number. DateTime is not unique due some reasons, for example 'Daylight saving time' - there could be two '2 AM' h. (Ambiguous Time).
I guess that its more than enough unsigned 32 bit integer, because practically there is a ZERO probability that while saving 4,294,967,295 transaction could occur.
Related
It seems like localtime_s() (which is the equivalent of the standard localtime_r) contains a critical section in MSVC.
For comparison, here are 2 sample apps, one does localtime_s in a loop, the other gmtime_s.
http://rextester.com/OQJ48177
http://rextester.com/JNDR45936
Profiling shows heavy lock contention inside isindst called from common_localtime_s<__int64>:
gmtime does not exhibit the issue:
Is there any way to work around this to get sane localtime_s performance in a multithreaded environment, provided I do need local times in my process?
Here is one proposed solution:
Record all times in whatever is the fastest. When presenting them to a user through a GUI, a log file, or whatever, do the conversion to local time then.
Since most GUIs and log output are single threaded, this should remove the contention from the rest of the program.
If the program is never presenting data to a user, then just write it out in the fast time format and use a post-processing tool to convert it or display it.
Since the standard implementation uses locks and there is not an easy way around it, you'll probably have to use a different implementation. I would recommend trying to base it off of GetTimeZoneInformationForYear, which will provide you with the offset from UTC for both standard and DST time and the dates during which DST is active. You also have the option calling this once for each year you care about and caching the result for use by all your threads.
Since gmtime_s is performing acceptably, I would recommend you use this to obtain the year. (Note that this is what localtime_s does.) Subtract off the appropriate Bias value provided by GetTimeZoneInformationForYear, then use gmtime_s again to split it into the date components.
In C++ I'm writing a function that converts time(NULL), which is all the seconds since January 1, 1970 in UTC time, to local time EST in military time format (hours:minutes:seconds). I'm honestly stumped how to mathematically do this so that the program stays accurate as time moves forward.
Also I'm well aware that there is a local time function but I'd like to build this function from the ground up. Does anyone have any advice or tips?
Also I'm well aware that there is a local time function but I'd like to build this function from the ground up. Does anyone have any advice or tips?
Why would you want to do this when there are plenty of free and well-tested packages? As mentioned in the comments, getting daylight savings time correct is non-trivial. Existing packages do just that, and they do it right, based on the IANA tzinfo database.
C options:
std::localtime(). This function uses a global variable; it is not thread safe.
localtime_r(). This is a POSIX function and is not a part of the C++ library. It does not exist on Windows.
localtime_s(). This is an optional C11 function. Even if it exists on your machine, it might not be a part of <ctime>.
C++ options:
Boost Date-Time, https://github.com/boostorg/date_time .
Howard Hinant's date-time module, https://github.com/HowardHinnant/date .
localtime() from glibc should do the job of calculating the date, provided the environment is set to the correct timezone; else use gmtime(). Building a string from the values is a separate job, see strftime() for that.
http://linux.die.net/man/3/localtime
If you want to learn about the algorithms for converting a count of days to a year/month/day triple (and back), here they are, highly optimized, explained in pains-taking detail (don't read while operating heavy machinery):
http://howardhinnant.github.io/date_algorithms.html
You should also know that most (all?) implementations of time() track an approximation of UTC called Unix Time. This count treats leap seconds simply as clock corrections to an imperfect clock. That means you can ignore the effect of leap seconds when converting Unix Time seconds to days (just divide by 86400).
For converting to EST, you have some choices (in order of increasing difficulty and accuracy):
You can ignore daylight savings time and always take the offset as -5 hours.
You can assume the current daylight savings rules, ignoring the fact that they have changed many times in the past, and will likely change again.
You can get the past and present rules from the IANA timezone database,
or your OS's local equivalent.
There are two things which you will need to consider:
1. Leap years
One extra day in a year, that is possible to calculate mathematically.
2. Leap seconds
Seconds inserted or removed as needed (so a minute can have 61 or 59 seconds).
Those are irregular and you will need a lookup table for them. Otherwise your conversion routine will not be correct.
List of them is available for example here: https://en.wikipedia.org/wiki/Leap_second
Is there a way to logically (not absolutely) add a time period to a local time and get a valid local time in the future?
For example, adding one day to 11 am on day 1 becomes 11 am on day 2 even if there's a daylight savings change (+/- one hour) in the meantime? If the time doesn't exist, it could return an error or the next valid time.
C++ standard mechanisms, Windows API functions, STL and/or Boost classes are all fine.
It sort of depends on your semantics.
If you are interested in what the local time might be in exactly half a day, then yes: you convert to UTC, add half a day, and convert back to local time.
On the other hand, if you operate primarily on local times (we do this in electricity load forecasting, for example), you might go the other way, ie 'how many hours away is our half-day future event?': you treat the local time as if it were UTC, add half a day, then treat that as a local time and convert back to UTC to do sanity checks - you have to be aware that some local times don't exist, and some local times are ambiguous.
If you're starting with a local time such as 2013-07-02 18:30:00 and you just want to go to the next calendar day at the same time, then despite the suggestions of others - you really don't want to involve UTC.
You are working in terms of calendar time. UTC is for measuring instantaneous time. Calendar time can have discontinuities for DST changes, but instantaneous time does not.
When you add a day to instantaneous time, you are always adding exactly 24 hours. But when you add a day to calendar time, you are advancing the calendar by one day position. That is subtle, but distinctly different. A calendar day might have 23, 24, or 25 actual hours to it.
When it comes to discontinuities, you will need to deal with the "spring-forward" transition, when the clocks skip an hour. If you add a day and end up in this gap, you would be referring to a time that doesn't exist on the calendar. You need to decide what you want to have happen in this case. Should it advance to the next possible time? Or should it error? Or perhaps you will add one more hour in this case. It's up to you.
The other discontinuity is the "fall-back" transition, when the clocks roll back an hour. There may be two actual instants that this single calendar position could refer to - but since you were only interested in the calendar time, then that shouldn't affect you directly. If you do want to know what instant it maps to, then you would again need to make a decision - perhaps involving some business logic or asking the user via UI.
In order to know where the discontinuities are, you will need a time zone database. The IANA database is the defacto standard.
For C++, you should look into using Boost's Date and Time support. They do use the standard database (aka "ZoneInfo"), and discuss using it here. You probably want to use a local_time.
If at all possible, stay away from POSIX time zone settings like PST8DST, while Boost supports them, they are cryptic and generally difficult to use internationally. The IANA time zones have lookup keys consisting of area and location, such as America/New_York or Europe/London.
Also look in Boost at local_time::ambiguous_result and local_time::time_label_invalid. Those will tell you if you are in a discontinuity.
I will also say that (IMHO), working with local date times, time zone, DST, etc. is much more complex in C++ than in any other language. There are great solutions for PHP, Python, .Net, Ruby, and others. Boost probably has the most comprehensive date/time implementation for C++, but it is no where near as simple as a library like Noda Time for .Net
UPDATE
I've come to realize that Boost's support for IANA/Olson time zones is flawed. Their time zone data file strips away the rich history of the IANA/Olson time zone database - which essentially recreates all of the problems of POSIX time zones that I described in the timezone tag wiki. Therefore, I no longer recommend using Boost for time zone conversion.
Instead, consider ICU, or the time zone functions of the GNU C library (as described here) - both of which use the IANA/Olson time zone data in its entirety.
I need to be able to convert local time to UTC on a Windows machine. I found the SystemTimeToTzSpecificLocalTime API that claims to do this, but then I read its description and found this:
The SystemTimeToTzSpecificLocalTime function may calculate the local time incorrectly under the following conditions:
The time zone uses a different UTC offset for the old and new years.
The UTC time to be converted and the calculated local time are in different years.
My first rhetorical question to MS, "How difficult is it to write a reliable API?"
And then, the actual question — how do you do it to work in 100% of the times? (And not like that API suggests.)
The answer to both your questions is that it is essentially impossible. This is one of the reasons computer science folks tend to hate dealing with local time.
Here are some examples of the confounding issues: When we jump back an hour from 2:00 to 1:00, there are two 1:30's. We also have no way to know today what the daylight savings time rules, if any, will be in ten years. Time zone boundaries are sometimes moved, so just knowing that you're in Pacific time today may not be enough to tell us what your time zone offset was 85 years ago. There is no way to know today what leap seconds may or may not be inserted in the future.
If you find yourself in a situation when it is possible, you will have to code in the specific circumstances that make those operations possible in your particular case. The operations provided are generalized and provide "usually correct" results without constraining the cases in which they can be used.
Moral: Don't use local time except for display. And even then, converting times in the past or future into local times is problematic.
Since the DST periods sometimes change in certain areas, it's hard to write something that is correct in all cases, past and future.
Nevertheless, I you need this functionality, I would suggest using the C functions (localtime, mktime, ...). I recently had a problem where the Windows function SystemTimeToTzSpecificLocalTime did not work correctly on a Citrix server (Citrix does quite some strange (but advanced) things regarding timezones), but the C functions worked correctly.
I'm going to be developing a small dedicated server in C/C++ that will require uptime of forever. I've been looking into some time functions as millisecond timing is required for calculations. I have 2 problems that I'm facing:
Using a 32bit integer to store the number of milliseconds since the operation began will wrap around at about the 49 days mark resetting to zero. I have thought about using 64 bit integers, using gettimeofday to retrieve microseconds but this brings me to the second part.
There doesn't seem to be any standard system calls for getting elapsed milliseconds that are platform independant
What should I do to resolve both these issues?
Use a 64bit integer, presuming that gives you enough time
You are correct; there is no standard. One possibility would be to use the Boost DateTime library, alternately find another or roll your own.
Good Luck!
As has already been said, the first problem you are going to confront is going to obtain a reliable millisecond-precise time.
I admit I am a bit phased by the question though.
I can understand the need for precise timing (millisecond level, even microsecond) but timing a 50days at a millisecond level seems... strange.
You should perhaps review your need first, but it is rare to need more than 6 or 7 significant digits... and I am afraid that you are trying to get a one size fit them all duration object.
Perhaps that you should instead classify your durations:
a few minutes at most > use millisecond precision
otherwise > use second precision (the famous count since Jan 1st 1970)
Because... what is the sense of 1/10 second at the scale of 2 months ?
Obvious. Use 64-bit integers with platform-specific code to get the number of milliseconds. On Unix including OSX, you want gettimeofday. On Windows, good luck getting a reliable millisecond-granularity time source; the code in the Tcl library to do this is really complex as there are some evil gotchas in the area.
answer to 1: If the "millisecond timing" that you are measuring is under around 20 days, you can subtract the times as unsigned values and check the result as signed value. This should give the right result with wrapping timers (wrapping from 0xffffffff to 0x00000000). If your timing is over 20 days, you need more bits.