<cfscript>
SetTimeZone("Asia/Karachi"); //same as our windows server timezone
</cfscript>
<cfoutput>
<cfset mydate = 'June 01, 2008'>
<cfset JobStartDate=CreateODBCDateTime(mydate)>
#JobStartDate#
</cfoutput>
Error: Date value passed to date function createDateTime is unspecified or invalid. Specify a valid date in createDateTime function.
I am using ColdFusion 2021 (update 4) on windows server. Under JVM details Java Default Locale is en_US.
Error can reproduced on: cffiddle.org
Would work just fine with other dates for e.g. July 01, 2008 (okay), May 15, 2009 (okay) etc. But shows error with June 01, 2008 (error) and April 07, 2002 (error). Not sure if there might be other dates.
Additional note: Can this issue be associated with the daylight saving in Pakistan?
Daylight Saving Revival
In 2008 Pakistan used daylight saving time for the first time since 2002 to address its energy crisis. The clocks moved one hour ahead (to UTC+6) at midnight between May 31 and June 1 in 2008. The need for daylight saving time during the peak summer season in Pakistan came in light of the country’s struggle for an approximate 4000-megawatt power shortfall. (reference)[https://www.timeanddate.com/news/time/pakistan-extends-dst-2008.html]
Any help would be greatly appreciated. Thanks
I'm afraid the (originally ~) accepted answer to this isn't correct. The problem is entirely down to the daylight savings issue the original poster mentioned.
The original code is this:
<cfset mydate = 'June 01, 2008'>
<cfset JobStartDate=CreateODBCDateTime(mydate)>
As mentioned, CreateODBCDateTime expects a date/time object, not a string, so the first thing CF needs to do is to convert 'June 01, 2008' to date/time, so the equivalent of this:
<cfset mydate = createDateTime(2008,6,1,0,0,0)>
I've added the hour, minute and seconds part there because they are necessary to create a date/time object. You've given it no time part, so CF has to assume zeros there.
And guess what? on June 1 2008 under the daylight savings rules in Pakistan, there is no such thing as 00:00:00. At the stroke of midnight, time shunted forward to 01:00:00. Hence the error:
Date value passed to date function createDateTime is unspecified or invalid
It's telling you exactly what the problem is. One will always get this when trying to use a time that doesn't exist due to daylight savings vagaries. It's exactly the same situation as if one tried to specify the date part as "Feb 32" or something more obviously invalid.
One will get the same error on 2009-04-15 for the same reason: that's when daylight saving started that year.
This demonstrates why servers should always be set to UTC. There are very seldom "unexpected" gaps in the time (the odd corrective leap-second notwithstanding), so these problems simply don't arise. If you use UTC and then adjust the timezone for display for humans when necessary, CF will always get it right.
Another point. Saying that code worked fine in older versions of CF is incorrect (this came up in comments to the earlier answer). SetTimeZone was only added to CFML on ColdFusion for CF2021, and the code in the question errors on earlier versions. So whatever you were or were not experiencing / testing with on older versions of CF was not this issue.
Can this issue be associated with the daylight saving in Pakistan?
No, this issue isn't associated with the daylight saving in Pakistan.
Update: As Adam Camaron correctly mentioned...
The problem is entirely down to the daylight savings issue the original poster mentioned.
My initial findings turned out to be incorrect. Please read Adam Camerons very interesting answer to this issue for further deep explanation. His answer should be the accepted one.
I'll keep my answer post active for further reference, because it might be helpfull giving a better overview about similar issues dealing with dateTime objects in CFML.
===========================
Orignal answer:
According to the cfml documentation about CreateODBCDateTime() you need to pass a dateTime-Object as an argument to CreateODBCDateTime(). Your variable mydate = 'June 01, 2008' is just a string that represents a date, but it's not a dateTimeobject.
July 01, 2008 (okay), May 15, 2009 (okay) etc. But shows error with June 01, 2008 (error) and April 07, 2002 (error)
The cfml engine will try somehow to deal with the submitted string and cast it to the correct data type, but this may or may not work: It's simply isn't offically supported, so I'd rather not do it that way.
To create a dateTimeObject you need to parse that string to a dateObject first, e.g. using lsParseDateTime(), and like #sos correctly commented, if you are using different locales, better to always pass the correct locale that the string content represents as an attribute:
<cfoutput>
<cfset mydate = 'June 01, 2008'>
<cfset JobStartDate=CreateODBCDateTime( lsParseDateTime( mydate, "en_US" ))>
#JobStartDate#
</cfoutput>
If your dateTime data is suitable, an alternative would be creating a dateTimeObject from scratch by using createDateTime() function first, e.g.:
<cfoutput>
<cfset mydate = createDateTime(2008,5,1,0,0,0)>
<cfset JobStartDate=CreateODBCDateTime( mydate )>
#JobStartDate#
</cfoutput>
Regarding timeshifts across the world, it depends how you are getting and saving your time data and to whom in the world you are delivering it to. In worldwide environments I'd usually save dates to UTCs and output it accordingly by using TimeZone functions.
Side note... because this is commonly missunderstood so I'm posting it here just for posterity: "locales" adapts the string to typical readable (traduced) strings, as they are commonly read and identified by the respective cultures, but they don't change timeZones.
To understand that a little more, I can warmly recommend watching this video about timeZones from Lucee. It's not from ColdFusion but it explains a lot about time internationalization and timezones in CFML and some of it pitfalls.
Related
I am writing a Test Case for a REST Controller. Code below:
private SimpleDateFormat dateFormat = new SimpleDateFormat("dd/MM/yy");
#Test
public void getByExternalTransactionId() throws Exception {
EquityFeeds equityFeeds = new EquityFeeds(423,"SAPEXTXN1", "GS", "ICICI", "BUY", dateFormat.parse("22/11/13"), 101.9f, "BLO", "Y",0);
when(equityFeedsService.findByExternalTransactionId("SAPEXTXN1")).thenReturn(equityFeeds);
mockMvc.perform(MockMvcRequestBuilders.get("/equityFeeds/getByExternalTransactionId/{externalTransactionId}", "SAPEXTXN1"))
.andExpect(status().isOk())
.andExpect(content().contentType(MediaType.APPLICATION_JSON_VALUE))
.andExpect(jsonPath("$.*", Matchers.hasSize(10)))
.andExpect(jsonPath("$.id", Matchers.is(423)))
.andExpect(jsonPath("$.externalTransactionId", Matchers.is("SAPEXTXN1")))
.andExpect(jsonPath("$.clientId", Matchers.is("GS")))
.andExpect(jsonPath("$.securityId", Matchers.is("ICICI")))
.andExpect(jsonPath("$.transactionType", Matchers.is("BUY")))
// .andExpect(jsonPath("$.transactionDate", Matchers.is(dateFormat.parse("22/11/13"))))
.andExpect(jsonPath("$.transactionDate", Matchers.is(Date.parse("22/11/13"))))
.andExpect(jsonPath("$.marketValue", Matchers.is(101.9f)))
.andExpect(jsonPath("$.sourceSystem", Matchers.is("BLO")))
.andExpect(jsonPath("$.priorityFlag", Matchers.is("Y")))
.andExpect(jsonPath("$.processingFee", Matchers.is(0)));
verify(equityFeedsService, times(1)).findByExternalTransactionId("1");
verifyNoInteractions(equityFeedsService);
}
Issue:
It's breaking in transactionDate which is java.util.Date in POJO. I have tried below in the test case:
.andExpect(jsonPath("$.transactionDate", Matchers.is(dateFormat.parse("22/11/13"))))
This gives me the output
java.lang.AssertionError: JSON path "$.transactionDate"
Expected: is <Fri Nov 22 00:00:00 IST 2013>
but: was <1385058600000L>
Expected :is <Fri Nov 22 00:00:00 IST 2013>
Actual :<1385058600000L>
Then I tried:
.andExpect(jsonPath("$.transactionDate", Matchers.is(Date.parse("22/11/13"))))
This gave me the output:
java.lang.AssertionError: JSON path "$.transactionDate"
Expected: is <1412965800000L>
but: was <1385058600000L>
Expected :is <1412965800000L>
Actual :<1385058600000L>
This looks to be very close. I understand that the difference in value is because the time the date was created in POJO is milliseconds is different from the time the date was created in the .andExpect(jsonPath("$.transactionDate", Matchers.is(Date.parse("22/11/13")))) lines and hence the values are different, as also the values are long since the values are in milliseconds.
My TestCase is failing only because of this field. I am really out of my mind as to how should I solve this. I have to use java.util.Date. Any others solution for this?
Edit: If I understand correctly (not familiar with Hamcrest), you need to feed an old-fashioned java.util.Date object into the EquityFeeds constructor and test that the long value that you get back out — 1385058600000L or 1 385 058 600 000 — agrees with the Date you specified.
java.time
To construct the java.util.Date that you need:
Instant transactionTime = LocalDate.of(2013, Month.NOVEMBER, 22)
.atStartOfDay(ZoneId.systemDefault())
.toInstant();
Date oldfashionedDate = Date.from(transactionTime);
System.out.println(oldfashionedDate);
Output so far when running in Asia/Kolkata time zone:
Fri Nov 22 00:00:00 IST 2013
Now just pass oldfashionedDate as the 6th argument to the EquityFeeds constructor. The point in the above lines of code is not only that we’re using the java.time, the modern Java date and time API, and it is very clear to read from the code what it does; it also makes testing easy. To produce the expected long value:
long expectedTransactionDateMillis = transactionTime.toEpochMilli();
System.out.println(expectedTransactionDateMillis);
1385058600000
Not being familiar with Hamcrest and not having tried I would expect that you just need to fill this value into your assertion:
.andExpect(jsonPath("$.transactionDate", Matchers.is(expectedTransactionDateMillis)))
The java.util.Date class is poorly designed and long outdated. I recommend you don’t use it, and I even more strongly recommend you don’t use the notoriously troublesome SimpleDateFormat class. Even though Date is used in your production code for historical reasons, I see no reason why you should repeat that mistake in your tests. Use java.time, it is so much nicer to work with. And conversions exist for when you need an object of one of the outdated types like Date.
What went wrong in your test?
Your first attempt:
.andExpect(jsonPath("$.transactionDate", Matchers.is(dateFormat.parse("22/11/13"))))
The result of dateFormat.parse("22/11/13") is a java.util.Date. Even though this is equal to the Date in your POJO, the value in your JSON is a long representing a count of milliseconds since the epoch (a practice you may want to change if you can). Since a Date and a long can never be equal, your test fails.
You then tried:
.andExpect(jsonPath("$.transactionDate", Matchers.is(Date.parse("22/11/13"))))
Now you are getting a long value alright. We can see that the expected and actual values have the same format in the printed output. There are two things wrong:
You are using the Date.parse method that has been deprecated since 1997. It has been so because it works unreliably across time zones. Even if you insisted on using Date, you should still stay far away from those deprecated methods and constructors.
Your date string gets parsed into 1 412 965 800 000, equal to 2014-10-11T00:00+05:30, so it’s nearly a year off. That method is not only deprecated, it is also confusing and parses your date string as the 11th day of the 22nd month of 2013. There is no 22nd month of the year, but the parse method just extrapolates, keeps counting months into 2014 and ends up with October that year. One more reason not to use that method.
Link
Oracle tutorial: Date Time explaining how to use java.time.
This strange behavior has recently came to my attention, while I was testing my Rails app on local environment in which I use around_filter to set the timezone to registered user (the default timezone is UTC).
What I did was that I registered a new user in my app. My current time was 10pm GMT-5 (March 3), and this user's created_at time was saved to database to 4am UTC (March 4). Now, I know that this time is saved in database with the timezone settings, but here comes the problem:
I use a graph for visual representation of daily registered users, and when I called the following function to tell me number of users registered in the last few days:
from ||= Date.today - 1.month
to ||= Date.today
where(created_at: from..to).group('DATE(created_at)').count
It would say that this user was registered in March 4, while it was in fact registered on March 3 from my perspective.
My question is:
How should I call where function and group by a created_at column, so that the dates with be affected correctly (according to my timezone) ?
Or is there something else that I should be doing differently?
I'm not a rubyist, so I'll let someone else give the specific code, but I can answer from a general algorithmic perspective.
If you're storing UTC in the database, then you need to query by UTC as well.
In determining the range of the query (the from and to), you'll need to know the start and stop times for "today" in your local time zone, and convert those each to UTC.
For example, I'm in the US Pacific time zone, and today is March 7th, 2015.
from: 2015-03-07T00:00:00-08:00 = 2015-03-07T08:00:00Z
to: 2015-03-08T00:00:00-08:00 = 2015-03-08T08:00:00Z
If you want to subtract a month like you showed in the example, do it before you convert to UTC. And watch out for daylight saving time. There's no guarantee the offsets will be the same.
Also, you'll want to use a half-open interval range that excludes the upper bound. I believe in Ruby that this is done with three dots (...) instead of two (at least according to this).
Grouping is usually a bit more difficult. I assume this is a query against a database, right? Well, if the db you're querying has time zone support, then you could use it convert the date to your time zone before grouping. Something like this (pseudocode):
groupby(DATE(CONVERT_TZ(created_at,'UTC','America/Los_Angeles')))
Since you didn't state what DB you're using, I can't be more specific. CONVERT_TZ is available on MySQL, and I believe Oracle and Postgres both have time zone support as well.
Date.today will default to your system's set timezone (which by the way should always be UTC, here's why) so if you want to use UTC, simply do Time.zone.now.to_date if rails is set to UTC
Otherwise you should do
Time.use_zone('UTC') do
Time.zone.now.to_date
end
After this you should display the created_at dates by doing object.created_at.in_time_zone('EST')
to show it in your current timezone
I'm trying to implement correct DST adjustment handling in my alarm clock app. So I'm reading description for DYNAMIC_TIME_ZONE_INFORMATION that I use to retrieve the current DST adjustment information via the GetTimeZoneInformationForYear API, and it says the following:
DaylightDate:
A SYSTEMTIME structure that contains a date and local
time when the transition from standard time to daylight saving time
occurs on this operating system. If the time zone does not support
daylight saving time or if the caller needs to disable daylight saving
time, the wMonth member in the SYSTEMTIME structure must be zero. If
this date is specified, the StandardDate member in this structure must
also be specified. Otherwise, the system assumes the time zone data is
invalid and no changes will be applied. To select the correct day in
the month, set the wYear member to zero, the wHour and wMinute members
to the transition time, the wDayOfWeek member to the appropriate
weekday, and the wDay member to indicate the occurrence of the day of
the week within the month (1 to 5, where 5 indicates the final
occurrence during the month if that day of the week does not occur 5
times).
If the wYear member is not zero, the transition date is
absolute; it will only occur one time. Otherwise, it is a relative
date that occurs yearly.
I'm also checking the current DST adjustments observed all over the world, and if the relative DST adjustments seem pretty straightforward, I'm not exactly clear how the following adjustments could be conveyed via DYNAMIC_TIME_ZONE_INFORMATION -- with just an absolute month and a day.
For instance:
Egypt
-----
DST Start: May 15
DST End: Last Friday September
or this one:
Iran
----
DST Start: March 21–22
DST End: September 21–22
Does anyone know how to do this?
To understand the time zone structures, it helps to look at the Windows registry under the following key:
HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Time Zones\
Here you'll find all of the built-in time zones of the Microsoft time zone database, which is maintained by Microsoft via Windows Updates.
Let's look at one of the example cases you mentioned:
.\Egypt Standard Time\
.\Egypt Standard Time\Dynamic DST\
From this we can see that there are specific DST rules defined for year 2005-2011. Outside this range, we fall back to the TZI value of the root entry.
You'll notice that there the 2014 entry for Egypt is missing. That's because Egypt gave almost no notice about the upcoming change. You can expect that there will soon be a hotfix from Microsoft available with the update.
The binary data in the registry is deserialized to a REG_TZI_FORMAT structure, which looks like this:
typedef struct _REG_TZI_FORMAT
{
LONG Bias;
LONG StandardBias;
LONG DaylightBias;
SYSTEMTIME StandardDate;
SYSTEMTIME DaylightDate;
} REG_TZI_FORMAT;
An issue you should be aware of is that Windows doesn't like time zones that transition right at midnight. The workaround is that instead of saying "00:00 on the last Friday in September", you have to say "23:59:59.999 on the last Thursday in September". However, here you have to be careful because rules like these can sometime lead to erroneous derived dates. To counter that, sometimes each year will have it's own rule. The recurrence-pattern format is still used rather than the fixed-date format, mostly for consistency purposes.
However, there's another problem - This structure can only support two daylight saving time transitions in a year. One at the DaylightDate when DST begins, and one at the StandardDate when DST ends. Since Egypt is enacting DST except for Ramadan, there will be four transitions. This also happened in Egypt in 2010, and also occurs regularly in Morocco. To deal with this design flaw, Microsoft has traditionally released multiple updates, timed to coincide with the changes. (For example, see KB2297272.)
I'll assume Microsoft will push out the multiple-update changes, so for sake of example we'll leave out the Ramadan period. This rule starts DST on the 2nd Wednesday in May at 23:59:59.999 and ends it on the last Thursday in September at 23:59:59.999.
"TZI" = 88ffffff 00000000 c4ffffff 000009000400050017003b003b00e703 000005000300020017003b003b00e703
That corresponds to a REG_TZI_FORMAT structure having these values (as JSON for clarity):
{
"Bias" : -120, // Standard offset is UTC+2
"StandardBias" : 0,
"DaylightBias" : -60, // Subtract an hour for DST
"StandardDate" : {
"wYear" : 0, // Recurrence pattern
"wMonth" : 9, // September
"wDayOfWeek" : 4, // Thursday
"wDay" : 5, // Last occurrence
"wHour" : 23,
"wMinute" : 59,
"wSecond" : 59,
"wMilliseconds" : 999
},
"DaylightDate" : {
"wYear" : 0, // Recurrence pattern
"wMonth" : 5, // May
"wDayOfWeek" : 3, // Wednesday
"wDay" : 2, // Second occurrence
"wHour" : 23,
"wMinute" : 59,
"wSecond" : 59,
"wMilliseconds" : 999
}
}
I think this answer is long enough, so I'll leave it to you to extrapolate the rules for Iran if you like. Though, I'll point out that the Windows data for Iran has been incorrect since 2009 and has yet to receive an update. :-/
As a side note, if you want to specify a fixed date rule, you can provide a non-zero "real" year value. Then the day field represents the actual day - rather than the occurrence. However, this is typically avoided because it only makes sense for the Dynamic DST rules that apply to individual years. It doesn't makes sense to use a fixed-date in the generic TZI entry in the root node.
UPDATE
Microsoft has released an update for Egypt for 2014 in KB2967990.
Iran is an odd one in that the DST transition dates don't conform to the normal rules which the Microsoft registry expects. Eg: 2nd Sunday in March. So I agree that you need to use absolute dates but achieved using the registry format. The DST transition date's weekdays vary nearly every year.
Iran DST transition dates are based on the persian calendar
https://mm.icann.org/pipermail/tz/2003-March/012053.html
So the dynamic registry method would be the answer with a lot of changes every single year!
We have a web server running Windows 2003 Standard and the time zone is set to GMT-06:00 Central Time, and the box is checked to adjust for daylight saving changes.
A web service on this server queries a datetime field and when the datetime is fetched it is correct. When the dataset is returned to the client 1 hour is subtracted from the datetime if the date is Mar 9-31, Apr 1-5, Oct 26-31, or Nov 1-5. These are the dates that the DST time change can happen on.
It does not matter when the data was saved to the database. If I save a date of 4/1/2013 today, it will be returned to the client minus the 1 hour.
We have verified that SQL is storing the date correctly since it is being returned to the web service correctly.
If I convert the date to a date string at the web server before returning it to the client, the correct date string is returned.
All dates outside of the possible DST dates are good.
All dates during DST that are not on a possible change date are good.
As an example, a date saved as 4/1/2013 12:00:00 AM will be returned as 3/31/2013 11:00:00 PM.
A date saved as 4/6/2013 12:00:00 AM will be returned as 4/6/2013 12:00:00 AM.
I added a web method to the web server to return a date value of Now() and it returns the correct date.
The only thing I have found that is similar was something about a XML rule that says all datetime values are to be transmitted using an offset. I'm not 100% sure but I don't think this is it because only some dates are changed.
Any thoughts/suggestions of what else to look at to get this resolved?
Edit: I have found some dates on the dates listed above that are correct.
I'm going to take a shot at expanding this question, since i've had a similar issue in serialization of data objects.
Take a System.DateTime object that you want to write into a DataColumn. The DateTime object will by default return the DateTime.Kind property as [Unspecified]. When the DataColumn is then set to this DateTime object the DataColumn has a DateTimeMode property that resets to [UnspecifiedLocal].
There is not offest during serialization of DataColumn when the DateTimeMode is [Unspecified] or [UTC]. But when the DateTimeMode is set to [UnspecifiedLocal] or [Local], the offeset is applied. This is where the time can go up or down depending on the timezone or daylight savings configuration.
Sad news is that I can share your problem, but don't have a decent solution. Hope this helps your search.
I can only think of some ugly solutions, but i have not tested any. If i find an elegant solution i will try post again.
I need to pull a date out of a string. Since not everyone uses the official ISO format when printing their dates, it is impractical to write a date parser for every possible date format that could be used, and I need to handle as many date formats as possible - I don't control the data and can't expect it to come in a specific format.
This seems like a problem that has probably already been solved ages ago, but my Google-fu is too weak to find the solution. :(
Does there already exist a C++ library that, given a string, will return the month, day, year, hour, minute, second, etc that is referenced in that string, if any?
Pseudocode:
string s1 = "There is an expected meteor shower this Thursday,"
"August 15th 2013 at 4:39 AM.";
string s2 = "20130815T04:39:00";
date d1 = magicConverter(s1);
date d2 = magicConverter(s2);
assert(d1 == d2);
You might use the code from here, but you need to configure a mask, that tells the code which time format is used. If you write a class routine, that takes a mask and a string and gets you out the time and is able to print in any format you like, you should be well prepared. You have to look in more detail, if it also supports Daynames and Monthnames. I got it to work in python with a module providing a function that seems pretty much the same.
For more detail:
Please look at the example 2013-08-03 again. Nobody and as follows no computer is able to tell you if this date belongs to August or April, except of having a mask telling JJJJ-MM-DD or JJJJ-DD-MM. Also this library may tell you only standard masked times. So it might lead you to August in this case. But as you said it can be any date declaration, thus it does not need to follow standards, thus it can also mean March. An other possibility is to tell you about the date from the context (e.g. a table with a column of all te same time formats by looking for the increase (which would also fail if you just look at one day per month for just one year).
Another example... if I ask you 2013-05-04... to which month does it belong? You might tell me... April. I would reply "no, to the 4th of May" and vice versa for May and 5th of April. If you tell me how to solve this puzzle with two possible solutions I would understand your downvote... please think before downvoting someone trying to help you.