Unmarshalling a marshalled time object is failing, because of a few characters
Test
declare the following:
// values
now := time.Now()
timeToJSON, _ := json.Marshal(now)
var obj time.Time
json.Unmarshal(timeToJSON, &obj)
then do the following test logic:
if !assert.Equal(t,
now.String(),
obj.String()) {
t.FailNow()
}
Expected
the test to pass, and the two objects to be equal
Actual
It fails:
--- FAIL: TestFromJSON (0.00s)
D:\dev2017\GO\src\ezsoft\apiserver_sdk\model\delete\deleteModel_test.go:94:
Error Trace: deleteModel_test.go:94
Error: Not equal:
expected: "2018-09-04 10:36:18.3627338 -0400 EDT m=+0.014000801"
actual : "2018-09-04 10:36:18.3627338 -0400 EDT"
Diff:
--- Expected
+++ Actual
## -1 +1 ##
-2018-09-04 10:36:18.3627338 -0400 EDT m=+0.014000801
+2018-09-04 10:36:18.3627338 -0400 EDT
Test: TestFromJSON
FAIL
FAIL ezsoft/apiserver_sdk/model/delete 1.336s
Error: Tests failed.
NOTE
I notice that, upon inspecting the output, that, somehow, some m=+[blah] is being appended to expected/actual.
I don't know why, however, and skimming RFC 3339 doesn't give me any hints why.
time.String() is not a reliable way to testing a time's value (unless you care about the monotonic clock value, as well). From the docs (Emphasis added):
func (Time) String
func (t Time) String() string
String returns the time formatted using the format string
"2006-01-02 15:04:05.999999999 -0700 MST"
If the time has a monotonic clock reading, the returned string includes a final field "m=±", where value is the monotonic clock reading formatted as a decimal number of seconds.
The returned string is meant for debugging; for a stable serialized representation, use t.MarshalText, t.MarshalBinary, or t.Format with an explicit format string.
For your use-case, it would be better to use the output of time.MarshalText()instead of time.String():
expected, _ := now.MarshalText()
actual, _ := obj.MarshalText()
if !assert.Equal(string(expected), string(actual)) ...
Per the documentation, the m value is the monotonic clock value, which can be removed using Truncate for comparisons not for the purpose of timing. The m field does not match because it is omitted from JSON, it is only generated by time.Now().
Try like so:
// values
now := time.Now().Truncate(0) // Truncate to remove monotonic clock portion
timeToJSON, _ := json.Marshal(now)
var obj time.Time
json.Unmarshal(timeToJSON, &obj)
The monotonic clock offset was added in order to allow for accurate timing of durations which span a wall clock change (e.g. NTP update, DST change, or leap second/smear).
Related
I want to convert a readable timestamp to UNIX time.
For example: I want to convert 2018-08-24 18:42:16 to 1535136136000.
Here is my syntax:
TO_UNIXTIME('2018-08-24 06:42:16') new_year_ut
My error is:
SYNTAX_ERROR: line 1:77: Unexpected parameters (varchar(19)) for function to_unixtime. Expected: to_unixtime(timestamp) , to_unixtime(timestamp with time zone)
You need to wrap the varchar in a CAST to timestamp:
to_unixtime(CAST('2018-08-24 06:42:16' AS timestamp)) -- note: returns a double
If your timestamp value doesn't have fraction of second (or you are not interested in it), you can cast to bigint to have integral result:
CAST(to_unixtime(CAST('2018-08-24 06:42:16' AS timestamp)) AS BIGINT)
If your readable timestamp value is a string in different format than the above, you would need to use date_parse or parse_datetime for the conversion. See https://trino.io/docs/current/functions/datetime.html for more information.
Note: when dealing with timestamp values, please keep in mind that: https://github.com/trinodb/trino/issues/37
I have the following lua code:
test_me=function()
if mood == "happy" then
return true, atable
else
return false, "go away"
end
end
Luaunit test file called test_mood.lua:
test_mymood=function()
luaunit.assertEquals(mymod.test_me,true)
end
When I run this unit test, it fails like so:
1..1
# Started on Wed May 16 20:14:05 2018
not ok 1 test_mymood
test_mood.lua:72: expected: true, actual: function: 0x97c699b3e20
# Ran 1 tests in 0.004 seconds, 0 successes, 1 failure
It is passing, but it's getting the table back i think.
If you can point me in the right direction, that'd be great. Reading their docs right now but haven't found what I'm looking for.
Shouldn't it be:
luaunit.assertEquals(mymod.test_me(),true)
with parentheses so that the function result is returned instead of the function itself?
You can see it wrong whan the message from the test says the actual value is a function:
test_mood.lua:72: expected: true, actual: function: 0x97c699b3e20
------------------------------------------^
You can use table.pack() to grab all return results, include the nil.
local r = table.pack(your_function())
for i = 1, r.n do
print(i, r[i]) -- one of return result maybe nil
end
I have a very simple string of the form
YYYYMMDDHHMMSS
Basically a full date/time string. Say an example is
20170224134523
Above implies
year: 2017
month: 02
day:24
hour:13
min:45
sec:23
I want to split it so that i can have it in variables (year, month, day, hour, min, sec). This is in Scala I want to. I was thinking should I use a 6-Tuple and on the right side I will use a regex or what as the most efficient way. If I want to do it in a concise way is what I am trying to think. Little bad with regular expressions.
Can anyone help?
I may want to have each variable in the 6-tuple as option type because otherwise that will also do my sanity check? Say if any variable comes out as None, I want to throw an exception
java.text.SimpleDateFormat handles this kind of date parsing well.
scala> val sdf = new SimpleDateFormat("yyyyMMddkkmmss")
sdf: java.text.SimpleDateFormat = java.text.SimpleDateFormat#8e10adc0
scala> val date = sdf.parse("20170224134523")
date: java.util.Date = Fri Feb 24 13:45:23 PST 2017
You can get the date, day, hours, etc from a successful parse of the date as the API shows below.
scala> res0.get
getClass getDate getDay getHours getMinutes getMonth getSeconds getTime getTimezoneOffset getYear
Further, I'd suggest wrapping the parse call in a Try to handle the successful and unsuccessful parsing.
scala> val date = Try(sdf.parse("20170224134523"))
date: scala.util.Try[java.util.Date] = Success(Fri Feb 24 13:45:23 PST 2017)
scala> val date = Try(sdf.parse("asdf"))
date: scala.util.Try[java.util.Date] = Failure(java.text.ParseException: Unparseable date: "asdf")
Here's the same thing using the newer LocalDateTime instead of Date and it's deprecated methods.
LocalDateTime.parse("20170224134523", DateTimeFormatter.ofPattern("yMMddkkmmss"))
java.time.LocalDateTime = 2017-02-24T13:45:23
Because it is a date string it probably makes sense to use a dedicated date parsing library and parse to a datetime type. Fortunatly, java provides a very good one with the java.time package.
val dateTime = LocalDateTime.parse("20170224134523", DateTimeFormatter.ofPattern("yyyyMMddHHmmss"))
Which will produce a LocalDateTime object (date and time without a timezone attached). If you need more complicated string parsing you can use a DateTimeFormatterBuilder to customize the date format exactly as you need it.
With such a predictable format you can grab it by position using a substring function (from, to) into a date class.
The regex pattern to grab the sections as groups is:
(\d{4})(\d{2})(\d{2})(\d{2})(\d{2})(\d{2})
Demo
I have a table, that contains date field (let it be date s_date) and description field (varchar2(n) desc). What I need is to write a script (or a single query, if possible), that will parse the desc field and if it contains a valid oracle date, then it will cut this date and update the s_date, if it is null.
But there are one more condition - there are must be exactly one occurence of a date in the desc. If there are 0 or >1 - nothing should be updated.
By the time I came up with this pretty ugly solution using regular expressions:
----------------------------------------------
create or replace function to_date_single( p_date_str in varchar2 )
return date
is
l_date date;
pRegEx varchar(150);
pResStr varchar(150);
begin
pRegEx := '((0[1-9]|[12][0-9]|3[01])[.](0[1-9]|1[012])[.](19|20)\d\d)((.|\n|\t|\s)*((0[1-9]|[12][0-9]|3[01])[.](0[1-9]|1[012])[.](19|20)\d\d))?';
pResStr := regexp_substr(p_date_str, pRegEx);
if not (length(pResStr) = 10)
then return null;
end if;
l_date := to_date(pResStr, 'dd.mm.yyyy');
return l_date;
exception
when others then return null;
end to_date_single;
----------------------------------------------
update myTable t
set t.s_date = to_date_single(t.desc)
where t.s_date is null;
----------------------------------------------
But it's working extremely slow (more than a second for each record and i need to update about 30000 records). Is it possible to optimize the function somehow? Maybe it is the way to do the thing without regexp? Any other ideas?
Any advice is appreciated :)
EDIT:
OK, maybe it'll be useful for someone. The following regular expression performs check for valid date (DD.MM.YYYY) taking into account the number of days in a month, including the check for leap year:
(((0[1-9]|[12]\d|3[01])\.(0[13578]|1[02])\.((19|[2-9]\d)\d{2}))|((0[1-9]|[12]\d|30)\.(0[13456789]|1[012])\.((19|[2-9]\d)\d{2}))|((0[1-9]|1\d|2[0-8])\.02\.((19|[2-9]\d)\d{2}))|(29\.02\.((1[6-9]|[2-9]\d)(0[48]|[2468][048]|[13579][26])|((16|[2468][048]|[3579][26])00))))
I used it with the query, suggested by #David (see accepted answer), but I've tried select instead of update (so it's 1 regexp less per row, because we don't do regexp_substr) just for "benchmarking" purpose.
Numbers probably won't tell much here, cause it all depends on hardware, software and specific DB design, but it took about 2 minutes to select 36K records for me. Update will be slower, but I think It'll still be a reasonable time.
I would refactor it along the lines of a single update query.
Use two regexp_instr() calls in the where clause to find rows for which a first occurrence of the match occurs and a second occurrence does not, and regexp_substr() to pull the matching characters for the update.
update my_table
set my_date = to_date(regexp_subtr(desc,...),...)
where regexp_instr(desc,pattern,1,1) > 0 and
regexp_instr(desc,pattern,1,2) = 0
You might get even better performance with:
update my_table
set my_date = to_date(regexp_subtr(desc,...),...)
where case regexp_instr(desc,pattern,1,1)
when 0 then 'N'
else case regexp_instr(desc,pattern,1,2)
when 0 then 'Y'
else 'N'
end
end = 'Y'
... as it only evaluates the second regexp if the first is non-zero. The first query might also do that but the optimiser might choose to evaluate the second predicate first because it is an equality condition, under the assumption that it's more selective.
Or reordering the Case expression might be better -- it's a trade-off that's difficult to judge and probably very dependent on the data.
I think there's no way to improve this task. Actually, in order to achieve what you want it should get even slower.
Your regular expression matches text like 31.02.2013, 31.04.2013 outside the range of the month. If you put year in the game,
it gets even worse. 29.02.2012 is valid, but 29.02.2013 is not.
That's why you have to test if the result is a valid date.
Since there isn't a full regular expression for that, you would have to do it by PLSQL really.
In your to_date_single function you return null when a invalid date is found.
But that doesn't mean there won't be other valid dates forward on the text.
So you have to keep trying until you either find two valid dates or hit the end of the text:
create or replace function fn_to_date(p_date_str in varchar2) return date is
l_date date;
pRegEx varchar(150);
pResStr varchar(150);
vn_findings number;
vn_loop number;
begin
vn_findings := 0;
vn_loop := 1;
pRegEx := '((0[1-9]|[12][0-9]|3[01])[.](0[1-9]|1[012])[.](19|20)\d\d)';
loop
pResStr := regexp_substr(p_date_str, pRegEx, 1, vn_loop);
if pResStr is null then exit; end if;
begin
l_date := to_date(pResStr, 'dd.mm.yyyy');
vn_findings := vn_findings + 1;
-- your crazy requirement :)
if vn_findings = 2 then
return null;
end if;
exception when others then
null;
end;
-- you have to keep trying :)
vn_loop := vn_loop + 1;
end loop;
return l_date;
end;
Some tests:
select fn_to_date('xxxx29.02.2012xxxxx') c1 --ok
, fn_to_date('xxxx29.02.2012xxx29.02.2013xxx') c2 --ok, 2nd is invalid
, fn_to_date('xxxx29.02.2012xxx29.02.2016xxx') c2 --null, both are valid
from dual
As you are going to have to do try and error anyway one idea would be to use a simpler regular expression.
Something like \d\d[.]\d\d[.]\d\d\d\d would suffice. That would depend on your data, of course.
Using #David's idea you could filter the ammount of rows to apply your to_date_single function (because it's slow),
but regular expressions alone won't do what you want:
update my_table
set my_date = fn_to_date( )
where regexp_instr(desc,patern,1,1) > 0
I have built a match pattern in RegexBuddy which behaves exactly as I expect. But I cannot transfer this to Delphi XE, at least when using the latest built in TRegEx or TPerlRegEx.
My real world code have 6 capture group but I can illustrate the problem in an easier example. This code gives "3" in first dialog and then raises an exception (-7 index out of bounds) when executing the second dialog.
var
Regex: TRegEx;
M: TMatch;
begin
Regex := TRegEx.Create('(?P<time>\d{1,2}:\d{1,2})(?P<judge>.{1,3})');
M := Regex.Match('00:00 X1 90 55KENNY BENNY');
ShowMessage(IntToStr(M.Groups.Count));
ShowMessage(M.Groups['time'].Value);
end;
But if I use only one capture group
Regex := TRegEx.Create('(?P<time>\d{1,2}:\d{1,2})');
The first dialog shows "2" and the second dialog will show the time "00:00" as expected.
However this would be a bit limiting if only one named capture group was allowed, but thats not the case... If I change the capture group name to for example "atime".
var
Regex: TRegEx;
M: TMatch;
begin
Regex := TRegEx.Create('(?P<atime>\d{1,2}:\d{1,2})(?P<judge>.{1,3})');
M := Regex.Match('00:00 X1 90 55KENNY BENNY');
ShowMessage(IntToStr(M.Groups.Count));
ShowMessage(M.Groups['atime'].Value);
end;
I'll get "3" and "00:00", just as expected. Is there reserved words I cannot use? I don't think so because in my real example I've tried completely random names. I just cannot figure out what causes this behaviour.
When pcre_get_stringnumber does not find the name, PCRE_ERROR_NOSUBSTRING is returned.
PCRE_ERROR_NOSUBSTRING is defined in RegularExpressionsAPI as PCRE_ERROR_NOSUBSTRING = -7.
Some testing shows that pcre_get_stringnumber returns PCRE_ERROR_NOSUBSTRING for every name that has the first letter in the range of k to z and that range is dependent of the first letter in judge. Changing judge to something else changes the range.
As i see it there is at lest two bugs involved here. One in pcre_get_stringnumber and one in TGroupCollection.GetItem that needs to raise a proper exception instead of SRegExIndexOutOfBounds
The bug seems to be in the RegularExpressionsAPI unit that wraps the PCRE library, or in the PCRE OBJ files that it links. If I run this code:
program Project1;
{$APPTYPE CONSOLE}
uses
SysUtils, RegularExpressionsAPI;
var
myregexp: Pointer;
Error: PAnsiChar;
ErrorOffset: Integer;
Offsets: array[0..300] of Integer;
OffsetCount, Group: Integer;
begin
try
myregexp := pcre_compile('(?P<time>\d{1,2}:\d{1,2})(?P<judge>.{1,3})', 0, #error, #erroroffset, nil);
if (myregexp <> nil) then begin
offsetcount := pcre_exec(myregexp, nil, '00:00 X1 90 55KENNY BENNY', Length('00:00 X1 90 55KENNY BENNY'), 0, 0, #offsets[0], High(Offsets));
if (offsetcount > 0) then begin
Group := pcre_get_stringnumber(myregexp, 'time');
WriteLn(Group);
Group := pcre_get_stringnumber(myregexp, 'judge');
WriteLn(Group);
end;
end;
except
on E: Exception do
Writeln(E.ClassName, ': ', E.Message);
end;
ReadLn;
end.
It prints -7 and 2 instead of 1 and 2.
If I remove RegularExpressionsAPI from the uses clause and add the pcre unit from my TPerlRegEx component, then it does correctly print 1 and 2.
The RegularExpressionsAPI in Delphi XE is based on my pcre unit, and the RegularExpressionsCore unit is based on my PerlRegEx unit. Embarcadero did make some changes to both units. They also compiled their own OBJ files from the PCRE library that are linked by RegularExpressionsAPI.
I have reported this bug as QC 92497
I have also created a separate report QC 92498 to request that TGroupCollection.GetItem raise a more sensible exception when requesting a named group that does not exist. (This code is in the RegularExpressions unit which is based on code written by Vincent Parrett, not myself.)