I'm developing a data fusion pipeline. It contains a wrangler node where I'm trying to create a new field that will contain the system date in timestamp format (yyyy-MM-dd'T'HH-mm-ss).
I've tried using the sentence:
set-column :sysdate (${logicalStartTime(yyyy-MM-dd'T'HH-mm-ss)})
But I receive the error:
Caused by: io.cdap.wrangler.api.DirectiveParseException: Error encountered while parsing 'set-column' : Error encountered while compiling '( 2022 -12-01T16-29-32 ) ' at line '1' and column '14'. Make sure a valid jexl transformation is provided.
Which would be the correct sentence?
I've tried:
set-column :sysdate (${logicalStartTime(yyyy-MM-ddHH-mm-ss)})
Which will result in something like "1877", as it substracts the numbers, and also tried:
set-column :sysdate (${logicalStartTime(yyyyMMddHHmmss)})
but the format isn't correct and can only be written if the field is a String.
You have the correct method, just incorrect syntax. The syntax you are looking for is set-column :sysdate ${logicalStartTime(yyyy-MM-dd'T'HH-mm-ss)}, you have to remove (). Then you can convert the string in datetime pattern in this format parse-as-datetime :sysdate "yyyy-MM-dd'T'HH-mm-ss".
I have a google sheet script that fetches youtube's video durations. The problem is the time data is in the ISO 8601 format.
For example:
PT3M23S
The formula I'm using right now does a good job converting this into a more readable format.
=iferror(REGEXREPLACE(getYoutubeTime(B20),"(PT)(\d+)M(\d+)S","$2:$3"))
It converts the above into a more readable format 3:23
Now the issue at hand is if the duration of the video is exactly 3 minutes or if the video is shorter than 1 minute regexreplace doesn't reformat it.
Instead it reads
PT4M OR PT53S
Is there a way to edit the formula to address each variant that potential could occur?
Where it would format PT4M into 4:00 or PT53S into 0:53
Lastly, if the seconds in the duration are between 1-9 the API returns a single digit value for the seconds. Which means the formula above will look wrong. For example, PT1M1S is formatted into 1:1 when it should read 1:01
It would be great if the formula could account for the first 9 seconds and add a 0 to make it more readable.
Thanks for reading this far, if anyone could help me out I'd very much appreciate it.
Just in case its easier to do this within the script itself here's the custom script that retrieves the video duration.
function getYoutubeTime(videoId){
var url = "https://www.googleapis.com/youtube/v3/videos?part=contentDetails&id=" + videoId;
url = url + "&key=";
var videoListResponse = UrlFetchApp.fetch(url);
var json = JSON.parse(videoListResponse.getContentText());
return json["items"][0]["contentDetails"]["duration"];
}
Is very ugly, but seems to work for the examples provided:
=iferror(left(mid(A1,3,len(A1)-2),find("M",mid(A1,3,len(A1)-2))-1)*60,0)+substitute(REGEXreplace(mid(A1,3,len(A1)-2),"(.+M)",""),"S","")
Outputs seconds, eg 203 from PT3M23S. To change to 00:03:23 wap the above formula in ( ... )/86400 and format result as Time.
Script Solution:
function iso8601HMparse(str) {
return str.replace(/PT(\d+(?=M))?M?(\d+(?=S))?S?/g,function(mm,p1,p2){//regex to get M and S value
return [0,p1,p2].map(function(e){
e = e ? e:0;
return ("00"+e).substr(-2); //fix them to 2 chars
}).join(':');
})
}
Splice it in your script like:
return iso8601HMparse(json["items"][0]["contentDetails"]["duration"].toString());
Spreadsheet Function:
=TEXT(1*REGEXREPLACE(REGEXREPLACE(A1,"PT(\d+M)?(\d+?S)?","00:00$1:00$2"),"[MS]",),"MM:SS")
Late to the party, but here's what I'm using:
=TIMEVALUE(
IFERROR(TEXT(MID(A1,3,FIND("H",A1)-3),"00"),"00")&":"&
IFERROR(TEXT(MID(A1,IFERROR(FIND("H",A1)+1,3),FIND("M",A1)-IFERROR(FIND("H",A1)+1,3)),"00"),"00")&":"&
IFERROR(TEXT(MID(A1,IFERROR(FIND("M",A1)+1,3),FIND("S",A1)-IFERROR(FIND("M",A1)+1,3)),"00"),"00"))
You can also stick ARRAYFORMULA in front of this and change A1 to a a column to get values for a whole list of them.
Here is my scenario. I am getting this date from a database:
11-AUG-15 10.38.00.000000000 AM
Is there any way to format this string to look something similar to mm/dd/yy?
So far I have tried the following with no luck:
DateFormat()
CreateODBCDate()
LSParseDateTime()
Every time I use one of the above, I get the following error:
11-AUG-15 10.38.00.000000000 AM is an invalid date or time string.
Any advise will be greatly appreciated.
Thanks!
parseDateTime("11-AUG-15 10.38.00.000000000 AM", "dd-MMM-yy hh.mm.ss.S aa");
Run me: http://trycf.com/gist/aac6d63777ae1b0e9aa3/acf?theme=monokai
Then you are free to use DateFormat() or DateTimeFormat()to format the date object.
I want all my files to be off format: 2013-03-31_142436.jpg
i.e. %Y-%m-%d_%H%M%S
I have a script to rename that way but would like to check if the filename is of the format first. I do:
for filename in files:
# check filename not already in file format
filename_without_ext = os.path.splitext(filename)[0];
How do I check filename_without_ext is of format %Y-%m-%d_%H%M%S
Use re:
import re
if re.match(r'\d{4}-\d{2}-\d{2}_\d{6}$', filename_without_ext):
pass # of the right format
This will just check it looks like it has a chance of being a valid date. Use Martijn's answer if you require it to be a valid date.
Just try to parse it as a timestamp:
from time import strptime
try:
strptime(filename_without_ext, '%Y-%m-%d_%H%M%S')
except ValueError:
# Not a valid timestamp
The strptime() test has the advantage that it guarantees that you have a valid datetime value, not just a pattern of digits that still could represent an invalid datetime ('1234-56-78_987654' is not a valid timestamp, for example).
I have some code that reads a datetime from a sqlite database, the datetime is returned as a string. when I try to convert it to a date using QDateTime::FromString it returns an invalid date. Below is the time as returned from the database and conversion.
Why is this failing to parse?
// -this is the value returned from the DB currentServerTime=2012-01-17 19:20:27.0
QString format("yyyy/MM/dd hh:mm:ss");
QString qCurrentServerTime(currentServerTime);
now = QDateTime::fromString(qCurrentServerTime, format);
No expert in QT, but if QDateTime::fromString() works as one would (reasonably) expect and according to this, you're not using the correct pattern.
You indicate the string read from the sqllite database is like "2012-01-17 19:20:27.0", then your format should be like yyyy-MM-dd HH:mm:ss.z.
In detail:
Your separator should by '-' not '/' (as you show in the example)
The time seems to be in 24 hours format (19 -> 7 p.m.) (so use HH instead of hh)
You have one digit for milliseconds, so add .z.