Match datetime format with Bash REGEX - regex

Question

I have data with this datetime format in bash:
28/11/13 06:20:05 (dd/mm/yy hh:mm:ss)
I need to reformat it like:
2013-11-28 06:20:05 (MySQL datetime format)
I am using the following regex:
regex='([0-9][0-9])/([0-9][0-9])/([0-9][0-9])\s([0-9][0-9]/:[0-9][0-9]:[0-9][0-9])'
if [[$line=~$regex]]
then
$line='20$3-$2-$1 $4';
fi
This produces an error:
./filename: line 10: [[09:34:38=~([0-9][0-9])/([0-9][0-9])/([0-9][0-9])\s([0-9][0-9]/:[0-9][0-9]:[0-9][0-9])]]: No such file or directory
UPDATE:
I want to read this file "line by line", parse it and insert data in mysql database:
'filenameX':
27/11/13 12:20:05 9984 2885 260 54 288 94 696 1852 32 88 27 7 154
27/11/13 13:20:05 9978 2886 262 54 287 93 696 1854 32 88 27 7 154
27/11/13 14:20:05 9955 2875 262 54 287 93 696 1860 32 88 27 7 154
27/11/13 15:20:04 9921 2874 261 54 284 93 692 1868 32 88 27 7 154
27/11/13 16:20:09 9896 2864 260 54 283 92 689 1880 32 88 27 7 154
27/11/13 17:20:05 9858 2858 258 54 279 92 683 1888 32 88 27 7 154
27/11/13 18:20:04 9849 2853 258 54 279 92 683 1891 32 88 27 7 154
27/11/13 19:20:04 9836 2850 257 54 279 93 683 1891 32 88 27 7 154
27/11/13 20:20:05 9826 2845 257 54 279 93 683 1892 32 88 27 7 154
27/11/13 21:20:05 9820 2847 257 54 278 93 682 1892 32 88 27 7 154
27/11/13 22:20:04 9810 2844 257 54 277 93 681 1892 32 88 27 7 154
27/11/13 23:20:04 9807 2843 257 54 276 93 680 1892 32 88 27 7 154
28/11/13 00:20:05 9809 2843 257 54 276 93 680 1747 29 87 17 6 139
28/11/13 01:20:04 9809 2842 257 54 276 93 680 1747 29 87 17 6 139
28/11/13 02:20:05 9809 2843 256 54 276 93 679 1747 29 87 17 6 139
28/11/13 03:20:04 9808 2842 256 54 276 93 679 1747 29 87 17 6 139
28/11/13 04:20:05 9808 2842 256 54 276 93 679 1747 29 87 17 6 139
28/11/13 05:20:39 9807 2842 256 54 276 93 679 1747 29 87 17 6 139
28/11/13 06:20:05 9804 2840 256 54 276 93 679 1747 29 87 17 6 139
Script:
#!/bin/bash
echo "Start!"
while IFS=' ' read -ra ADDR;
do
for line in $(cat results)
do
regex='([0-9][0-9])/([0-9][0-9])/([0-9][0-9]) ([0-9][0-9]:[0-9][0-9]:[0-9]$
if [[ $line =~ $regex ]]; then
$line="20${BASH_REMATCH[3]}-${BASH_REMATCH[2]}-${BASH_REMATCH[1]} ${BASH_REMATCH[4]}"
fi
echo "insert into table(time, total, caracas, anzoategui) values('$line', '$line', '$line', '$line', '$line');"
done | mysql -user -password database;
done < filenameX
Result:
time | total | caracas | anzoategui |
0000-00-00 00:00:00 | 9 | 9 | 9 |
2027-11-13 00:00:00 | 15 | 15 | 15 |

Answer 1

Note: This answer was accepted based on fixing the bash-focused approach in the OP. For a simpler, awk-based solution see the last section of this answer.
Try the following:
line='28/11/13 06:20:05' # sample input
regex='([0-9][0-9])/([0-9][0-9])/([0-9][0-9]) ([0-9][0-9]:[0-9][0-9]:[0-9][0-9])'
if [[ $line =~ $regex ]]; then
line="20${BASH_REMATCH[3]}-${BASH_REMATCH[2]}-${BASH_REMATCH[1]} ${BASH_REMATCH[4]}"
fi
echo "$line" # -> '2013-11-28 06:20:05'
As for why your code didn't work:
As #anubhava pointed out, you need at least 1 space to the right of [[ and to the left of ]].
Whether \s works in a bash regex is platform-dependent (Linux: yes; OSX: no), so a single, literal space is the safer choice here.
Your variable assignment was incorrect ($line = ...) - when assigning to a variable, never prefix the variable name with $.
Your backreferences were incorrect ($1, ...): to refer to capture groups (subexpressions) in a bash regex you have to use the special ${BASH_REMATCH[#]} array variable; ${BASH_REMATCH[0]} contains the entire string that matched, ${BASH_REMATCH[1]} contains what the first capture group matched, and so on; by contrast, $1, $2, ... refer to the 1st, 2nd, ... argument passed to a shell script or function.
Update, to address the OP's updated question:
I think the following does what you want:
# Read input file and store each col. value in separate variables.
while read -r f1 f2 f3 f4 f5 f6 f7 f8 f9 f10 f11 f12 f13 f14 f15; do
# Concatenate the first 2 cols. to form a date + time string.
dt="$f1 $f2"
# Parse and reformat the date + time string.
regex='([0-9][0-9])/([0-9][0-9])/([0-9][0-9]) ([0-9][0-9]:[0-9][0-9]:[0-9][0-9])'
if [[ "$dt" =~ $regex ]]; then
dt="20${BASH_REMATCH[3]}-${BASH_REMATCH[2]}-${BASH_REMATCH[1]} ${BASH_REMATCH[4]}"
fi
# Echo the SQL command; all of them are piped into a `mysql` command
# at the end of the loop.
# !! Fill the $f<n> variables in as needed - I don't know which ones you need.
# !! Make sure the number column name matches the number of values.
# !! Your original code had 4 column names, but 5 values, causing an error.
echo "insert into table(time, total, caracas, anzoategui) values('$dt', '$f3', '$f4', '$f5');"
done < filenameX | mysql -user -password database
Afterthought: The above solution is based on improvements to the OP's code; below is a streamlined solution that is a one-liner based on awk (spread across multiple lines for readability - tip of the hat to #twalberg for the awk-based date reformatting):
awk -v sq=\' '{
split($1, tkns, "/");
dt=sprintf("20%s-%s-%s", tkns[3], tkns[2], tkns[1]);
printf "insert into table(time,total,caracas,anzoategui) values(%s,%s,%s,%s);",
sq dt " " $2 sq, sq $3 sq, sq $4 sq, sq $5 sq
}' filenameX | mysql -user -password database
Note: To make quoting inside the awk program simpler, a single quote is passed in via variable sq (-v sq=\').

Answer 2

Perl is handy here.
dt="28/11/13 06:20:05"
perl -MTime::Piece -E "say Time::Piece->strptime('$dt', '%d/%m/%y %T')->strftime('%Y-%m-%d %T')"
2013-11-28 06:20:05

Answer 3

This does the trick without any overly complicated regex invocations:
echo "28/11/13 06:20:05" | awk -F'[/ ]' \
'{printf "20%s-%s-%s %s\n", $3, $2, $1, $4}'
Or, as suggested by #fedorqui in the comments, if the source of your timestamp is date, you can just give it the formatting options you want...

Answer 4

Spaces are mandatory in BASH so use:
[[ "$line" =~ $regex ]] && echo "${line//\//-}"
Also you cannot use \s in BASH so use this regex:
regex='([0-9][0-9])/([0-9][0-9])/([0-9][0-9]) ([0-9][0-9]:[0-9][0-9]:[0-9][0-9])'

Answer 5

thanks all for the sample above.
"T" not appended
$line='"2020-11-26 10:20:01.000000","the size of the table is 3.5" (inches)","2020-12-11 10:20:02"'
$echo "$line" | sed -r 's#(\d{4}-\d{2}-\d{2}) (\d{2}:\d{2}:\d{2})#\2T\1#g'
"2020-11-26 10:20:01.000000","the size of the table is 3.5" (inches)","2020-12-11 10:20:02"
"T" appended only to middle of first column and not any other column with date format in the row
$awk '/[0-9]{4}-(0[1-9]|1[0-2])-(0[1-9]|[1-2][0-9]|3[0-1]) (2[0-3]|[01][0-9]):[0-5][0-9]*/{print}' test_file |sed -e 's/\s/\T/'
"2020-11-26T10:20:01.000000","the size of the table is 3.5" (inches)","2020-12-11 10:20:02"
example from above with grouping
$ line='"2020-11-26 10:20:01.000000","the size of the table is 3.5" (inches)","2020-12-11 10:20:02"'
$ regex='([0-9][0-9])-([0-9][0-9])-([0-9][0-9]) ([0-9][0-9]:[0-9][0-9]:[0-9][0-9])'
$ if [[ $line =~ $regex ]]; then line="20${BASH_REMATCH[3]}-${BASH_REMATCH[2]}-${BASH_REMATCH[1]}T${BASH_REMATCH[4]}"; fi
$ echo "$line"
2026-11-20T10:20:01
#...the intention is to append "T" between date and time (same field) on all fields within huge csv file with millions of records, not just first column, all having same date format YYYY-MM-DD HH24:MI:SS