Remove line breaks using linux command - regex

Question

My database log file looks like this...
vi test.txt
'2020-03-27T08:00:24Z UTC [ db=xdb user=root pid=9037 userid=100 xid=36076898 ]' LOG: SELECT nspname FROM pg_namespace ORDER BY nspname
'2020-03-27T08:00:24Z UTC [ db=xdb user=root pid=9037 userid=100 xid=36076899 ]' LOG: SET search_path TO "public"
'2020-03-27T08:00:24Z UTC [ db=xdb user=root pid=9037 userid=100 xid=36076900 ]' LOG: SELECT typname
FROM pg_type
WHERE typnamespace = (SELECT oid FROM pg_namespace WHERE nspname = current_schema())
'2020-03-27T08:00:24Z UTC [ db=xdb user=root pid=9037 userid=100 xid=36076897 ]' LOG: SELECT datname FROM pg_database ORDER BY datname
Because of line breaks like '\n' and '\r' I am not able to check the complete query. For e.g.
# grep '2020' test.txt
'2020-03-27T08:00:24Z UTC [ db=xdb user=root pid=9037 userid=100 xid=36076898 ]' LOG: SELECT nspname FROM pg_namespace ORDER BY nspname
'2020-03-27T08:00:24Z UTC [ db=xdb user=root pid=9037 userid=100 xid=36076899 ]' LOG: SET search_path TO "public"
'2020-03-27T08:00:24Z UTC [ db=xdb user=root pid=9037 userid=100 xid=36076900 ]' LOG: SELECT typname
'2020-03-27T08:00:24Z UTC [ db=xdb user=root pid=9037 userid=100 xid=36076897 ]' LOG: SELECT datname FROM pg_database ORDER BY datname
As you can see, the line "FROM pg_type" is missing in the above output. How do I remove line breaks in this text file? I will need to keep line break before '2020' since that is another query.
How do I write a regular expression that will remove all breaks between "LOG:" and "'2020-"

Answer 1

A bit of a dirty solution, but you could do something like:
cat my_log_file.log | tr '\n' ' ' | sed "s/\('[0-9]\{4\}\)/\r\n\1/g"
# OR, simpler version:
tr '\n' ' ' < my_log_file.log | sed "s/\('[0-9]\{4\}\)/\r\n\1/g"
basically, you delete all '\n', and then you add them again where they should be

Answer 2

$ awk '{printf "%s%s", (/^\047/ ? ors : ofs), $0; ors=ORS; ofs=OFS} END{printf "%s", ors}' file
'2020-03-27T08:00:24Z UTC [ db=xdb user=root pid=9037 userid=100 xid=36076898 ]' LOG: SELECT nspname FROM pg_namespace ORDER BY nspname
'2020-03-27T08:00:24Z UTC [ db=xdb user=root pid=9037 userid=100 xid=36076899 ]' LOG: SET search_path TO "public"
'2020-03-27T08:00:24Z UTC [ db=xdb user=root pid=9037 userid=100 xid=36076900 ]' LOG: SELECT typname FROM pg_type WHERE typnamespace = (SELECT oid FROM pg_namespace WHERE nspname = current_schema())
'2020-03-27T08:00:24Z UTC [ db=xdb user=root pid=9037 userid=100 xid=36076897 ]' LOG: SELECT datname FROM pg_database ORDER BY datname

Answer 3

awk 'match($0, r) && NR>1 {print ""}
{printf "%s", $0} END {print ""}
' r="^'2020" test.txt

Answer 4

This might work for you (GNU sed):
sed '/^'\''2020/{:a;N;/^\('\''2020\).*\n\1/!s/\n/ /;ta;P;D}' file
If a line begins '2020, append the next line and if that line does not begin '2020, replace the newline between the lines with a space, append the next line and repeat. Otherwise print/delete the first line and repeat.
The OP has expressed How do I write a regular expression that will remove all breaks between "LOG:" and "'2020-".To handle any year, use:
sed '/^'\''[1-9][0-9][0-9][0-9]/{:a;N;/^'\''[1-9][0-9][0-9][0-9].*\n'\''[1-9][0-9][0-9][0-9]/!s/\n/ /;ta;P;D}' file