First line shows how the data is saved in Excel.
Second line shows how same data is stored in CSV file.
BOOKID123,None,{"present":true,"ID":{"1234":{"present":true,"answer":{"code":23,"message":"Ready to publish"}}}}
BOOKID123,None,"{""present"":true""ID"":{""1234"":{""present"":true","answer"":{""code"":23","message"":""Ready to publish""}}}}"
Postman is able to parse the JSON shown in third column of CSV file.
But Newman command line cannot parse the JSON.
Following are the errors:
Invalid closing quote at line 2; found "\"" instead of delimiter ","
So I escaped all the double quotes. Then the next error was:
Number of columns on line 2 does not match header
which is due to the commas in the JSON.
Is the only solution to use JSON file instead of CSV file? It was easier adding data in Excel hence I chose the CSV route.
In postman support forum I've recieved the solution to put commas in qoutas: Canada"," Toronto. But it works only in runner. Newman does not accept such data fields in CSV. The only decision I suggest is to escape commas in json file for newman:
"username": "abhinav,",
forum quote
I was putting quotes round the entire string which does not work
e.g. “data2,with comma”
putting quotes round the comma only does work
e.g. data2","with comma
Related
When trying to upload a csv file to BigQuery with the following params:
bq load --quote='"' --null_marker='\\N' --source_format=CSV sendpulse.user_orders gs://data-sendpulse/user_orders_3.csv
I get an error when trying to parse the following row:
"0","63800.00","1","0","Service \"Startup Pack\""
Obviously, Bigquery doesn't treat backslash as an escape character for inner quotes, but is there a way to specify backslash as an escape character?
Tried different options and always got errors.
Update:
Quote in a quoted csv value is escaped with another quote and there is no setting for an escape character.
I don't see a better workaround than replacing all \" with ' or "" in your files.
Problem description:
Parse a csv file (with a pipe character as a delimiter) with pipe in one of the data fields. This data field will always be in an XML tags i.e., (starting tag) and (closing tag). So, I am looking to parse the csv file with some kind of exclusion logic to ignore delimiters if found within the tags.
My goal is to parse this data corrected pipe delimited file (as shown below in the Expected result) using Pentaho Data Integration tool to load into our database. After the data correction it is plain and simple to read the csv file.
Sample data:
abc| <evar29> d|e|f</evar29> | ghijk
xxx| yyyy| <evar29>z|z</evar29>
Expected Result ("|" replaced within evar29 tags with a "##"):
abc| <evar29> d ## e ## f</evar29> | ghijk
xxx| yyyy| <evar29>z ## z</evar29>
For your case: (?<=<evar29>.*)(?=.*</evar29>)\|
For general: (?<=<.+?>.*)(?=.*<.+?>)\|
Answering my own question here after reading about sed and awk. However, this doesn't seem to be working well for multiple occurrences of the pipe character within those tags. I am currently working on that. Appreciate any help.
Command: sed -n 's/<evar29>\(.*\)|\(.*\)<\/evar29>/<evar29>\1##\2<\/evar29>/pg' test.txt
Description: Substitute the pipe character that occurs in between the evar29 tags.
The string right after the evar29 starting tag is broken down and captured using capture groups and then concatenated later using the desired character (in my case ##).
Command to replace the character and write to a file is below:
sed -i 's/<evar29>\(.*\)|\(.*\)<\/evar29>/<evar29>\1##\2<\/evar29>/g' test.txt
Hope this helps anyone looking for a solution of this kind.
There is a txt file containing multiple lines with - Browser("something").page("something_else").webEdit("some").
I need to retrieve the names of the browser, page and fields (names surrounded by double quotes ) and replace the line with "something_somethingelse_some" (concatinating the names of the browser, page n filed respectively), please help.
the names can be anything so we should go with regex. Note we have to convert everything comes in the above format within the text file till the EOF..
You may try this:
^Browser\("(.*?)"\).page\("(.*?)"\).webEdit\("(.*?)"\).*$
and replace by:
$1_$2_$3
Regex Demo
trying to figure out next case:
I have txt file with parameters
environment=trank
Browser=iexplore
id=1988
Url=www.google.com
maautomate=no
When I parse this txt file with regex pattern like
/environment=([^\s]+)/
I got "trankBrow" as result, or
/Url=([^\s]+)/
I got www.google.commaautomate=no
So why second parameters appended? And how to get "trank" only?
environment=([^\\s]+)
You need to use this. \s in your case is escaping s and so the output is trankBrow because after that s is there.
I am trying to extract email details from raw text using pig.
Here's the sample data:
Sample data for email abc.123#gmail.com
Sample data for email xyz#abc.com
I am trying with REGEX method, Regular expression i took from: http://www.mkyong.com/regular-expressions/how-to-validate-email-address-with-regular-expression/
here's the script:
A = Load '----' using PigStorage as (value: chararray);
B = FOREACH A GENERATE FLATTEN(REGEX_EXTRACT_ALL(value, '^[_A-Za-z0-9-\\+]+(\\.[_A-Za-z0-9-]+)*#[A-Za-z0-9-]+(\\.[A-Za-z]{2,})$')) AS (f1: chararray)
dump B;
After dumping the output into the terminal, i am getting blank output:
()
()
Is there any problem in script syntax?
Please share some links also regarding regular expression writing, it would be very much helpful.
Your help is appreciated, thank you.
For following input data
abc.123#gmail.com
xyz#abc.com
Output of your code is
.123 .com
.com
So there are couple of problems in your code
You need to add parenthesis around the whole regex to capture the complete email address. The code should then work if you have only one token (word or email-id) in each line
If each input line can be a sentence, then you have to first tokenize and then on tokens you can to do regex match.
The reason that the regex you have works only on token and not on line is "^" indicates beginning of string and "$" indicates end of string, so the match is going to successful only when the entire line is an email-id which means you can have only one token per line.