I managed to extract the following response and comma separate it. It's comma seperated string and I'm only interested in comma separated values of the account_id's. How do you pattern match using sed?
Input: ACCOUNT_ID,711111111119,ENVIRONMENT,dev,ACCOUNT_ID,111111111115,dev
Expected Output: 711111111119, 111111111115
My $input variable stores the input
I tried the below but it joins all the numbers and I would like to preserve the comma ','
echo $input | sed -e "s/[^0-9]//g"
I think you're better served with awk:
awk -v FS=, '{for(i=1;i<=NF;i++)if($i~/[0-9]/){printf sep $i;sep=","}}'
If you really want sed, you can go for
sed -e "s/[^0-9]/,/g" -e "s/,,*/,/g" -e "s/^,\|,$//g"
$ awk '
BEGIN {
FS = OFS = ","
}
{
c = 0
for (i = 1; i <= NF; i++) {
if ($i == "ACCOUNT_ID") {
printf "%s%s", (c++ ? OFS : ""), $(i + 1)
}
}
print ""
}' file
711111111119,111111111115
...for all characters but the first letter of every word on a line excluding the first word. All text is English language.
Would like to use sed to convert input like this:
Mary had a little lamb
It's fleece was white as snow
to this:
Mary h__ a l_____ l___
It's f_____ w__ w____ a_ s___
For a project that looks at cued recall.
Looked at several intros to sed and regex. Would be using the flavor of sed on the terminal shipped with MacOS 10.14.5.
This might work for you (GNU sed):
sed -E 'h;y/'\''/x/;s/\B./_/g;G;s/\S+\s*(.*)\n(\S+\s*).*/\2\1/' file
Make a copy of the current line in the hold space. Translate ''s to `x's so that such words can be filled with underscores other than the first letter of each word. Append the copied line and using grouping and back references replace the first word of the line unadulterated.
sed is for doing simple s/old/new operations on individual strings, that is all. For anything else you should be using awk, e.g. with GNU awk for the 3rd arg to match():
$ awk '{
out = $1
$1 = ""
while ( match($0,/(\S)(\S*)(.*)/,a) ) {
out = out OFS a[1] gensub(/./,"_","g",a[2])
$0 = a[3]
}
print out $0
}' file
Mary h__ a l_____ l___
It's f_____ w__ w____ a_ s___
With any awk in any shell on every UNIX box including the default awk on MacOS:
$ awk '{
out = $1
$1 = ""
while ( match($0,/[^[:space:]][^[:space:]]*/) ) {
str = substr($0,RSTART+1,RLENGTH-1)
gsub(/./,"_",str)
out = out OFS substr($0,RSTART,1) str
$0 = substr($0,RSTART+RLENGTH)
}
print out $0
}' file
Mary h__ a l_____ l___
It's f_____ w__ w____ a_ s___
Here is another awk script (all awk versions), I enjoyed creating for this quest.
script.awk
{
for (i = 2; i <= NF; i++) { # for each input word starting from 2nd word
head = substr($i,1,1); # output word head is first letter from current field
tail = substr("____________________________", 1, length($i) - 1); # output word tail is computed from template word
$i = head tail; # recreate current input word from head and tail
}
print; # output the converted line
}
input.txt
Mary had a little lamb
It's fleece was white as snow
run:
awk -f script.awk input.txt
this could be also condensed into single line:
awk '{for (i = 2; i <= NF; i++) $i = substr($i,1,1) substr("____________________________", 1, length($i) - 1); print }' input.txt
output is:
Mary h__ a l_____ l____
It's f_____ w__ w____ a_ s___
I enjoyed this task.
I have this command here, and I have a problem achieving a good format.
In this lines,
DATE*2014*09*23
VAL*0001*ABC
N3*Sample
VAL*0002*XYZ
My desired output here is like this:
["ABC", "XYC"]
I tried this code:
perl -nle 'print $& if /VAL\*[0-9]*\*\K.*/' file | awk '{ printf "\"%s\",", $0 }'
resulting only:
"ABC","XYZ",
Another thing is that when printing only one value.
If it happens that a file is like this:
DATE*2014*09*23
VAL*0001*ABC
N3*Sample
my desired output would only be like this (ignoring the output of having []):
"ABC"
You can do all with awk:
#!/usr/bin/awk -f
BEGIN {FS="*"; i=0; ORS=""}
$1=="VAL" {a[i++]=$3}
END {
if (i>1) {
print "[\"" a[0]
for (j = 1; j < i; j++)
print "\",\"" a[j]
print "\"]"
}
if (i==1)
print "\"" a[0] "\""
}
I have an output file that I am trying to process into a formatted csv for our audit team.
I thought I had this mastered until I stumbled across bad data within the output. As such, I want to be able to handle this using awk.
MY OUTPUT FILE EXAMPLE
Enter password ==>
o=hoster
ou=people,o=hoster
ou=components,o=hoster
ou=websphere,ou=components,o=hoster
cn=joe-bloggs,ou=appserver,ou=components,o=hoster
cn=joe
sn=bloggs
cn=S01234565
uid=bloggsj
cn=john-blain,ou=appserver,ou=components,o=hoster
cn=john
uid=blainj
sn=blain
cn=andy-peters,ou=appserver,ou=components,o=hoster
cn=andy
sn=peters
uid=petersa
cn=E09876543
THE OUTPUT I WANT AFTER PROCESSING
joe,bloggs,s01234565;uid=bloggsj,cn=joe-bloggs,ou=appserver,ou=components,o=hoster
john,blain;uid=blainj;cn=john-blain,ou=appserver,ou=components,o=hoster
andy,peters,E09876543;uid=E09876543;cn=andy-peters,ou=appserver,ou=components,o=hoster
As you can see:
we always have a cn= variable that contains o=hoster
uid can have any value
we may have multiple cn= variables without o=hoster
I have acheived the following:
cat output | awk '!/^o.*/ && !/^Enter.*/{print}' | awk '{getline a; getline b; getline c; getline d; print $0,a,b,c,d}' | awk -v srch1="cn=" -v repl1="" -v srch2="sn=" -v repl2="" '{ sub(srch1,repl1,$2); sub(srch2,repl2,$3); print $4";"$2" "$3";"$1 }'
Any pointers or guidance is greatly appreciated using awk. Or should I give up and just use the age old long winded method a large looping script to process the file?
You may try following awk code
$ cat file
Enter password ==>
o=hoster
ou=people,o=hoster
ou=components,o=hoster
ou=websphere,ou=components,o=hoster
cn=joe-bloggs,ou=appserver,ou=components,o=hoster
cn=joe
sn=bloggs
cn=S01234565
uid=bloggsj
cn=john-blain,ou=appserver,ou=components,o=hoster
cn=john
uid=blainj
sn=blain
cn=andy-peters,ou=appserver,ou=components,o=hoster
cn=andy
sn=peters
uid=petersa
cn=E09876543
Awk Code :
awk '
function out(){
print s,u,last
i=0; s=""
}
/^cn/,!NF{
++i
last = i == 1 ? $0 : last
s = i>1 && !/uid/ && NF ? s ? s "," $NF : $NF : s
u = /uid/ ? $0 : u
}
i && !NF{
out()
}
END{
out()
}
' FS="=" OFS=";" file
Resulting
joe,bloggs,S01234565;uid=bloggsj;cn=joe-bloggs,ou=appserver,ou=components,o=hoster
john,blain;uid=blainj;cn=john-blain,ou=appserver,ou=components,o=hoster
andy,peters,E09876543;uid=petersa;cn=andy-peters,ou=appserver,ou=components,o=hoster
If you want to try this on a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk , /usr/xpg6/bin/awk , or nawk
This awk script works for your sample and produces the sample output:
BEGIN { delete cn[0]; OFS = ";" }
function print_info() {
if (length(cn)) {
names = cn[1] "," sn
for (i=2; i <= length(cn); ++i) names = names "," cn[i]
print names, uid, dn
delete cn
}
}
/^cn=/ {
if ($0 ~ /o=hoster/) dn = $0
else {
cn[length(cn)+1] = substr($0, index($0, "=") + 1)
uid = $0; sub("cn", "uid", uid)
}
}
/^sn=/ { sn = substr($0, index($0, "=") + 1) }
/^uid=/ { uid = $0 }
/^$/ { print_info() }
END { print_info() }
This should help you get started.
awk '$1 ~ /^cn/ {
for (i = 2; i <= NF; i++) {
if ($i ~ /^uid/) {
u = $i
continue
}
sub(/^[^=]*=/, x, $i)
r = length(r) ? r OFS $i : $i
}
print r, u, $1
r = u = x
}' OFS=, RS= infile
I assume that there is an error in your sample output: in the 3d record the uid should be petersa and not E09876543.
You might want look at some of the "already been there and done that" solutions to accomplish the task.
Apache Directory Studio for example, will do the LDAP query and save the file in CSV or XLS format.
-jim
I am stuck on the last step and hope someone could help me out.
I have 2 files:
file1 =
Green Apple
file2 =
Green Apple/quantity/100
Red Peach/quantity/200
My code:
grep -f file1 file2 | sed 's/quantity(.*)/totalXYZ/'
= Green Apple/totalXYZ
I would like to apply it to file2 so the result would look like this:
file2 =
Green Apple/totalXYZ
Red Peach/quantity/200
Thanks
You can get it using AWK this way:
awk -f grepandreplace.awk file2 file1
The code of "grepandreplace.awk" is:
#!/usr/bin/awk -f
BEGIN {
FS = "/";
f_grep = ARGV[2];
ARGC = 2;
while ((getline < f_grep) > 0) {
grep[$1] = "";
}
}
{
if ($1 in grep) {
print $1"/totalXYZ";
} else {
print $0;
}
}
And I got your desired output:
Green Apple/totalXYZ
Red Peach/quantity/200
Any questions? leave me a comment.
You can get it in one line using 2 greps: the first one for matching and the second one using the invert-matching (-v).
And then append the second output to the first one:
echo -e `grep -f file1 file2 | sed 's/quantity.*/totalXYZ/'`"\n"`grep -vf file1 file2`