Hi i am trying to create a script in csh where i have to cut the name field and print it to the screen. when i do the command ( cut /etc/passwd -f5 -d":" ) by itself it works fine, all the names are in different lines, but this doesn't happen when i insert it in the script like this:
#!bin/csh
set name=`cut /etc/passwd -f5 -d":"`
echo $name
They all appear one after the other. I have tried many things but none work, what i am doing wrong?
Thanks
It's possible to do this -- but in my opinion it's not worth doing.
You can set a variable to a value that contains newlines, but the only way I know of to do so is to use a set command with a multi-line string, with backslashes to join the lines.
Here's how I did it:
% ( echo -n "set name = '" ; \
cut /etc/passwd -f5 -d":" | sed 's/$/\\\/' ; echo "'" ) \
>! tmp
% source tmp
% echo $name:q
I had to use $name:q rather than "$name"; when I type echo "$name" I get an Unmatched ". error.
As GigaWatt said in a comment, if all want to do is display the result, you're better off just executing the cut command; there's no point in saving it in a variable.
If you need to use the output of the cut command more than once, you can save it to a file:
% cut /etc/passwd -f5 -d":" > tmp
and use the contents of the file -- or you can just re-run the cut command if you don't want to create a temporary file.
It's also worth noting that the /etc/passwd file doesn't necessarily contain information about all the accounts on a system. Some systems supplement it with NIS or LDAP. getent passwd accounts for all that (unless you have an old or limited system that doesn't have the getent command).
Bourne-based shells, including bash, tend to handle this kind of thing more cleanly. In bash:
$ name="$(cut /etc/passwd -f5 -d":")"
$ echo "$name"
Only the final newline at the end of the very last line is lost; the echo command will add it. Consider using a shell other than csh, or another scripting language like Python or Perl. csh is widely considered to be a poor scripting language.
I found a Similar discussion. Basically, the new line characters are not saved in the variable when cut is run in a csh script. So we add a newline character using awk.
echo in csh doesn't understand the \n character, even with -e option. So we can just use printf. Here's the answer:
#!bin/csh
set name=`cut /etc/passwd -f5 -d":" | awk '{printf("%s\\n", $0)'}`
printf "%b\n" "$name"
Harshad
Related
I have a mainfile.txt that contains
*
* Some text
*
Using the command
while read file; do gsed -i '1i'"$(cat mainfile.txt)" "$file";
I insert the text from mainfile.txt into the beginning of every file that matches some criteria. However, it seems like the different lines in mainfile.txt are causing trouble. The error says gsed: can't find label for jump to `o'. This error does not occur when mainfile.txt contains one line only. When trying to find a solution I only found out how to insert new lines in sed, which is not exactly what I am looking for.
i requires that each line to insert end in a backslash, except the last. If that's not the case for your file, it won't work.
ed is a better choice for editing files than the non-standard sed -i, though if you're restricting yourself to GNU sed/gsed instead of whatever the OS-provided system sed is, that's less of an issue.
With either command, the best solution to insert the contents of one file in another is to use the r command instead to read the contents of a file into the buffer after the addressed line (It acts more like a than i that way):
printf "%s\n" "0r mainfile.txt" w | ed -s "$file"
Unfortunately, sed doesn't take an address of 0 to mean "before the first line" like ed does so it's harder to use r here in it.
Of course, just prepending one file to another can easily be done without either command:
cat mainfile.txt "$file" > temp.txt && mv -f temp.txt "$file"
or using sponge(1) from the moreutils package:
cat mainfile.txt "$file" | sponge "$file"
This might work for you (GNU sed):
sed -i '1ecat mainfile.txt' "$file"
On first line only of the file $file, evaluate the command cat mainfile.txt, then print all lines as normal.
$file will be updated with the lines of mainfile.txt prepended.
Alternative if the $file has at least 2 lines:
sed -i -e '1h;1r mainfile.txt' -e '1d;2H;2g' "$file"
I want to extract the output of a command run through shell script in a variable but I am not able to do it. I am using grep command for the same. Please help me in getting the desired output in a variable.
x=$(pwd)
pw=$(grep '\(.*\)/bin' $x)
echo "extracted is:"
echo $pw
The output of the pwd command is /opt/abc/bin/ and I want only /root/abc part of it. Thanks in advance.
Use dirname to get the path and not the last segment of the path.
You can use:
x=$(pwd)
pw=`dirname $x`
echo $pw
Or simply:
pw=`dirname $(pwd)`
echo $pw
All of what you're doing can be done in a single echo:
echo "${PWD%/*}"
$PWD variable represents current directory and %/* removes last / and part after last /.
For your case it will output: /root/abc
The second (and any subsequent) argument to grep is the name of a file to search, not a string to perform matching against.
Furthermore, grep prints the matching line or (with -o) the matching string, not whatever the parentheses captured. For that, you want a different tool.
Minimally fixing your code would be
x=$(pwd)
pw=$(printf '%s\n' "$x" | sed 's%\(.*\)/bin.*%\1%')
(If you only care about Bash, not other shells, you could do sed ... <<<"$x" without the explicit pipe; the syntax is also somewhat more satisfying.)
But of course, the shell has basic string manipulation functions built in.
pw=${x%/bin*}
I'm having a problem using grep.
I have a file http://pastebin.com/HxAcciCa that I want to check for certain patterns. And when I"m trying to search for it grep returns all the lines provided that the pattern already exists in the given file.
To explain more this is the code that I'm running
grep -F "ENVIRO" "$file_pos" >> blah
No matter what else I try even if I provide a whole line as a pattern bash always returns all the lines.
These are variations of what I'm trying:
grep -F "E20" "$file_pos" >> blah
grep E20 "$file_pos" >> blah
grep C:\E20-II\ENVIRO\SSNHapACS480.dll "$file_pos" >> blah
grep -F C:\E20-II\ENVIRO\SSNHapACS480.dll "$file_pos" >> blah
Also for some strange reasons when adding the -x option to grep, it doesn't return any line despite the fact that the exact pattern exists.
I've searched the web and the bash documentation for the cause but couldn't find anything.
My final test was the following
grep -F -C 1 "E20" "$store_pos" >> blah #store_pos has the same value as $file_pos
I thought maybe it was printing the lines after the result but that was not the case.
I was using the blah file to see the output.
Also I'm using Linux mint rebecca.
Finally although the naming is quite familiar this question is not similiar to Why does grep match all lines for the pattern "\'"
And finally I would like to say that I am new to bash.
I suspect The error might be due to the main file http://pastebin.com/HxAcciCa rather than the code?
From the comments, it appears that the file has carriage returns delimiting the lines, rather than the linefeeds that grep expects; as a result, grep sees the file as one huge line, that either matches or fails to match as a whole.
(Note: there are at least three different conventions about how to delimit the lines in a "plain text" file -- unix uses newline (\n), DOS/Windows uses carriage return followed by newline (\r\n), and pre-OSX versions of MacOS used just carriage return (\r).)
I'm not clear on how your file wound up in this format, but you can fix it easily with:
tr '\r' '\n' <badfile >goodfile
or on the fly with:
tr '\r' '\n' <badfile | grep ...
Check the line endings in your input file: file, wc -l.
Check you are indeed using the correct grep: which grep.
Use > to redirect the output, or | more or | less to not be confused by earlier attempts you are appending to.
Edit: Looks like your file has the wrong line endings (old Mac OS (CR) perhaps). If you have dos2unix you can try to convert them to Unix style line endings (LF).
I don't have access to a PC at the moment, but what could possibly help you troubleshoot:
1. Use grep --color -F to see if it matches correctly.
2. After your statement, use | cat -A to see if there's any surprising control characters, lines should end in $, any other characters like \I or \M can sometimes be a headache.
I suspect number 2 as it seems to be Windows output. In which case you can cat filename | dos2unix | grep stmt should solve it
Did you save the dos2unix output as another file?
Just double check the file, it should be similar to this:
[root#pro-mon9001 ~]# cat -A Test.txt
Windows^M$
Style^M$
Files^M$
Are^M$
Hard ^M$
To ^M$
Parse^M$
[root#pro-mon9001 ~]# dos2unix Test.txt
dos2unix: converting file Test.txt to Unix format ...
[root#pro-mon9001 ~]# cat -A Test.txt
Windows$
Style$
Files$
Are$
Hard$
To$
Parse$
Now it should parse properly - so just verify that it did convert the file properly
Good luck!
This should be a basic question for a lot of people, but I am a biologist with no programming background, so please excuse my question.
What I am trying to do is rename about 100,000 gzipped data files that have existing name of a code (example: XG453834.fasta.gz). I'd like to name them to something easily readable and parseable by me (example: Xanthomonas_galactus_str_453.fasta.gz).
I've tried to use sed, rename, and mmv, to no avail. If I use any of those commands on a one-off script then they work fine, it's just when I try to incorporate variables into a shell script do I run into problems. I'm not getting any errors, just no names are changed, so I suspect it's an I/O error.
Here's what my files look like:
#! /bin/bash
# change a bunch of file names
file=names.txt
while IFS=' ' read -r r1 r2;
do
mmv ''$r1'.fasta.gz' ''$r2'.fasta.gz'
# or I tried many versions of: sed -i 's/"$r1"/"$r2"/' *.gz
# and I tried many versions of: rename -i 's/$r1/$r2/' *.gz
done < "$file"
...and here's the first lines of my txt file with single space delimiter:
cat names.txt
#find #replace
code1 name1
code2 name2
code3 name3
I know I can do this with python or perl, but since I'm stuck here working on this particular script I want to find a simple solution to fixing this bash script and figure out what I am doing wrong. Thanks so much for any help possible.
Also, I tried to cat the names file (see comment from Ashoka Lella below) and then use awk to move/rename. Some of the files have variable names (but will always start with the code), so I am looking for a find & replace option to just replace the "code" with the "name" and preserve the file name structure.
I suspect I am not escaping the variable within the single tick of the perl expression, but I have poured over a lot of manuals and I can't find the way to do this.
If you're absolutely sure than the filenames doesn't contain spaces of tabs, you can try the next
xargs -n2 < names.txt echo mv
This is for DRY run (will only print what will do) - if you satisfied with the result, remove the echo ...
If you want check the existence ot the target, use
xargs -n2 < names.txt echo mv -i
if you want NEVER allow overwriting of the target use
xargs -n2 < names.txt echo mv -n
again, remove the echo if youre satisfied.
I don't think that you need to be using mmv, a simple mv will do. Also, there's no need to specify the IFS, the default will work for you:
while read -r src dest; do mv "$src" "$dest"; done < names.txt
I have double quoted the variable names as it is generally considered good practice but in this case, a space in either of the filenames will result in read not working as you expect.
You can put an echo before the mv inside the loop to ensure that the correct command will be executed.
Note that in your file names.txt, the .fasta.gz suffix is already included, so you shouldn't be adding it inside the loop aswell. Perhaps that was your problem?
This should rename all files in column1 to column2 of names.txt. Provided they are in the same folder as names.txt
cat names.txt| awk '{print "mv "$1" "$2}'|sh
How to extract whatever string comes after a matched pattern in Shell script. I know this functionality in Perl scripting, but i dont know in Shell scripting.
Following is the example,
Subject_01: This is a sample subject and this may vary
I have to extract whatever string that follows "Subject_01:"
Any help please.
It depends on your shell.
If you're using bourne shell or bash or (I believe) pdksh, then you can do fancy stuff like this:
$ string="Subject_01: This is a sample subject and this may vary"
$ output="${string#*: }"
$ echo $output
This is a sample subject and this may vary
$
Note that this is pretty limited in terms of format. The line above requires that you have ONE space after your colon. If you have more, it will pad the beginning of $output.
If you're using some other shell, you may have to do something like this, with the cut command:
> setenv string "Subject_01: This is a sample subject and this may vary"
> setenv output "`echo '$string' | cut -d: -f2`"
> echo $output
This is a sample subject and this may vary
> setenv output "`echo '$string' | sed 's/^[^:]*: *//'`"
> echo $output
This is a sample subject and this may vary
>
The first example uses cut, which is very small and simple. The second example uses sed, which can do far more, but is a (very) little heavier in terms of CPU.
YMMV. There's probably a better way to handle this in csh (my second example uses tcsh), but I do most of my shell programming in Bourne.