I want to get content in file until find a regex in content. I need this to report my last changelog
Example:
## <small>0.27.3 (2019-03-18)</small>
* Fix integration tests
* Change log message
## <small>0.27.2 (2019-03-18)</small>
* Change find to filter
* Fix bug in typo
* Format message in request
I want a regex to return oly the content of my latest version. Example:
## <small>0.27.3 (2019-03-18)</small>
* Fix integration tests
* Change log message
How can I make this using sed, grep or awk?
Thanks for this
Edit:
I can made it:
CHANGELOG_MESSAGE=$(head -n 1 CHANGELOG.md)$(head -n 20 CHANGELOG.md | tail -n 19 | sed '/##/q' | awk '!/##/')
I think that this solution is a few complex, but works
try this:
sed '1p;1,/^##/!d;/##/d' CHANGELOG.md
explanation
1p # print first line
;
1,/^##/!d # delete everything after second ##-line
;
/##/d # delete all lines start with ##
output
## <small>0.27.3 (2019-03-18)</small>
* Fix integration tests
* Change log message
Related
I have a line in a file as below:
2 14 * * * /run/opt/server/autoi.sh
And I want to insert "root" before /run/opt/server/autoi.sh as below:
2 14 * * * root /run/opt/server/autoi.sh
I have tried the following command
sed '//run/i root' filename
but it gives the following error:
sed: -e expression #1, char 0: no previous regular expression
Could you please help me to find a fix for it?
Use a different separator character, and you need to use the s command to substitute, e.g:
sed 's#/run#root /run#' filename
or without repeating /run:
sed 's#/run#root &#' filename
You are using the wrong sed command. The i command will insert a new line before the matching line, and of course, //run is not a valid regex at all.
The general form of a sed command is
<address> <action>
where address could be a regex or a line number, and action is a command.
In fact, you want an action without an address, which means it will be applied to every input line; and the action you want to perform is a substitution.
sed 's%/run%root &%' filename
We are using the & convenience shorthand to repeat the string which matched the first regex, and an alternate regex separator instead of / so that / does not itself get interpreted as a regex separator (equivalently, you could backslash-escape it, but here, that produces something called leaning toothpick syndrome).
This will print the results to standard output, not modify the file. Once you have verified that you get the results you want, you might want to add an -i option to modify the input file. (On some platforms, such as *BSD -- which includes MacOS -- you need -i '' with an empty argument.)
Simply substitute very first / with root / as follows.
sed 's/\//root \//' Input_file
sed -e 's/run/root\/run/' abc
Example:
[root#myIP tmp]# cat abc
2 14 * * * /run/opt/server/autoi.sh
[root#myIP tmp]# sed -e 's/run/root\/run/' abc
2 14 * * * /root/run/opt/server/autoi.sh
Edited: To add a username. this should do the trick
[root#myIP tmp]# sed -e 's/\//root \//' abc
2 14 * * * root /run/opt/server/autoi.sh
I'm writing a script intended to run under bash in OS X. I have Markdown files that look like this:
# File name
## Heading 1
Some text
## Heading 2
* List item 1
* List item 2
## Some other section
...
I'm trying to read everything inside Heading 2 into a variable, and I've tried using sed, grep, and perl, but can't get a working solution. Of those tools, it looked to (in theory) be possible and easiest with Perl, especially given that I need multiple lines. It looks like this regex works (at least with javascript syntax):
## Heading 2\s+(.+)\s+
I'd like to keep it a one-liner, and stick to only tools available on stock OS X (El Capitan, 10.11). Assume I only know "Heading 2" up front, not the following header caption.
With sed :
$ myvar=$(sed "/^## $1$/,/^## Heading/!d;//d;/^$/d" file)
$ echo "$myvar"
* List item 1
* List item 2
You can remove /^$/d if you want to preserve blank lines.
Update :
I've replaced single quotes with double quotes to allow shell expansion.
You can call it with ./scriptname.sh "Heading 2".
Some explanations :
/^## $1$/,/^## Heading/ applies subsequent commands to lines matching the first pattern up to next line containing the second pattern.
!d deletes all lines except those corresponding to the range.
//d matches the same pattern as the address(es) and removes it.
Using sed:
head2="$(sed -n '/## Heading 2/,/## Heading 3/{s/^## Heading .*//;p;}' file)"
echo "$head2"
* List item 1
* List item 2
Using perl:
head2="$(perl -0pe 's/(?s).*## Heading 2\s*(.*)\s*## Heading 3.*/\1/' file)"
echo "$head2"
* List item 1
* List item 2
You can also install gnu grep using home brew and use this regex:
head2="$(grep -zoP '## Heading 2\s*\K[\s\S]*(?=\s*## Heading 3)' file)"
You can use awk ranges:
awk '/^## Heading 2/,/^## Heading [^2]/ {if (!/^## Heading 2/&&!/^## Heading [^2]/) { print}}'
or with variables
awk '/s/,/e/ {if ($0 !~ s && $0 !~ e) { print}}' s='^## Heading 2' e='^## Heading [^2]'
I have the following code to replace version string from a set of files
ack --ignore-file=is:HISTORY.md -l --print0 '1\.1\.1' | xargs -0 perl -pi -e 's/1\.1\.1/1\.1\.2/g'
Now, I realized there are some lines in the doxygen comment that also have the version string like this.
/**
* Generate Tag id from Tag name
*
* #since 1.1.1
* #static
* #access public
*
*/
How can I modify the above snippet so that lines that contain #since will be excluded?
To exclude lines with #since you could try this instead of your current perl replace code:
!/\#since/ && s/1\.1\.1/1.1.2/g
or even
/\#since/ || s/1\.1\.1/1.1.2/g
Summary / 'gist of' version,
if I have a set of messages with subject [SUB] and body [BODY] like below, How can I add a newline after the subject only if [BODY] exists (And replace the place holders with *)
[SUB] some subject. [BODY] some body lines
with newline chars and !### bunch of other *## chars
without [(BODY)] or [(SUB)]... and more stuff
[SUB] Another subject. with no body [BODY]
[SUB] another [BODY] some body.
I want this to be formatted like
* some subject.
some body lines
with newline chars and !### bunch of other *## chars
without [(BODY)] or [(SUB)]... and more stuff
* Another subject. with no body
* another
some body.
What I really wanna do,
So I am trying to auto-generate my CHANGELOG.md file from the git log output. The problem is, I need to put newline char only if the body of the commit message is non empty.
The current code looks like this, (broken into two lines)
git log v0.1.0..v0.1.2 --no-merges --pretty=format:'* %s -- %cn | \
[%h](http://github.com/../../commit/%H) %n%b' | grep -v Minor | grep . >> CHANGELOG.md
and a sample output,
* Added run information display (0.1.2) -- ... | [f9b1f6c](http://github.com/../../commit/...)
+ Added runs page to show a list of all the runs and run inforation, include sorting and global filtering.
+ Updated run information display panel on the run-info page
+ Changed the links and their names around.
* Update README.md -- abc | [2a90998](http://github.com/../../commit/...)
* Update README.md -- xt | [00369bd](http://github.com/../../commit/...)
You see here, the lines starting with the * are the commits, and the lines starting on + are just a part of the body for the first commit. Right now it adds a %n (newline) in front of all the body sections regardless of whether it's empty or not. I want to add this ONLY if its non empty (probably even after removing the whitespaces)
How would I achieve this? my knowledge of sed and awk is almost non-existing, and trying to learn didn't help much.
(I will can make sure all the code in the body is indented, so it wont confuse list of commits with lists in the body)
My Answer
i'm sure jthills answer is correct (and maye even a better way to do it), but while I was looking to figure out what his meant, i came up wit this. Hope it will help myself or someone in he future,
I am pasting the full shell script that I used,
mv CHANGELOG.md CHANGELOG.md.temp
printf '### Version '$1' \n\n' > CHANGELOG.md
git log $2..$1 --no-merges --pretty=format:'[SUB]%s -- %cn | \
[%h](http://github.com/<user>/<gitrepo>/commit/%H) [BODY]%b' | grep -v Minor | \
sed '{:q;N;s/\s*\[BODY\][\n\s]*\[SUB\]/\n\[SUB\]/;b q}' | \
sed 's/\[SUB\]/* /g' |
sed 's/\[BODY\]/\n\n/'>> CHANGELOG.md
cat CHANGELOG.md.temp >> CHANGELOG.md
rm CHANGELOG.md.temp
I am basically prepending the new commit log to the CHANGELOG.md using the temp file. Please feel free to suggest shorter versions for this 3 sed commands
Tag your syntax in the git log output. This will handle inserting the newlines properly, the rest you know:
git log --pretty=tformat:'%s%xFF%x01%b%xFF%x02' \
| sed '1h;1!H;$!d;g # buffer it all (see comments for details)
s/\xFF\x01\xff\x02//g # strip null bodies
s/\xFF\x01/\n/g # insert extra newline before the rest
s/\xFF.//g # cleanup
'
(edit: quote/escape typos)
For your first file in your question, you could try the following:
awk -f r.awk input.txt
where input.txt is the input file, and r.awk is :
{
line=line $0 ORS
}
END {
while (getSub()) {
getBody()
print "* " subj
if (body) {
print ""
print body
}
}
}
function getBody(ind) {
ind=index(line,"[SUB]")
if (ind) {
body=substr(line,1,ind-1)
line=substr(line,ind)
}
else
body=line
sub(/^[[:space:]]*/,"",body)
sub(/[[:space:]]*$/,"",body)
}
function getSub(ind,ind2) {
ind=index(line,"[SUB]")
if (ind) {
ind=ind+5
ind2=index(line,"[BODY]")
subj=substr(line, ind, ind2-ind)
line=substr(line,ind2+6)
return 1
}
else
return 0
}
gives output:
* some subject.
some body lines
with newline chars and !### bunch of other *## chars
without [(BODY)] or [(SUB)]... and more stuff
* Another subject. with no body
* another
some body.
I wrestled with this way longer than expected, simply trying to get a git log output with some sed tweaking of the git message to format/extract our JIRA messages.
Here is my solution:
logsheet = "!f() { git log --format='%h ^ %<(80,trunc)%s ^ A:%<(20,trunc)%an ^ D:%ad ' --no-merges --date=short $1 | sed -e 's/\\([A-Z]*-[0-9]*\\)/\\1 ^/'; }; f"
The escapes, the shell function with a ! were all needed because I had an arg as well as a pipe.
:-)
I am running a command which returns 96 .txt files for each hour of a particular date.
so finally it gives me 24*96 files for one day in a directory.
My aim is to extract data for four months which will result in 30*24*96*4 files in a directory.
After I get the data I need to extract certain "pattern" from each of the files and display that as output.
1) Below script is only for one day where date is hardcoded in the script
2) I need to make it work for all days in a month and I need to run it from june to october
3) As data is huge , my disk will run out of space so I don't want to create these many files instead i just want to grep on the fly and get only one output file .
How can i efficiently do this ?
My shell script looks like this
for R1 in {0..9}; do
for S1 in {0..95}; do
echo $R1 $S1
curl -H "Accept-Encoding: gzip" "http://someservice.com/getValue?Count=96&data=$S1&fields=hitType,QueryString,pathInfo" | zcat > 20101008-mydata-$R1-$S1.txt
done
done
This returns the files I need.
After that, I extract a URL pattern from each of the file grep "test/link/link2" * | grep category > 1. output
you can use this awk command to get URLs
awk -vRS="</a>" '/href/&&/test.*link2/&&/category/{gsub(/.*<a.*href=\"|\".*/,"");print}' file
Here's how to loop over 4 months worth of dates
#!/usr/bin/perl
use strict;
use warnings;
use Date::Simple ':all';
for (my $date = ymd(2010,4,1), my $end = ymd(2010,8,1);$date < $end; $date++) {
my $YYYYMMDD = $date->format("%Y%m%d");
process_one_day($YYYYMMDD); # Add more formats if needed as parameters
}
sub process_one_day {
my $YYYYMMDD = shift;
# ...
# ... Insert your code to process that date
# ... Either call system() command on the sample code in your question
# ... Or better yet write a native Perl equivalent
# ...
# ... For native processing, use WWW::Mechanize to extract the data from the URL
# ... and Perl's native grep() to grep for it
}