I'd like to preface that I am unable to make changes to the underlying source code. This is code that gets checked out for each project for a team and I cannot make any changes at this time.
Okay so essentially, in a particular .cpp file, let's say foo.cpp, there is a unique line somewhere in the middle that reads:
FT_BAR, 1,
where 1 could be any number (but is going to be 1,2,3,4,5... practically never anything higher)...
I'd like to have a Bash or Perl script that allows me to automatically find this number and increase it by one. For what purpose you may ask... well, it would save me precious seconds multiple times per day and spare me a great deal of tedium endured by opening and closing this file to increase this number.
What is the best approach to this problem? I'm sure I will be embarrassed by a ridiculously simple one-line solution or some standard Unix tool that does exactly this, but I've been unable to find this so please forgive me if this is the case.
How about
perl -pe's/(\d+)/$1+1/e if /FT_BAR, \d+,/' foo.cpp > new.cpp
Perl has the e regex flag for evaluate.
use strict;
use warnings;
while ( <DATA> ) {
s/FT_BAR, (\d+),/"FT_BAR, ".($1+1).","/eg;
print;
}
__DATA__
FT_BAR, 1,
You can turn this into a one liner:
perl -pi.bak -e 's/FT_BAR, (\d+),/"FT_BAR, ".($1+1).","/e;' test_re.txt
Related
Currently I am working on a fairly large project. In order to increase the quality of our code, we decided to enforce the treatement of return values (Error Codes) for every function. GCC supports a warning concerning the return value of a function, however the function definition has to be preceeded by the following flag.
static __attribute__((warn_unused_result)) ErrorCode test() { /* code goes here */ }
I want to implement a bashscript that parses the entire source code and issues a warning in case the
__attribute__((warn_unused_result))
is missing.
Note that all functions that require this kind of modification return a type called ErrorCode.
Do you think this is possible via a bash script ?
Maybe you can use sed with regular expressions. The following worked for me on a couple of test files I tried:
sed -r "s/ErrorCode\s+\w+\s*(.*)\s*\{/__attribute__((warn_unused_result)) \0/g" test.cpp
If you're not familiar with regex, the pattern basically translates into:
ErrorCode, some whitespace, some alphanumerics (function name), maybe some whitespace, open parenthesis, anything (arguments), close parenthesis, maybe some whitespace, open curly brace.
If this pattern is found, it is prefixed by __attribute__((warn_unused_result)). Note that this only works if you are putting the open curly brace always in the same line as the arguments and you don't have line breaks in your function declarations.
An easy way I could imagine is via ctags. You create a tag file over all your source code, and then parse the tags file. However, I'm not quite sure about the format of the tags file. The variant I'm using here (Exuberant Ctags 5.8) seems to put an "f" in the fourth column, if the tag represents a function. So in this case I would use awk to filter all tags that represent functions, and then grep to throw away all lines without __attribute__((warn_unused_result)).
So, in a nutshell, first you do
$ ctags **/*.c
This creates a file called "tags" in the current directory. The command might also be ctags-exuberant, depending on your variant. The **/*.c is a glob pattern that might work in your shell - if it doesn't, you have to supply your source files in another way (look at the ctagsoptions).
Then you filter the funktions:
$ cat tags | awk -F '\t' '$4 == "f" {print $0}' | grep -v "__attribute__((warn_unused_result))"
No, it is not possible in the general case. The C++ grammar is the most complex of all the languages I know of, and C++ is not parsable via regular expressions in the general case. You might succeed if you limit yourself to a very narrow set of uses, but I am not sure how feasible it is in your case.
I also do not think the excersise is worth the effort, since sometimes ignoring the result of the function is an OK thing.
First off, thank you for everyone's help in advance! I've been learning Unix in school and have been doing well up until this most recent homework assignment.
I'm trying to figure out what the best way to approach this particular part in my homework assignment.
I have a headers file which I must separate into two separate files. There are two parts to this part of the assignment:
First, the first two lines of the file go into one file. I did this by doing:
head -2 headers > file1
However, the next request is to take two column headers (--Regular-- and --Overtime--) and put them into another file...which is what I'm having trouble with.
The header file looks like this:
Merry Land Training Academy
Pay Report
Week of June 12, 1999
--Regular--- --Overtime-- Gross Net
Employee Hours Rate Hours Rate Pay Pay
I know that grep can only match lines that contain the pattern, however how can I remove characters after the last two -- in Overtime?
For example, my grep will return the following:
egrep 'Regular' headers
--Regular--- --Overtime-- Gross Net
I know I could manually do a sed replace of "Gross" and "Net" after doing a grep to remove the words, however I know this is inefficient.
This command will be part of a script which will contain many other processes (which I have been able to do thus far).
In my research online, I know a lot of people recommend using awk, however we have not yet learned this in the course.
Again, thank you in advance. I really look forward to learning from everyone's experience.
Why do you think using sed would be inefficient? Certainly piping grep to sed would be a mistake, but sed is pretty good. You haven't really defined the problem very well, but assuming that you can distinguish a header by the existence of the string --, you could simply do:
sed -n -e '/--/s/[^-]*$//p' input > output
This will take all lines that contain -- and output everything up to the final -. If you only want to print the first such line:
sed -n -e '/--/{s/[^-]*$//p;q;}' input > output
I am writing a long document and I am frequently formatting some terms to italics. After some time I realized that maybe that is now what I want so I would like to remove all the latex commands that format text to italics.
Example:
\textit{Vim} is undoubtedly one of the best editors ever made. \textit{LaTeX} is an extremely powerful, intelligent typesetter. \textbd{Vim-LaTeX} aims at bringing together the best of both these worlds
How can I run a substitution command that recognizes all the instances of \textit{whatever} and changes them to just whatever without affecting different commands such as \textbd{Vim-LaTeX} in this example?
EDIT: As technically the answer that helps is the one from Igor I will mark that one as the correct one. Nevertheless, Konrad's answer should be taken into account as it shows the proper Latex strategy to follow.
You shouldn’t use formatting commands at all in your text.
LaTeX is built around the idea of semantic markup. So instead of saying “this text should be italic” you should mark up the text using its function. For instance:
\product{Vim} is undoubtedly one of the best editors ever made. \product{LaTeX}
is an extremely powerful, intelligent typesetter. \product{Vim-LaTeX} aims at
bringing together the best of both these worlds
… and then, in your preamble, a package, or a document class, you (re-)define a macro \product to set the formatting you want. That way, you can adapt the macro whenever you deem necessary without having to change the code.
Or, if you want to remove the formatting completely, just make the macro display its bare argument:
\newcommand*\product[1]{#1}
Use this substitution command:
% s/\\textit{\([^}]*\)}/\1/
If textit can span muptiple lines:
%! perl -e 'local $/; $_=<>; s/\\textit{([^}]*)}/$1/g; print;'
And you can do this without perl also:
%s/\\textit{\(\_.\{-}\)}/\1/g
Here:
\_. -- any symbol including a newline character
\{-} -- make * non-greedy.
I've made a translator in perl for a messageboard migration, All I do is applying regexes and print the result. I write stdout to a file and here we go ! But the problem is that my program won't work after 18 MB written !
I've made a translate.pl ( https://gist.github.com/914450 )
and launch it with this line :
$ perl translate.pl mydump.sql > mydump-bbcode.sql
Really sorry for quality of code but I never use perl... I tried sed for same work but didn't manage to apply the regex I found in original script.
[EDIT]
I reworked the code and sanitized some regexes (see gist.github.com/914450) but I'm still stuck. When I splited the big dump in 15M files, I launched translate.pl 7(processes) by 7 to use all cores but the script stops at a variable size. a "tail" command doesn't show a complex message on any url when it stops...
Thanks Guys ! I let you know if I manage finally
yikes - start with the basics:
use strict;
use warnings;
..at the top of your script. It will complain about not properly declaring your lexicals, so go ahead and do that. I don't see anything obvious that would be truncating your file, but perhaps one or more of your regexes is pathological. Also, the undefs at the end are not needed.
For what you are doing, you might consider just using sed
You say the "script stops". It keeps running but produces no more output? Or actually stops running? If it stops running, what does:
perl translate.pl mydump.sql > mydump-bbcode.sql
echo $?
show? And if you add a print STDERR "done!\n"; after your loop, does that show up?
Perl can certainly handle files much larger than 18 MB. I know because I routinely run files of 5 GB through Perl.
I think that your problem is in while($html=<FILE>).
Whenever $html is set to an empty line the while will evaluate as False and exit the loop.
You need to use something like while( defined( $html = <FILE> ) )
Edit:
Hmm. I had always thought you need the defined but in my testing just now it didn't exit on blank lines or 0. Must be more of that special Perl magic that mostly works the way you intend -- except when it doesn't.
Indeed if you restructure the while loop enough you can fool Perl into working the way I always thought it worked. (And it might have, in Perl 4 or in earlier versions of Perl 5)
This will fail:
$x = <>;
chomp $x;
while( $x ) {
print $x;
$x = <>;
chomp $x;
}
There could be any number of things going on:
Try adding $| = 1; to the top of your script. This will make all output unbuffered.
One of your regexes is going crazy and is deleting strings when you're not expecting it.
You've run out of disk space.
There's nothing really wrong with your script (other than you're missing use strict; use warnings; and you're not using the three-argument form of open()) that would cause it to stop working after some magic number of bytes.
Hello guys and Thank you so much for your help and ideas !
After trying to cut and parallelize the jobs, I tried to cut my program in 3 programs, translate1.pl, translate2.pl and 3... the job is done, and it's fast by 8 active cores !
then my launcher.sh starts successively the 3 scripts for each splitted file. done with 2 loops and here we go :)
Regards, Yoann
This is one of my most dreaded C/C++ compiler errors:
file.cpp(3124) : fatal error C1004: unexpected end-of-file found
file.cpp includes almost a hundred header files, which in turn include other header files. It's over 3000 lines. The code should be modularized and structured, the source files smaller. We should refactor it. As a programmer there's always a wish list for improving things.
But right now, the code is a mess and the deadline is around the corner. Somewhere among all these lines—quite possibly in one of the included header files, not in the source file itself—there's apparently an unmatched brace, unmatched #ifdef or similar.
The problem is that when something is missing, the compiler can't really tell me where it is missing. It just knows that when it reached end of the file it wasn't in the right parser state.
Can you offer some tools or other hints / methodologies to help me find the cause for the error?
If the #includes are all in one place in the source file, you could try putting a stray closing brace in between the #includes. If you get an 'unmatched closing brace' error when you compile, you know it all balances up to that point. It's a slow method, but it might help you pinpoint the problem.
One approach: if you have Notepad++, open all of the files (no problem opening 100 files), search for { and } (Search -> Find -> Find in All Open Documents), note the difference in count (should be 1). Randomly close 10 files and see if the difference in count is still 1, if so continue, else the problem is in one of those files. Repeat.
Handy tip:
For each header file, auto-generate a source file which includes it, then optionally contains an empty main method, and does nothing else. Compile all of these files as test cases, although there's no point running them.
Provided that each header includes its own dependencies (which is a big "provided"), this should give you a better idea which header is causing the problem.
This tip is adapted from Google's published C++ style guide, which says that each component's source files should include the interface header for that component before any other header. This has the same effect, of ensuring that there is at least one source file which will fail to compile, and implicate that header, if there's anything wrong with it.
Of course it won't catch unmatched braces in macros, so if you use much in the way of macros, you may have to resort to running the preprocessor over your source file, and examining the result by hand and IDE.
Another handy tip, which is too late now:
Check in more often (on a private branch to keep unfinished code out of everyone else's way). That way, when things get nasty you can diff against the last thing that compiled, and focus on the changed lines.
Hints:
make small changes and recompile after each small change
for unmatched braces, use an editor/IDE that supports brace match hilighting
if all else failds, Ye Olde method of commenting out blocks of code using a binary chop approach works for me
Very late to the party, but on linux you can use fgrep -o
-o, --only-matching
Print only the matched (non-empty) parts of a matching line,
with each such part on a separate output line.
So if you fgrep -o { then you'll get a list of all the opening braces in your file.
You can then pipe that to wc -l and that will give you the number of opening braces in your file.
With a bit of bash arithmetic, you can do the same with closing braces, and then print the difference.
In the below example I'm using git status -s to get the short-format output of all modified files from my git repo, and then I'm iterating over them to find which files may have mismatched braces
for i in $(git status -s | awk '{print $2}'); do
open=$(fgrep -o { $i | wc -l); # count number of opening braces
close=$(fgrep -o } $i | wc -l); # count number of closing braces
diff=$((open-close)); # find difference
echo "$diff : $i"; # print difference for filename
done
I think using some editor with brace highlighting will help. There should also be some tools around that do automatic indention on code.
EDIT: Does this vim script help? It seems to do #ifdef highlighting.
Can you create a script that actually does all the including, and then write the whole stuff to a temporary file? Or try the compiler for helping you with that? Then you can use the bracket highlighting features of various editors to find the problem and you'll probably be able to identify the file. (An extra help can be to have your script add a comment around every included file).
This might not be relevant to your problem, but I was getting an "Unexpected #else" error when framing some header files in an #if/#else/#endif block.
I found that if I set the problem modules to not use pre-compiled headers, the problem went away. Something to do with the "#pragma hdrstop" should not be within an #if/#endif.
Have a look at this question (Highlighting unmatched brackets in vim)
Pre-compile your code first, this will create a big chunk with the include files stuffed into the same file. Then you can use those brace-matching scripts.
I recently ran into this situation while refactoring some code and I'm not a fan of any of the answers above. The problem is that they neglect to incorporate a pretty basic assumption:
Any given file (most likely) will have matching braces and if/endif macros.
While it is true one can have a C++ source file that opens a bracket or an if block and includes another module that closes it, I have never seen this in practice. Something like the following:
foo.cpp
namespace Blah{
#include "bar.h"
bar.h:
}; /// namespace Blah
So with the assumption that any given file / module contains matching sets of braces and preprocessor directives, this problem is now trivial to solve. We just need a script to read the files / count the brackets & directives that it finds and report mismatches.
I've implemented one in Ruby here, feel free to take and adapt to your needs:
https://gist.github.com/movitto/6c6d187f7a350c2d71480834892552ab
I just spent an hour with a problem like this. It was very hard to spot, but in the end, I had typed a double # on one line:
##ifdef FEATURE_X
...
That broke the world.
Easy one to search for though.
The simplest and fastest solution is to comment file from the end by consistent blocks. If the error still exists, then you not yet comment the open brace.