Expanding path variable in makefile using SED on Windows - regex

On Windows machine, a makefile is taking path option and creating another file by appending this path value.
My problem is that path variable is not expanding correct in resultant file.
For example
$ make var=c:\test\kernel
by using below makefile code this $(var) value is being appending to output file
all:
#sed -i '1 i\export PATH := $(var)' output.txt
Expected result
export PATH := c:\test\kernel
But instead I'm getting
export PATH := c: estkernel
So, how I can fix this problem in makefile?

First, I strongly urge you to always use forward slashes in paths even on Windows, especially when working with make. There are very few programs on Windows that won't work with forward-slashes (mainly old-school CMD commands etc.) and using backslashes in tools which have their provenance in UNIX will always be an uncomfortable fit.
For your situation you can do something like this:
all:
#sed -i '1 i\export PATH := $(subst \,\\,$(var))' output.txt
to convert your backslashes to escaped backslashes.

Related

Why can't I debug C++ file which contains spacebars in path (vs code)? [duplicate]

I have a directory containing several files, some of which have spaces in their names:
Test workspace/
Another directory/
file1.ext
file2.ext
demo 2012-03-23.odp
I use GNU's $(wildcard) command on this directory, and then iterate over the result using $(foreach), printing everything out. Here's the code:
FOO := $(wildcard *)
$(info FOO = $(FOO))
$(foreach PLACE,$(FOO),$(info PLACE = $(PLACE)))
Here's what I would expect to see printed out:
Test workspace
Another directory
file1.ext
file2.ext
demo 2012-03-23.odp
Here's what I would actually get:
Test
workspace
Another
directory
file1.ext
file2.ext
demo
2012-03-23.odp
The latter is obviously of no use to me. The documentation for $(wildcard) flat-out states that it returns a "space-separated list of names" but completely fails to acknowledge the huge problems this raises. Nor does the documentation for $(foreach).
Is it possible to work around this? If so, how? Renaming every file and directory to remove the spaces is not an option.
The bug #712 suggests that make does not handle names with spaces. Nowhere, never.
I found a blog post saying it's partially implemented by escaping the spaces with \ (\\ seems to be typo or formatting artefact), but:
It does not work in any functions except $(wildcard).
It does not work when expanding lists of names from variables, which includes the special variables $?, $^ and $+ as well as any user-defined variable. Which in turn means that while $(wildcard) will match correct files, you won't be able to interpret the result anyway.
So with explicit or very simple pattern rules you can get it to work, but beyond that you are out of luck. You'll have to look for some other build system that does support spaces. I am not sure whether jam/bjam does, scons, waf, ant, nant and msbuild all should work.
GNU Make does very poorly with space-separated filenames.
Spaces are used as delimiters in word list all over the place.
This blog post summarizes the situation well, but WARNING: it incorrectly uses \\ rather than \
target: some\ file some\ other\ file
some\ file some\ other\ file:
echo done
You can also use variables, so this would also work
VAR := some\ file some\ other\ file
target: $(VAR)
$(VAR):
echo done
Only the wildcard function recognizes the escaping, so you can't do anything fancy without lots of pain.
But don't forget that your shell uses spaces as delimiters too.
If I wanted to change the echo done to touch $#, I'd have to add slash to escape it for my shell.
VAR := some\ file
target: $(VAR)
$(VAR):
touch $(subst \,\\,$#)
or, more likely, use quotes
VAR := some\ file some\ other\ file
target: $(VAR)
$(VAR):
touch '$#'
In the end, if you want to avoid a lot of pain, both in GNU make, and in your shell, don't put spaces in your filenames. If you do, hopefully the limited capabilities of Make will be sufficient.
This method will also allow use of listed file names such as $? and user variables that are lists of files.
The best way to deal with spaces in Make is to substitute spaces for other characters.
s+ = $(subst \ ,+,$1)
+s = $(subst +,\ ,$1)
$(call s+,foo bar): $(call s+,bar baz) $(call s+,bar\ baz2)
# Will also shows list of dependencies with spaces.
#echo Making $(call +s,$#) from $(call +s,$?)
$(call s+,bar\ baz):
#echo Making $(call +s,$#)
$(call s+,bar\ baz2):
#echo Making $(call +s,$#)
Outputs
Making bar baz
Making bar baz2
Making foo bar from bar baz bar baz2
You can then safely manipulate lists of file names using all the GNU Make
functions. Just be sure to remove the +'s before using these names in a rule.
SRCS := a\ b.c c\ d.c e\ f.c
SRCS := $(call s+,$(SRCS))
# Can manipulate list with substituted spaces
OBJS := $(SRCS:.c=.o)
# Rule that has object files as dependencies.
exampleRule:$(call +s,$(OBJS))
# You can now use the list of OBJS (spaces are converted back).
#echo Object files: $(call +s,$(OBJS))
a\ b.o:
# a b.o rule commands go here...
#echo in rule: a b.o
c\ d.o:
e\ f.o:
Outputs
in rule: a b.o
Object files: a b.o c d.o e f.o
This info is all from the blog that everyone else was posting.
Most people seem to be recommending using no spaces in paths or using Windows 8.3 paths, but if you must use spaces, escaping spaces and substitution works.
If you are willing to rely on your shell a bit more, this gives a list which can hold names with spaces just fine:
$(shell find | sed 's: :\\ :g')
The original question said that "renaming is not an option", yet many commenters have pointed out that renaming is pretty much the only way Make can handle spaces. I suggest a middle way: Use Make to temporarily rename the files and then rename them back. This gives you all the power of Make with implicit rules and other goodness, but doesn't mess up your file naming scheme.
# Make cannot handle spaces in filenames, so temporarily rename them
nospaces:
rename -v 's/ /%20/g' *\ *
# After Make is done, rename files back to having spaces
yesspaces:
rename -v 's/%20/ /g' *%20*
You could call these targets by hand with make nospaces and make yesspaces, or you can have other targets depends on them. For example, you might want to have a "push" target which makes sure to put the spaces back in filenames before syncing files back with a server:
# Put spaces back in filenames before uploading
push: yesspaces
git push
[Sidenote: I tried the answer which suggested using +s and s+ but it made my Makefile harder to read and debug. I gave up on it when it gave me guff over implicit rules likes: %.wav : %.ogg ; oggdec "$<".]

sed not working with variables

I am trying to compile a large project that I downloaded from Internet and I have a custom script that runs configure, make and make install. The problem is that I want to pass some compatibility flags but the project uses libtool which doesn't recognize LDFLAGS and I have to find another way to pass my compatibility flags. So in my script after running configure I have code that looks like this:
COMPATFLAG="-gcc-name=/usr/local/gcc-4.2.1/bin/gcc"
sed -i "s/CC -shared/CC -shared ${COMPATFLAG}/g" libtool
${COMPATFLAG} may get different values depending on what platform is being compiled for so I cannot put it in clear text in the sed statement. The problem is that it doesn't work - I get an empty space instead of the value of ${COMPATFLAG}. If I instead write sed -i "s/CC -shared/CC -shared TEST/g" libtool it adds the string "TEST" to the file. If I use single quotes I get the ${COMPATFLAG} string and not the variable's value. If I try to export ${COMPATFLAG} it still doesn't work. I don't know why it goes wrong. Any idea?
Since COMPATFLAG does have forward slashes you should use a different regex delimiter in sed:
sed -i "s#CC -shared#& ${COMPATFLAG}#g" libtool
Also I have used # here in order to avoid repetition of matched string in replacement pattern.
Thanks to #JonathanLeffler for this tip, you can even use a control character as regex delimiter:
sed -i "s^GCC -shared^G& ${COMPATFLAG}^Gg" libtool
Type ^G as ctrl-V-G in shell
You can use pipe | instead of / with a double quote. That works for the global variables that include /.
sed -e "s|CC -shared|CC -shared ${COMPATFLAG}|g" libtool

path setting for c++ include headers for vim

My vim has path settings as shown below.
path=.,/usr/include,,
I think this is a default setting of 'path' I guess.
Because of this, g f opens c header files under the cursor.
But on C++ file C++ header files are not opened because the C++ header file location is not added to path variable of vim.
set path+=/usr/include/c++/4.6
I think that this setting on vimrc would be a solution.
But the problem is the actual directory location for C++ header file would be changed in every different linux distributions and g++ compiler versions.
How can I set path for c++ header files in a portable manner?
let g:gcpp_headers_path = system("g++ --version | grep g++ | awk '{print \"/usr/include/c++/\"$NF}'")
execute 'set path+=' . g:gcpp_headers_path
Now I am using this above:
This works with g++ environment. Not tested with other compilers.
If there's a limited number of locations, a simple conditional in ~/.vimrc will do:
if isdirectory('/usr/include/c++/4.6')
set path+=/usr/include/c++/4.6
elseif isdirectory(...
If you have a lot of different systems, and don't want to maintain all variations in a central place, you can move the system-dependent settings to a separate, local-only file, and invoke that from your ~/.vimrc, like this:
" Source system-specific .vimrc first.
if filereadable(expand('~/local/.vimrc'))
source ~/local/.vimrc
endif
I recently had the same problem, so here is my solution for documentation purposes:
1) I added the following to my .bashrc:
# add additional search paths to vim.
VIM_EXTPATHS="$HOME/.vim.extpaths"
if [ ! -e "$VIM_EXTPATHS" ] || [ "/usr/bin/cpp" -nt "$VIM_EXTPATHS" ]; then
echo | cpp -v 2>&1 | \
awk '/^#include </ { state=1 } /End of search list/ { state=0 } /^ / && state { print "set path+=" substr($0, 2) "/**2" }' > $VIM_EXTPATHS
fi
2) I added the following to my .vimrc:
" add extra paths.
let s:extpaths=expand("$HOME/.vim.extpaths")
if filereadable(s:extpaths)
execute "source ".s:extpaths
endif
On my system, the contents of the .vim.extpaths file are as follows:
set path+=/usr/lib/gcc/x86_64-linux-gnu/8/include/**2
set path+=/usr/local/include/**2
set path+=/usr/lib/gcc/x86_64-linux-gnu/8/include-fixed/**2
set path+=/usr/include/x86_64-linux-gnu/**2
set path+=/usr/include/**2
The **2 means that ViM will search two directories deep inside these directories. Now gf will find all the C++ headers I need. If you increase the depth, searches will take a lot more time, so don't set this number too high.
#note: for #include <chrono>, ViM will go to /usr/include/boost/chrono, which, funny enough, is a directory. I wonder why go file will open a directory, maybe this should be reported as a bug. To get to the correct chrono header you have to type 2gf.
The following Vimscript code, intended for a .vimrc file, updates path to include the search paths used by the preprocessor.
if executable('gcc')
let s:expr = 'gcc -Wp,-v -x c++ - -fsyntax-only 2>&1 | grep "^ " | sed "s/^ //"'
let s:lines = systemlist(s:expr)
for s:line in s:lines
execute 'set path+=' . fnameescape(s:line)
endfor
endif
I have similar code in my .vimrc, but with additional special-case handling.
There are specific environment variables for the compiler to examine. If you are using gcc/g++ in a linux/Unix environment, then the variables are C_INCLUDE_PATH and CPLUS_INCLUDE_PATH. If you are using bash/sh then use export VARIABLE=value or if you are using csh/tcsh then use setenv VARIABLE value or if you are using some other shell then you will need to look that up. In these examples VARIABLE is either C_INCLUDE_PATH and CPLUS_INCLUDE_PATH. I hope this helps.

Automatically fix filename cases in C++ codebase?

I am porting a C++ codebase which was developed on a Windows platform to Linux/GCC. It seems that the author didn't care for the case of filenames, so he used
#include "somefile.h"
instead of
#include "SomeFile.h"
to include the file which is actually called "SomeFile.h". I was wondering if there is any tool out there to automatically fix these includes? The files are all in one directory, so it would be easy for the tool to find the correct names.
EDIT: Before doing anything note that I'm assuming you either have copies of the files off ot the side or preferably that you have a baseline version in source control should you need to roll back for any reason.
You should be able to do this with sed: Something like sed -i 's/somefile\.h/SomeFile.H/I' *.[Ch]
This means take a case-insensitive somefile (trailing /I) and do an in-place (same file) replacement (-i) with the other text, SomeFile.H.
You can even do it in a loop (totally untested):
for file in *.[Ch]
do
sed -i "s/$file/$file/I" *.[Ch]
done
I should note that although I don't believe this applies to you, Solaris sed doesn't support -i and you'd have to install GNU sed or redirect to a file and rename.
Forgive my, I'm away from my linux environment right now so I can't test this myself, but I can tell you what utilities you would need to use to do it.
Open a terminal and use cd to navigate to the correct directory.
cd ~/project
Get a list of all of the .h files you need. You should be able to accomplish this with the shell's wildcard expansion without any effort.
ls include/*.h libs/include/*.h
Get a list of all of the files in the entire project (.c, .cpp, .h, .whatever), anything that can #include "header.h". Again, wildcard expansion.
ls include/*.h libs/include/*.h *.cpp libs/*.cpp
Iterate over each file in the project with a for loop
for f in ... # wildcard file list
do
echo "Looking in $f"
done
Iterate over each header file with a for loop
for h in ... # wildcard header list
do
echo "Looking for $h"
done
For each header in each project file, use sed to search for #include "headerfilename.h", and replace with #include "HeaderFileName.h" or whatever the correct case is.
Warning: Untested and probably dangerous: This stuff is a place to start and should be thoroughly tested before use.
h_escaped=$(echo $h | sed -e 's/\([[\/.*]\|\]\)/\\&/g') # escapes characters in file name
argument="(^\s*\#include\s*\")$h_escaped(\"\s*\$)" # I think this is right
sed -i -e "s/$argument/\$1$h\$2/gip"`
Yes, I know it looks awful.
Things to consider:
Rather than going straight to running this on your production codebase, test it thoroughly first.
sed can eat files like a VCR can eat tapes.
Make a backup.
Make another backup.
This is an O(N^2) operation involving hard disk access, and if your project is large it will run slowly. If your project is not gigantic, don't bother, but if it is, consider doing something to pipe sed's output to other seds.
Your search should be case insensitive: it should match #include, #INCLUDE, #iNcLuDe, and any combination of case present in the existing header filename, as well as any amount of whitespace between the include and the header. Bonus points if you preserve whitespace.
Use Notepad++ to do a 'Find in Files' and replace.
From toolbar:
Search - Find in Files.
Then complete the 'Find what' and 'Replace with'.

Setting and using path to data directory with GNU AutoTools

I am trying to use GNU AutoTools for my C++ project. I have written configure.ac, makefile.am etc. I have some files that are used by the program during execution e.g. template files, XML schema etc. So, I install/copy these files along the executable, for which I use something like:
abcdir = $(bindir)/../data/abc/
abc_DATA = ../data/knowledge/abc.cc
Now it copies the file correctly and My program installation structure looks somethings as follows:
<installation_dir>/bin/<executableFile>
<installation_dir>/data/abc/abc.cc
Now the problem is that in the source code I actually use these files (abc.cc etc.) and for that I need path of where these files resides to open them. One solution is to define (using AC_DEFINE) some variable e.g. _ABC_PATH_ that points to the path of installation but how to do that exactly?. OR is there any better way to do that. For example, in source code, I do something like:
...
ifstream input(<path-to-abc-folder> + "abc.cc"); // how to find <path-to-abc-folder>?
..
The AC_DEFINE solution is fine in principle, but requires shell-like variable expansion to take place. That is, _ABC_PATH_ would expand to "${bindir}/../data/abs", not /data/abc.
One way is to define the path via a -D flag, which is expanded by make:
myprogram_CPPFLAGS += -D_ABC_PATH='\"${abcdir}\"'
which works fine in principle, but you have to make include config.status in the dependencies of myprogram.
If you have a number of such substitution variables, you should roll out a paths.h file that is
generated by automake with a rule like:
paths.h : $(srcdir)/paths.h.in config.status
sed -e 's:#ABC_PATH#:${abcdir}:' $< > $#
As a side-note, you do know about ${prefix} and ${datarootdir} and friends, don't you? If not, better read them up; ${bindir}/.. is not necessarily equal to ${prefix} if the user did set ${exec_prefix}.