Problem summary
I am trying to install an open-source parallel finite-element code called TACS and available at this github repository. To comply with the indicated prerequisites, I followed the instructions at this github repository, which allowed me to install SuiteSparse and METIS on Windows with precompiled BLAS/LAPACK DLLs. For the MPI, I installed both the Intel MPI Library and Open MPI through Cygwin. The final step should be to compile running make, however this command is not directly available in Windows 10. As a consequence, I explored the options suggested in this question, unfortunately without success. I feel at a dead end, any help will be appreciated.
What I've tried
Please have a look below at my attempts. I am mainly a Windows user and I don't know much of compiling programs using Makefile. My current understanding is that the Makefile that I am trying to compile is written for Linux and whatever GNU compiler for Windows I use will not work because of the different syntax needed. Please correct me if I am wrong. What I can't understand is why I get errors also when I try to compile with Ubuntu Bash for Windows 10 (last attempt of the list below).
Visual Studio nmake
Running the Developer Command Prompt for VS 2019 as administrator, I typed nmake -f Makefile in TACS base directory and I got Makefile.in(28) : fatal error U1001: syntax error : illegal character '{' in macro Stop.
Chocolatey make
Running Windows Command Prompt as administrator with C:\ProgramData\chocolatey\bin at the top of PATH environment variable, I typed make in TACS base directory and I got
"" was unexpected at this time.
make: *** [Makefile:23: default] Error 255
Cygwin make
Running Windows Command Prompt as administrator with C:\cygwin64\bin at the top of PATH environment variables, I typed make in TACS base directory and I got three types of error:
error: expected ',' or '...' before numeric constant
error: cannot convert 'int*' to 'idx_t*' {aka 'long int*'}
error: no matching function for call to 'TACSSchurMat::getBCSRMat(BCSRMat**, NULL, NULL, NULL)'
I tried to change some variable names in the affected scripts (like N_ in place of _N) and I got rid of the first and the third type of error, but not of the second one.
GnuWin make
Running Windows Command Prompt as administrator with C:\Program Files (x86)\GnuWin32\bin at the top of PATH environment variables, I typed make in TACS base directory and I got
"" was unexpected at this time.
make: *** [Makefile:23: default] Error 255
MinGW mingw32-make
Running Windows Command Prompt as administrator with C:\MinGW\bin at the top of PATH environment variables, I typed mingw32-make in TACS base directory and I got
"" was unexpected at this time.
Makefile:23: recipe for target 'default' failed
mingw32-make: *** [default] Error 255
MSYS MinGW 64-bit make
Running Windows Command Prompt as administrator with C:\msys64\usr\bin at the top of PATH environment variables, I typed make in TACS base directory and I got
make[1]: mpicxx: No such file or directory
make[1]: *** [../TACS_Common.mk:28: C:/Users/qa21944/git/tacs/src/TACSAssembler.o] Error 127
make[1]: *** Waiting for unfinished jobs....
make[1]: mpicxx: No such file or directory
make[1]: *** [../TACS_Common.mk:28: C:/Users/qa21944/git/tacs/src/TACSCreator.o] Error 127
make[1]: Leaving directory '/c/Users/qa21944/git/tacs/src'
make: *** [Makefile:23: default] Error 1
This is quite difficult for me to understand, as the mpicxx file is C:\Program Files (x86)\Intel\oneAPI\mpi\latest\bin, which in turn is in the PATH environment variable. When I tried to add C:\cygwin64\bin to PATH (below C:\msys64\usr\bin) and to rerun make, I got
0 [main] opal_wrapper (14432) C:\cygwin64\bin\opal_wrapper.exe: *** fatal error - cygheap base mismatch detected - 0x180352408/0x180357408.
This problem is probably due to using incompatible versions of the cygwin DLL.
Search for cygwin1.dll using the Windows Start->Find/Search facility
and delete all but the most recent version. The most recent version *should*
reside in x:\cygwin\bin, where 'x' is the drive on which you have
installed the cygwin distribution. Rebooting is also suggested if you
are unable to find another cygwin DLL.
I tried to follow these instructions and then rebooted my computer, but nothing changed.
Ubuntu Bash for Windows 10 make
This attempt was inspired by this answer. I downloaded Ubuntu from Microsoft Store and installed make. From the Ubuntu Bash I typed make in TACS base directory and I got
make[1]: Entering directory '/mnt/c/Users/qa21944/git/tacs/src'
Makefile:26: *** target pattern contains no '%'. Stop.
make[1]: Leaving directory '/mnt/c/Users/qa21944/git/tacs/src'
make: *** [Makefile:23: default] Error 1
I do not understand why I should get this error. I also made sure that all lines started with a tab rather than white spaces, but nothing changed.
Codes
Below you can find the Makefile.in and Makefile that I am using.
Makefile.in
# Do not modify this file. Copy this file to Makefile.in and then modify it.
# In order to get TACS to compile, you'll need to fill in the
# following path information. Some of the items below are required
# only if you're going to use the python interface.
# the full path to the root TACS directory
TACS_DIR = C:/Users/qa21944/git/tacs
CXX = mpicxx
RM = rm -f
PYTHON = python
PYTHON_CONFIG = python-config
# Set up for parallel make
MAKE = make -j 8
# Set the ar flags
AR_FLAGS = rcs
# Flags for debugging and regular compilation versions
EXTRA_DEBUG_CC_FLAGS = -fPIC -g
EXTRA_CC_FLAGS = -fPIC -O3
# Use this if you have problems with mpich
# TACS_DEF = -DMPICH_IGNORE_CXX_SEEK
# Defines whether to use static or dynamic linking
# TACS_LD_CMD=${TACS_DIR}/lib/libtacs.a
TACS_LD_CMD=-L${TACS_DIR}/lib/ -Wl,-rpath,${TACS_DIR}/lib -ltacs
# For linux systems, use the following settings:
SO_EXT=so
SO_LINK_FLAGS=-fPIC -shared
# For MAC OS X, use the following settings:
# SO_EXT=so
# SO_LINK_FLAGS=-fPIC -dynamiclib
# This uses the default installation of LAPACK.
# Use an optimized version of LAPACK if available.
# You may also have to include -lblas as well.
LAPACK_LIBS = -LC:/SP_ROOT/lapack_windows/x64 -llapack -lpthread -lblas
# For MAC OSX use the accelerate framework
# LAPACK_LIBS=-framework accelerate
# METIS is handy for partitioning graphs, but can be problematic for
# compilation. If you compile METIS using a C++ compiler you must add
# -DTACS_CPLUSPLUS_METIS to the TACS_DEF arguments below. If you
# compile METIS using a C compiler, there should be no issues.
METIS_INCLUDE = -IC:/SP_ROOT/build/install/include
METIS_LIB = -LC:/SP_ROOT/build/install/lib -lmetis
# AMD is a set of routines for ordering matrices. It is not required by default.
AMD_INCLUDE = -IC:/SP_ROOT/build/install/include/suitesparse
AMD_LIBS = -LC:/SP_ROOT/build/install/lib -llibamd
Makefile
# ============================================
#
# Make file for TACS_DIR/
#
# ============================================
include Makefile.in
include TACS_Common.mk
TACS_SUBDIRS = src \
src/bpmat \
src/elements \
src/elements/dynamics \
src/elements/basis \
src/elements/shell \
src/constitutive \
src/functions \
src/io
TACS_OBJS := $(addsuffix /*.o, ${TACS_SUBDIRS})
default:
#if [ "${TACS_IS_COMPLEX}" = "true" ]; then \
echo "Building Complex TACS"; \
for subdir in $(TACS_SUBDIRS) ; do \
echo "making $# in $$subdir"; \
echo; (cd $$subdir && $(MAKE) TACS_DIR=${TACS_DIR} TACS_DEF="${TACS_DEF} -DTACS_USE_COMPLEX") || exit 1; \
done \
else \
echo "Building Real TACS"; \
for subdir in $(TACS_SUBDIRS) ; do \
echo "making $# in $$subdir"; \
echo; (cd $$subdir && $(MAKE) TACS_DIR=${TACS_DIR}) || exit 1; \
done \
fi
${CXX} ${SO_LINK_FLAGS} ${TACS_OBJS} ${TACS_EXTERN_LIBS} -o ${TACS_DIR}/lib/libtacs.${SO_EXT}
#if [ "${TACS_IS_COMPLEX}" = "true" ]; then \
echo "ctypedef complex TacsScalar" > tacs/TacsTypedefs.pxi; \
echo "TACS_NPY_SCALAR = np.NPY_CDOUBLE" > tacs/TacsDefs.pxi; \
echo "dtype = complex" >> tacs/TacsDefs.pxi; \
else \
echo "ctypedef double TacsScalar" > tacs/TacsTypedefs.pxi; \
echo "TACS_NPY_SCALAR = np.NPY_DOUBLE" > tacs/TacsDefs.pxi; \
echo "dtype = np.double" >> tacs/TacsDefs.pxi; \
fi
debug:
#if [ "${TACS_IS_COMPLEX}" = "true" ]; then \
echo "Building Complex TACS"; \
for subdir in $(TACS_SUBDIRS) ; do \
echo "making $# in $$subdir"; \
echo; (cd $$subdir && $(MAKE) debug TACS_DIR=${TACS_DIR} TACS_DEF="${TACS_DEF} -DTACS_USE_COMPLEX") || exit 1; \
done \
else \
echo "Building Real TACS"; \
for subdir in $(TACS_SUBDIRS) ; do \
echo "making $# in $$subdir"; \
echo; (cd $$subdir && $(MAKE) debug TACS_DIR=${TACS_DIR}) || exit 1; \
done \
fi
${CXX} ${SO_LINK_FLAGS} ${TACS_OBJS} ${TACS_EXTERN_LIBS} -o ${TACS_DIR}/lib/libtacs.${SO_EXT}
#if [ "${TACS_IS_COMPLEX}" = "true" ]; then \
echo "ctypedef complex TacsScalar" > tacs/TacsTypedefs.pxi; \
echo "TACS_NPY_SCALAR = np.NPY_CDOUBLE" > tacs/TacsDefs.pxi; \
echo "dtype = complex" >> tacs/TacsDefs.pxi; \
else \
echo "ctypedef double TacsScalar" > tacs/TacsTypedefs.pxi; \
echo "TACS_NPY_SCALAR = np.NPY_DOUBLE" > tacs/TacsDefs.pxi; \
echo "dtype = np.double" >> tacs/TacsDefs.pxi; \
fi
interface:
${PYTHON} setup.py build_ext --inplace
complex_interface:
${PYTHON} setup.py build_ext --inplace --define TACS_USE_COMPLEX
complex: TACS_IS_COMPLEX=true
complex: default
complex_debug: TACS_IS_COMPLEX=true
complex_debug: debug
clean:
${RM} lib/libtacs.a lib/libtacs.so
${RM} tacs/*.so tacs/*.cpp
#for subdir in $(TACS_SUBDIRS) ; do \
echo "making $# in $$subdir"; \
echo; \
(cd $$subdir && $(MAKE) $# TACS_DIR=${TACS_DIR}) || exit 1; \
done
Edit: I am adding a snippet of TACS_Common.mk as requested in the comments.
TACS_Common.mk
TACS_LIB = ${TACS_DIR}/lib/libtacs.a
TACS_INCLUDE = -I${TACS_DIR}/src \
-I${TACS_DIR}/src/bpmat \
-I${TACS_DIR}/src/elements \
-I${TACS_DIR}/src/elements/dynamics \
-I${TACS_DIR}/src/elements/basis \
-I${TACS_DIR}/src/elements/shell \
-I${TACS_DIR}/src/constitutive \
-I${TACS_DIR}/src/functions \
-I${TACS_DIR}/src/io
# Set the command line flags to use for compilation
TACS_OPT_CC_FLAGS = ${TACS_DEF} ${EXTRA_CC_FLAGS} ${METIS_INCLUDE} ${AMD_INCLUDE} ${TACS_INCLUDE}
TACS_DEBUG_CC_FLAGS = ${TACS_DEF} ${EXTRA_DEBUG_CC_FLAGS} ${METIS_INCLUDE} ${AMD_INCLUDE} ${TACS_INCLUDE}
# By default, use the optimized flags
TACS_CC_FLAGS = ${TACS_OPT_CC_FLAGS}
# Set the linking flags to use
TACS_EXTERN_LIBS = ${AMD_LIBS} ${METIS_LIB} ${LAPACK_LIBS}
TACS_LD_FLAGS = ${EXTRA_LD_FLAGS} ${TACS_LD_CMD} ${TACS_EXTERN_LIBS}
# This is the one rule that is used to compile all the
# source code in TACS
%.o: %.cpp
${CXX} ${TACS_CC_FLAGS} -c $< -o $*.o
#echo
#echo " --- Compiled $*.cpp successfully ---"
#echo
I can't answer but maybe I can orient you.
First nmake is not make. It will not work with any makefile not written specifically as an nmake makefile. And it's only available on Windows. So, best to just forget it exists.
Second, it's important to understand how make works: rules in makefiles are a combination of targets/prerequisites, and a recipe. The recipe is not in "makefile" syntax, it's a shell script (batch file). So make works in tandem with the shell, to run commands. Which shell? On POSIX systems like GNU/Linux and MacOS it's very simple: a POSIX shell; by default /bin/sh.
On Windows systems it's much less simple: there are a lot of options. It could be cmd.exe. It could be PowerShell. It could be a POSIX shell, that was installed by the user. Which one is chosen by default, depends on how your version of make was compiled. That's why you see different behaviors for different "ports" of make to Windows.
So, if you look at the makefiles you are trying to use you can see they are unquestionably written specifically for a POSIX system and expect a POSIX shell and a POSIX environment. Any attempt to use a version of make that invokes cmd.exe as its default shell will fail immediately with syntax errors ("" was unexpected at this time.).
OK, so you find a version of make that invokes a POSIX shell, and you don't get that error anymore.
But then you have to contend with another difference: directory separators. In Windows they use backslash. In POSIX systems, they use forward slash and backslash is an escape character (so it's not just passed through the shell untouched). If you are going to use paths in a POSIX shell, you need to make sure your paths use forward slashes else the shell will remove them as escape characters. Luckily, most Windows programs accept forward slashes as well as backslashes as directory separators (but not all: for example cmd.exe built-in tools do not).
Then you have to contend with the Windows abomination known as drive letters. This is highly problematic for make because to make, the : character is special in various places. So when make sees a line like C:/foo:C:/bar its parser will get confused, and you get errors. Some versions of make compiled for Windows enable a heuristic which tries to see if a path looks like a drive letter or not. Some just assume POSIX-style paths. They can also be a problem for the POSIX shell: many POSIX environments on Windows map drive letters to standard POSIX paths, so C:\foo is written as /c/foo or /mnt/c/foo or something else. If you are adding paths to your makefile you need to figure out what the right mapping, if any, is and use that.
That's not even to start discussing the other differences between POSIX and Windows... there are so many.
From what you've shown above, this project was not written with any sort of portability to Windows in mind. Given the complexity of this, that's not surprising: it takes a huge amount of work. So you have these options that I can see:
Port it yourself to be Windows-compatible
Try to get it working inside cygwin (cygwin is intended to be a POSIX-style environment that runs on Windows)
Try to get it working in WSL
Install a virtual machine using VMWare, VirtualBox, etc. running a Linux distribution and build and run it there
Unfortunately I don't know much about the pros and cons of these approaches so I can't advise you as to the best course.
The route I chose, long long ago, was to get rid of Windows entirely and just use GNU/Linux. But of course that won't be possible for everyone :).
You need a real answer: you can't compile using the Windows Command Prompt.
Get yourself MSYS2 -- which will also install MinGW-w64 -- and follow the setup instructions. Then launch the MSYS shell to get a unix-y terminal prompt (zsh, IIRC), and change directory to the head of the project folder.
To access the Windows filesystem the root directory will be either /c or /mnt/c (sorry, on my mobile ATM, I can improve this in a day or two). For example,
C:\Users\qa21944\git\tacs
becomes
/c/Users/qa21944/git/tacs
From there the GNU/Windows make command should work:
mingw32-make
There is also this post to use more modern *nix environments under Windows:
How to install and use "make" in Windows?
Thanks may be overkill for what you need, though.
When I finish traveling and can sit down to look at this properly I can improve this answer's details, but getting a Linux shell terminal is what you need.
Related
I am currently trying to install NIST's sclite, which is part of SCTK 2.4.0 (github or current version). I am attempting the install on Cygwin in bash. The installation is done using make.
What I've Done
I made a directory for the install and navigated to that directory
mkdir sctk2.4.0
cd sctk2.4.0
( You'll possibly need $ cd /path/to/dir/sctk2.4.0 .)
I cloned the project from github
git clone https://github.com/chinshr/sctk.git
navigated into the base folder
cd sctk
then I started following the instructions in the INSTALL file.
Running
make config
worked fine, but after typing
make all
I got the output that follows
(mkdir -p bin)
(cd src; make all)
make[1]: Entering directory '/cygdrive/c/David/programs/sctk2.4.0/sctk/src'
(cd asclite; make all)
make[2]: Entering directory '/cygdrive/c/David/programs/sctk2.4.0/sctk/src/asclite'
(cd core; make all)
make[3]: Entering directory '/cygdrive/c/David/programs/sctk2.4.0/sctk/src/asclite/core'
g++ -o asclite -g -Os alignment.o segment.o sgml_reportgenerator.o alignedsegmentiterator.o reportgenerator.o speechset.o segmentsgroup.o logger.o tokenalignment.o sgml_generic_reportgenerator.o recording.o statistics.o compressedlevenshteinmatrix.o segmentor.o id.o trntrn_segmentor.o linestyle_inputparser.o inputparser.o levenshteinmatrix.o levenshtein.o uemfilter.o speakermatch.o spkrautooverlap.o graphalignedsegment.o rawsys_reportgenerator.o graphalignedtoken.o timedobject.o stt_scorer.o aligner.o arraylevenshteinmatrix.o graph.o main.o trn_inputparser.o alignedspeech.o token.o alignedsegment.o graph_coordinate.o rttm_inputparser.o scorer.o properties.o ctmstmrttm_segmentor.o filter.o speech.o alignedspeechiterator.o stm_inputparser.o checker.o ctm_inputparser.o lzma/LzFind.o lzma/LzmaEnc.o lzma/Alloc.o lzma/LzmaLib.o lzma/LzmaDec.o -lm
alignment.o: file not recognized: File format not recognized
collect2: error: ld returned 1 exit status
make[3]: *** [makefile:62: asclite] Error 1
make[3]: Leaving directory '/cygdrive/c/David/programs/sctk2.4.0/sctk/src/asclite/core'
make[2]: *** [makefile:12: all] Error 2
make[2]: Leaving directory '/cygdrive/c/David/programs/sctk2.4.0/sctk/src/asclite'
make[2]: *** [makefile:12: all] Error 2
make[2]: Leaving directory '/cygdrive/c/David/programs/sctk2.4.0/sctk/src'
make: *** [makefile:20: all] Error 2
I've looked at this SO post, but I've determined that the alignment.o file is not corrupted. Just in case, I tried a few make clean and even re-cloned the project from github, but I still get the same error.
Can anyone help me complete this installation, or at least get to the next step?
System Details
$ uname -a
CYGWIN_NT-6.1 MyMachine 2.10.0(0.325/5/3) 2018-02-02 15:16 x86_64 Cygwin
$ bash --version
GNU bash, version 4.4.12(3)-release (x86_64-unknown-cygwin) ...
$ gcc --version
gcc (GCC) 6.4.0 ...
$ g++ --version
g++ (GCC) 6.4.0 ...
$ make --version
GNU Make 4.2.1
Built for x86_64-unknown-cygwin ...
$ systeminfo | sed -n 's/^OS\ *//p'
Name: Microsoft Windows 7 Enterprise
Version: 6.1.7601 Service Pack 1 Build 7601
Manufacturer: Microsoft Corporation
Configuration: Member Workstation
Build Type: Multiprocessor Free
Note
I'm asking about this problem and then giving an answer to my own question. (I like that StackOverflow is allowing us to do that.) Hopefully, this will make it easier for people to help me with the problems I ran into further in the installation.
The next problem I ran into is discussed here. You can see this next problem in the answer to this problem.
This is the 'EASIER' solution.
Here are the details on what I called "the kaldi solution". Right now, it's just a list of commands without details. As shown here, these commands will install a sclite-2.4.10 directory under the $HOME (~) directory:
$ cd
$ git clone https://github.com/kaldi-asr/kaldi.git
$ cd kaldi/tools
$ extras/check_dependencies.sh
$ make -j $(nproc --all)
$ cp -R sctk-2.4.10 ~/
$ cd
$ rm -rf kaldi
$ cd sctk-2.4.10/
$ cp $HOME/.bashrc "${HOME}/.bashrc.$(date +%Y%m%d-%H%M%S).bak"
$ echo -e "\n\n## Allow access to sclite, rfilter, etc" >> $HOME/.bashrc
$ echo 'export PATH='"$(pwd)/bin"':$PATH' >> $HOME/.bashrc
$ source ~/.bashrc
See this question/answer for details on how to use it on Windows.
(See my comment under the question for the kaldi solution.)
The solution to this problem was in the README, as solutions often are. Note: There was another problem which came up after this problem was solved. See the bottom of this answer for help with that.
Here is the command I used to get the pertinent info from the README.
cat README | tail -13
and here is the pertinent info
64 bits Compilation
With big alignments, sctk needs to be compiled in 64 bits.
By default, the C/C++ software are compiled in 32 bits with the options (-Os)
but can be compiled in 64 bits, -m64 is added to the CFLAGS variable in:
src/asclite/core/makefile
src/asclite/test/makefile
src/rfilter1/makefile
src/sclite/makefile
Example of CFLAGS:
For OSX 10.4+: -fast -m64 -arch x86_64 -arch ppc64
So, I went to the makefiles listed (except rfilter1, see below) and changed the code there, replacing each -Os with -m64. Do this ONLY for the makefiles that are listed. I'll give an example for one of the listed files, but note you will have to do it for the others.
cd sctk
vim src/asclite/core/makefile
When the file was open, I found the line:
CFLAGS = -g -Os
which I changed to
CFLAGS = -g -m64
(pressed "i" to get into INSERT mode, made the change, pressed "Esc", then pressed ":wq" (Write and Quit) followed by "Enter")
I made the changes in all the listed files EXCEPT src/rfilter1/makefile, because that file had no -Os in it. This ended up being important, as the install wouldn't work if I had changed this file at all.
After this was completed, I ran make clean, but I DID NOT run make config, because this would have undone the changes we just make. I went directly to:
make all
This gets us past where we were before.
This problem was taken care of, but there was another problem:
In file included from main.cpp:20:0:
recording.h:122:36: error: template argument 2 is invalid
map<string, Filter::Filter*> filters;
^
recording.h:122:36: error: template argument 4 is invalid
which I asked about here.
AN EVEN EASIER SOLUTION - Taking advantage of a new, edited version of sclite
This is in case someone finds the answer here useful. I know there are no votes here, but I just got a trophy telling me this is my first question to get 1000 views, so I'll update my answer to show the easiest way to get things done.
TL;DR
https://www.nist.gov/itl/iad/mig/tools
https://github.com/usnistgov/SCTK
% cd /the/dir/where/i/want/to/install
% git clone https://github.com/usnistgov/SCTK.git
% cd SCTK
From the git master README (quoted) with some comments I've put in.
% make config
% sed -i 's#[-]Os#-m64#g' src/asclite/core/makefile
% sed -i 's#[-]Os#-m64#g' src/asclite/test/makefile
% sed -i 's#[-]Os#-m64#g' src/sclite/makefile
% make clean
% ## Possible edit to the `rfilter1 makefile`, which are
% ## described at the end of the answer but were not necessary
% ## for me.
% make all
% make check
% make install
% make doc
I also add the executables' directory to my PATH and make the documentation available via the man command.
% pwd
/the/dir/where/i/want/to/install/SCTK
% # back up your `.bashrc`
% cp $HOME/.bashrc "${HOME}/.bashrc.$(date +%Y%m%d-%H%M%S).bak"
% # persistent path changes
% echo -e "\n\n## Allow access to sclite, rfilter, etc" >> $HOME/.bashrc
% # your machine might use something other than `export` for this. CHECK!
% echo 'export PATH='"$(pwd)/bin"':$PATH' >> $HOME/.bashrc
% # make changes availabel this session
% source ~/.bashrc
% # man stuff
% cd doc
% cp -r ./* /usr/man/man1
END OF THE TL;DR SECTION
Details
Since when I posted this in May 2018, there have finally been some updates made to the software:
I posted my question in May 2018, and the updates were made in Fall 2018. They partly fix the problems I ran into here, but some of the information in the README and some makefiles are useful to note here.
The 32- to 64- bit issue (changing -Os to -m64, as done above) was find-able via the README.
% cat -n README.md | grep -A 4 "64 bits Compilation"
61 **64 bits Compilation**:
62 With big alignments, sctk needs to be compiled in 64 bits. By default, the C/C++ software are compiled in 32 bits with the options (`-Os`) but can be compiled in 64 bits. To do so, `-m64` is added to the CFLAGS variable in `src/asclite/core/makefile`, `src/asclite/test/makefile`, `src/rfilter1/makefile` and `src/sclite/makefile`.
63
64 Example of `CFLAGS` for OSX 10.4+: `-fast -m64 -arch x86_64 -arch ppc64`
65
Here is line 62 with word wrap
With big alignments, sctk needs to be compiled in 64 bits. By default, the C/C++ software are compiled in 32 bits with the options (-Os) but can be compiled in 64 bits. To do so, -m64 is added to the CFLAGS variable in
src/asclite/core/makefile,
src/asclite/test/makefile,
src/rfilter1/makefile and
src/sclite/makefile.
Since there was no -Os in src/rfilter1/makefile, I didn't make any changes.
I was able to finish the installation with no problem (including no failed tests). Here is my system info.
$ uname -a
CYGWIN_NT-10.0 MyMachine 3.0.7(0.338/5/3) 2019-04-30 18:08 x86_64 Cygwin
$ bash --version | head -n 1
GNU bash, version 4.4.12(3)-release (x86_64-unknown-cygwin)
$ gcc --version | head -n 1
gcc (GCC) 7.4.0
$ g++ --version | head -n 1
g++ (GCC) 7.4.0
$ make --version | head -n 2
GNU Make 4.2.1
Built for x86_64-unknown-cygwin
$ systeminfo | sed -n 's/^OS\ *//p'
Name: Microsoft Windows 10 Enterprise
Version: 10.0.17134 N/A Build 17134
Manufacturer: Microsoft Corporation
Configuration: Member Workstation
Build Type: Multiprocessor Free
However, it seems that some other people trying to compile on Cygwin have had issues. Here is some more info from the README
% grep "Special Note to Cygwin users" README.md
*Special Note to Cygwin users:* it has been reported that compilation of `rfilter1` can fail in some case, please read the OPTIONS part of the `rfilter1/makefile` and adapt accordingly before retrying compilation.
Well, let's look at the makefile for rfilter1, and see what some of you might need to do.
% head -n 15 src/rfilter1/makefile | tail -7
########################### Options for compilation #########################
####### If you have an very new version of GCC, the strcmp* family of functions
####### is included in the distribution. Changing the value of OPTIONS to
####### be blank will diable the use of supplied versions of these functions.
####### In particular, this behavior has been noted on some versions of cygwin
OPTIONS=-DNEED_STRCMP=1 $(CPPFLAGS) $(CFLAGS) $(LDFLAGS)
So, if you have rfilter1 compilation problems, change the non-commented line to
OPTIONS= $(CPPFLAGS) $(CFLAGS) $(LDFLAGS)
I'm building a C++ project using GNU Make (version 3.80). The makefile is auto-generated from the tool I'm using (IBM Rational Rhapsody). An example of this makefile is at the end of this post.
This makefile has a mechanism that allows me to specify a directory for object files (the OBJ_DIR variable). If this is set, the variable CREATE_OBJ_DIR is set up with the command if not exist $(OBJ_DIR) mkdir $(OBJ_DIR). This is then called for each object file in the project.
Running this makefile without setting an object file directory works as expected; the code is compiled without issues. But running it with OBJ_DIR set to 'build' causes the following error:
C:\Users\XXX\AppData\Local\Temp\make52963.sh: C:\Users\XXX\AppData\Local\Temp\make52963.sh: line 2: syntax error: unexpected end of file
C:\Tools\XXX\x86-win32\bin\make.exe: *** [build/Example.o] Error 2
I'm certain the issue is within the rule for '/build/Example.o', when $(CREATE_OBJ_DIR) is called. If I manually edit the rule and replace $(CREATE_OBJ_DIR) with mkdir $(OBJ_DIR), the command is executed correctly. If I then replace it with if not exist build mkdir build directly, to eliminate any issues due to variable expansion, the same error appears.
Other things I have tried:
Run a cmd shell with the same environment variables set as when the makefile is called, and attempted to run the if not exist build mkdir build command. No issues with this.
Ensure that no trailing characters are present in the command run within the makefile. None appear to be present.
My only conclusion at this point is that something about if statements causes the makefile to fail, but I'm not sure what. Is there anything else I should try to track down the source of this problem? Am I missing something obvious.
Let me know if more details are required.
Note: I've edited this makefile pretty heavily, so it's just to give an idea of what I'm using, and probably won't execute. Some of the environment variables below are set up in a batch file prior to calling make, but I'm confident they're not part of the issue I'm seeing, as the makefile works correctly except in the situation described above.
CPU = XXX
TOOL = gnu
INCLUDE_QUALIFIER=-I
LIB_CMD=$(AR)
LINK_CMD=$(LD)
CPP_EXT=.cpp
H_EXT=.h
OBJ_EXT=.o
EXE_EXT=.out
LIB_EXT=.a
TARGET_NAME=Example
all : $(TARGET_NAME)$(EXE_EXT) Example.mak
TARGET_MAIN=Example
LIBS=
INCLUDE_PATH=
ADDITIONAL_OBJS=
OBJS= \
build/Example.o \
OBJ_DIR=build
ifeq ($(OBJ_DIR),)
CREATE_OBJ_DIR=
else
CREATE_OBJ_DIR= if not exist $(OBJ_DIR) mkdir $(OBJ_DIR)
endif
build/Example.o : src/Example.cpp
#echo Compiling src/Example.cpp
$(CREATE_OBJ_DIR)
#$(CXX) $(C++FLAGS) -o build/Example.o src/Example.cpp
You are thinking to complex. A far simpler solution here is to use:
mkdir -p $(OBJ_DIR)
This will also make it work if OBJ_DIR=my/little/obj/dir/deep/down/the/rabit/hole.
Look at the following Makefile:
OBJ_DIR=foo
CREATE_OBJ_DIR= if not exist $(OBJ_DIR) mkdir $(OBJ_DIR)
$(info CREATE_OBJ_DIR=$(CREATE_OBJ_DIR))
all:
$(CREATE_OBJ_DIR)
and it's output:
% make
CREATE_OBJ_DIR=if not exist foo mkdir foo
if not exist foo mkdir foo
/bin/sh: 1: Syntax error: end of file unexpected (expecting "then")
Makefile:8: recipe for target 'all' failed
make: *** [all] Error 2
Your "if" statement is simply not valid shell syntax. On the other hand if OBJ_DIR is empty then CREATE_OBJ_DIR is empty and that is valid.
From entire day I am trying to install OverSim [http://www.oversim.org/wiki/OverSimInstall]
The make file looks like this:
all: checkmakefiles
cd src && $(MAKE)
clean: checkmakefiles
cd src && $(MAKE) clean
cleanall: checkmakefiles
cd src && $(MAKE) MODE=release clean
cd src && $(MAKE) MODE=debug clean
rm -f src/Makefile
makefiles:
cd src && opp_makemake -f --deep --make-so -o inet -O out $$NSC_VERSION_DEF
checkmakefiles:
#if [ ! -f src/Makefile ]; then \
echo; \
echo '======================================================================='; \
echo 'src/Makefile does not exist. Please use "make makefiles" to generate it!'; \
echo '======================================================================='; \
echo; \
exit 1; \
fi
doxy:
doxygen doxy.cfg
tcptut:
cd doc/src/tcp && $(MAKE)
I am using Omnet5.1.1 as omnet4.2.2 is not supported on Ubuntu16.04, my gcc version is 5.4.1.
Every time I try to build this make all, it gives header file not found error where as in actual the header files are present inside the project directory.
In file included from applications/ethernet/EtherAppCli.cc:21:0:
applications/ethernet/EtherAppCli.h:21:22: fatal error: INETDefs.h: No such file or directory
The includes are done like this:
#include "INETDefs.h" //available at src/linklayer/contract/
#include "MACAddress.h" //available at src/base/
project structure:
How could I resolve this build error?
This is a basic difference between newer OMNeT++ versions 5.x and the older OMNeT++ versions 3.x and 4.x.
As far as I remember Oversim, it was released for build with OMNeT 3.x and 4.2 as well as the older INET releases.
These old versions used parameters like --deep to search for include files, that's why the included files are just named and not entered with a complete path.
The newer INET and OMNeT releases use hierarchical path settings for include files. The complete paths have to be given for the compiler to access the included file.
So for INET version 3.x and OMNeT++ version 5.x, an include looks like: #include "inet/common/INETDefs.h"
Oversim does not include the complete paths for included headers, that's why you have errors when using Oversim with newer OMNeT releases.
The first option is to either use an older OMNeT version. Either install an older GCC in parallel on your system or set-up a virtual machine with an older Ubuntu if you like.
The second (and more complex) option is to adopt all include paths or define all necessary paths via the -I option of the compiler/linker.
Frankly, I'd suggest to use the older OMNeT++ 4.2.2 version...
The #include directive searches first of all inside the same directory as the file containing the directive and then in a preconfigured list of standard system directories.
If you don't want to move the header files to the same directory as EtherAppCli.cc, you will have to add the paths to these header files to this preconfigured list, usually with the compiler option
-Ipath/to/dir
I'm not sure if this is what is intended in the app your'e compiling but this is more or less what you can do.
Check to see if you missed anything in the installation guide.
Use OMNet 4.6 instead. Keep using inet-20111118. You should be able to build inet normally. Then build Oversim
I am trying to build gcc 4.7.2 using a custom prefix $PREFIX
I have built and installed all the prerequisites into my prefix location, and then successfully configured, built and installed gcc.
The problem that I now have is that $PREFIX is not in the library search path, and therefore the shared libraries cannot be found.
$PREFIX/bin $ ./g++ ~/main.cpp
$PREFIX/libexec/gcc/x86_64-suse-linux/4.7.2/cc1plus: \
error while loading shared libraries: \
libcloog-isl.so.1: \
cannot open shared object file: No such file or directory
What works, but isn't ideal
If I export LD_LIBRARY_PATH=$PREFIX/lib then it works, but I'm looking for something which works without having to set environment variables.
If I use patchelf to set the RPATH on all the gcc binaries then it also works; however this involves searching out all elf binaries and iterating over them calling patchelf, I would rather have something more permanent.
What I think would be ideal for my purposes
So I'm hoping there is a way to have -Wl,-rpath,$PREFIX/lib passed to make during the build process.
Since I know the paths won't need to be changed this seems like the most robust solution, and can be also be used for when we build the next gcc version.
Is configuring the build process to hard code the RPATH possible?
What I have tried, but doesn't work
Setting LDFLAGS_FOR_TARGET prior to calling configure:
All of these fail:
export LDFLAGS_FOR_TARGET="-L$PREFIX/lib -R$PREFIX/lib"
export LDFLAGS_FOR_TARGET="-L$PREFIX/lib"
export LDFLAGS_FOR_TARGET="-L$PREFIX/lib -Wl,-rpath,$PREFIX/lib"
Setting LDFLAGS prior to calling configure:
export LDFLAGS="-L$PREFIX/lib -Wl,-rpath,$PREFIX/lib"
In any event I worry that these will override any of the LDFLAGS gcc would have had, so I'm not sure these are a viable option even if they could be made to work?
My configure line
For completeness here is the line I pass to configure:
./configure \
--prefix=$PREFIX \
--build=x86_64-suse-linux \
--with-pkgversion='SIG build 12/10/2012' \
--disable-multilib \
--enable-cloog-backend=isl \
--with-mpc=$PREFIX \
--with-mpfr=$PREFIX \
--with-gmp=$PREFIX \
--with-cloog=$PREFIX \
--with-ppl=$PREFIX \
--with-gxx-include-dir=$PREFIX/include/c++/4.7.2
I've found that copying the source directories for gmp, mpfr, mpc, isl, cloog, etc. into the top level gcc source directory (or using symbolic links with the same name) works everywhere. This is in fact the preferred way.
You need to copy (or link) to those source directory names without the version numbers for this to work.
The compilers do not need LD_LIBRARY_PATH (although running applications built with the compilers will need an LD_LIBRARY_PATH to the $PREFIX/lib64 or something like that - but that's different)
Start in a source directory where you'll keep all your sources.
In this source directory you have your gcc directory either by unpacking a tarball or svn...
I use subversion.
Also in this top level directory you have, say, the following source tarballs:
gmp-5.1.0.tar.bz2
mpfr-3.1.1.tar.bz2
mpc-1.0.1.tar.gz
isl-0.11.1.tar.bz2
cloog-0.18.0.tar.gz
I just download these and update to the latest tarballs periodically.
In script form:
# Either:
svn checkout svn://gcc.gnu.org/svn/gcc/trunk gcc_work
# Or:
bunzip -c gcc-4.8.0.tar.bz2 | tar -xvf -
mv gcc-4.8.0 gcc_work
# Uncompress sources.. (This will produce version numbered directories).
bunzip -c gmp-5.1.0.tar.bz2 | tar -xvf -
bunzip -c mpfr-3.1.1.tar.bz2 | tar -xvf -
gunzip -c mpc-1.0.1.tar.gz | tar -xvf -
bunzip -c isl-0.11.1.tar.bz2 | tar -xvf -
gunzip -c cloog-0.18.0.tar.gz | tar -xvf -
# Link outside source directories into the top level gcc directory.
cd gcc_work
ln -s ../gmp-5.1.0 gmp
ln -s ../mpfr-3.1.1 mpfr
ln -s ../mpc-1.0.1 mpc
ln -s ../isl-0.11.1 isl
ln -s ../cloog-0.18.0 cloog
# Get out of the gcc working directory and create a build directory. I call mine obj_work.
# I configure the gcc binary and other outputs to be bin_work in the top level directory. Your choice. But I have this:
# home/ed/projects
# home/ed/projects/gcc_work
# home/ed/projects/obj_work
# home/ed/projects/bin_work
# home/ed/projects/gmp-5.1.0
# home/ed/projects/mpfr-3.1.1
# home/ed/projects/mpc-1.0.1
# home/ed/projects/isl-0.11.1
# home/ed/projects/cloog-0.18.0
mkdir obj_work
cd obj_work
../gcc_work/configure --prefix=../bin_work <other options>
# Your <other options> shouldn't need to involve anything about gmp, mpfr, mpc, isl, cloog.
# The gcc build system will find the directories you linked,
# then configure and compile the needed libraries with the necessary flags and such.
# Good luck.
I've been using this configure option with gcc-4.8.0, on FreeBSD, after building and installing gmp, isl and cloog:
LD_LIBRARY_PATH=/path/to/isl/lib ./configure (lots of other options) \
--with-stage1-ldflags="-rpath /path/to/isl/lib -rpath /path/to/cloog/lib -rpath /path/to/gmp/lib"
and the resulting gcc binary does not need any LD_LIBRARY_PATH. The LD_LIBRARY_PATH for configure is needed because it compiles a test program to check for the ISL version, which would fail if it didn't find the ISL shared lib.
I tried it on Linux (Ubuntu) where it failed during configuring because the -rpath args were passed to gcc instead of ld. I could fix this by using
--with-stage1-ldflags="-Wl,-rpath,/path/to/isl/lib,-rpath,/path/to/cloog/lib,-rpath,/path/to/gmp/lib"
instead.
Just using configure --with-stage1-ldflags="-Wl,-rpath,/path/to/lib" was not enough for me to build gcc 4.9.2, bootstrap failed in stage 2. What works is to pass he flags directly to make via
make BOOT_LDFLAGS="-Wl,-rpath,/path/to/lib"
I got this from https://gcc.gnu.org/ml/gcc/2008-09/msg00214.html
While it still involves setting environment variables, what I do is that I define LD_RUN_PATH, which sets the rpath. That way the rest of the system can keep using the system provided libraries instead of using the ones that your gcc build generates.
I am going to make a suggestion that I believe solves your problem, although it definitely does not answer your question. Let's see how many downvotes I get.
Writing a generic wrapper script to set LD_LIBRARY_PATH and then to run the executable is easy; see https://stackoverflow.com/a/7101577/768469.
The idea is to pass something like --prefix=$PREFIX/install to configure, building an install tree that looks like this:
$PREFIX/
install/
lib/
libcloogXX.so
libgmpYY.so
...
bin/
gcc
emacs
...
bin/
.wrapper
gcc -> .wrapper
emacs -> .wrapper
.wrapper is a simple shell script:
#!/bin/sh
here="${0%/*}" # or use $(dirname "$0")
base="${0##*/}" # or use $(basename "$0")
libdir="$here"/../install/lib
if [ "$LD_LIBRARY_PATH"x = x ] ; then
LD_LIBRARY_PATH="$libdir"
else
LD_LIBRARY_PATH="$libdir":"$LD_LIBRARY_PATH"
fi
export LD_LIBRARY_PATH
exec "$here"/../install/bin/"$base" "$#"
This will forward all arguments correctly, handle spaces in arguments or directory names, and so forth. For practical purposes, it is indistinguishable from setting the rpath like you want.
Also, you can use this approach not only for gcc, but for your entire my-personal-$PREFIX tree. I do this all the time in environments where I want an up-to-date suite of GNU tools, but I do not have (or want to admit to have) root access.
Try to add your $PREFIX to /etc/ld.so.conf and then run ldconfig:
# echo $PREFIX >> /etc/ld.so.conf
# ldconfig
This will recreate cache that is used by runtime linker and it will pick up your libraries.
WARNING: This operation will cause ALL applications to use your newly compiled libraries in $PREFIX instead of default location
While compiling Linux kernel modules that depend on each other, linker gives undefined symbol warnings like
Building modules, stage 2.
MODPOST
*** Warning: "function_name1" [module_name] undefined!
*** Warning: "function_name2" [module_name] undefined!
*** Warning: "function_name3" [module_name] undefined!
The unresolved symbols are resolved as soon as module is inserted into kernel using insmod or modprobe. Is there any way to get rid of the linker warning, though?
I have read through 3 Google SERP's on this issue - it seems nobody knows the answer. Are these linker warnings supposed to be this way when you build a kernel module?
Use KBUILD_EXTRA_SYMBOLS as below:
KBUILD_EXTRA_SYMBOLS='your module path'/Module.symvers
Finally, I got it. Thanks to shodanex for putting me on the right track.
Update: Be very careful when applying this fix to builds for older versions of kernel, as there is a bug in Makefile.modpost file in older versions of the kernel which makes your build misbehave and build wrong targets when you specify KBUILD_EXTMOD option.
You have to specify the paths to the source of the modules you depend on in KBUILD_EXTMOD make parameter.
Say, you have a module foo that depends on symbols from module bar.
Source files for foo are in foo/module/ and source files for bar are in bar/module/
The make command in Makefile for foo probably looks like
make ARCH=$$ARCH CROSS_COMPILE=$$CROSS_COMPILE -C $$LINUX_DIR \
M=`pwd`/module \
modules
(the exact line may differ in your project).
Change it to
make ARCH=$$ARCH CROSS_COMPILE=$$CROSS_COMPILE -C $$LINUX_DIR \
M=`pwd`/module \
KBUILD_EXTMOD=`pwd`/../bar/module \
modules
(we added the KBUILD_EXTMOD=pwd/../bar/module \ line, where pwd/../bar/module is a path to sources of kernel module we depend on.
One would expect KBUILD_EXTRA_SYMBOLS parameter to work this way, however it's KBUILD_EXTMOD.
No they are not. Wheter you build your code in-tree or out of tree, this message should not be displayed.I think you should fix your Makefile. Here is an example makefile. Not perfect, but used to work (until 2.6.26, did not try it since) :
ifneq ($(KERNELRELEASE),)
# We were called by kbuild
obj-m += mymodule.o
mymodule-objs := mymodule_usb.o a.o b.o c.o
else # We were called from command line
KDIR := /lib/modules/$(shell uname -r)/build
PWD := $(shell pwd)
default:
#echo ' Building FOO drivers for 2.6 kernel.'
#echo ' PLEASE IGNORE THE "Overriding SUBDIRS" WARNING'
$(MAKE) -C $(KDIR) SUBDIRS=$(PWD) modules
install:
./do_install.sh *.ko
endif # End kbuild check
clean:
rm -f -r *.o *.ko .*cmd .tmp* core *.i
For further documentation, you can check the kernel tree, the kbuild process is documented
Related to the above technique of using KBUILD_EXTMOD, and the question of which kernel versions it works under:
andycjw indicated it didn't work for him in 2.6.12
It didn't work for me in 2.6.15 (broke my module build)
Looking through the kernel commits, I see a number of changes to Makefile.modpost that seem related in 2.6.26 and 2.6.28, so I expect one of those is the limit.
My need to be tailored to your tree.
In our source we created a SYMBOLSDIR that is a path to all the modules
SYMBOLSDIR = 'some path'
make (same as above example) $(KERNELDIR) MODVERDIR=$(SYMBOLSDIR) modules