How to control the cl.exe command that nvcc builds - python-2.7

I just installed CUDA 6.5, VS2013 Community Edition, and pyCUDA. I already had python 2.7.8 installed. I am new to CUDA and VS2013 development but not python. I verified my CUDA install by building some of the CUDA VS2013 sample solutions as both 32- and 64-bit so things work up to that point.
Now I'm trying to verify my pyCUDA install by running this test program.
### from: http://documen.tician.de/pycuda/tutorial.html
import pycuda.gpuarray as gpuarray
import pycuda.driver as cuda
import pycuda.autoinit
import numpy as np
# copy to gpu
a_gpu = gpuarray.to_gpu(np.random.randn(4,4).astype(np.float32))
# double it
a_doubled = (2 * a_gpu).get()
print('a_doubled', a_doubled)
When run, it produces this error:
[snip]
File "B:\Anaconda2\lib\site-packages\pycuda-2014.1-py2.7-win-amd64.egg\pycuda\compiler.py", line 250, in do_compile
return compile_plain(source, options, keep, nvcc, cache_dir)
File "B:\Anaconda2\lib\site-packages\pycuda-2014.1-py2.7-win-amd64.egg\pycuda\compiler.py", line 132, in compile_plain
stderr=stderr.decode("utf-8", "replace"))
CompileError: nvcc compilation of e:\temp\cb4\tmpadhjeh\kernel.cu failed
[command: nvcc -cubin -keep -cudart shared -arch sm_52 -m64 -Ib:\anaconda2\lib\site-packages\pycuda-2014.1-py2.7-win-amd64.egg\pycuda\cuda -keep kernel.cu]
[stdout:kernel.cu]
[stderr:
'B:\VisualStudioCom2013\VC\bin\amd64"\cl.exe #kernel.cpp1.ii.res > "kernel.cpp1.ii' is not recognized as an internal or external command,
operable program or batch file.]
Note the extraneous double quotes in the cl.exe command which are causing the error. Without them I can manually run B:\VisualStudioCom2013\VC\bin\amd64\cl.exe #kernel.cpp1.ii.res > kernel.cpp1.ii. It completes just fine and a valid kernel.cpp1.ii is generated.
Is there some way to control the cl.exe command that nvcc builds? Nothing in the nvcc manual jumped out at me, but with all those options I certainly might've have missed it.
Also posted in this Nvidia developers forum.

This is a bug in nvcc for CUDA 6.5. After following #talonmies suggestion to look through nvcc.profile, I started trying combinations of profile settings and command line options. I narrowed it down to this: when --keep is on the command line AND compiler-bindir is in the nvcc.profile, the malformed cl.exe compile command that includes double quotes is generated. The solution for keeping intermediate files is to put cl.exe in the path and remove compiler-bindir from nvcc.profile. Logged as a bug to NVIDIA.
UPDATE:
The problem does not manifest in CUDA 7.0.

Related

VS Code building .exe files in Windows 10, how do I change to build for windows 10 compatibility

So I am having trouble building my .exe files in visual studio code for my current windows OS. for some reason, my .exe files, when I run a diagnostic on them, seem to only be compatible for windows 8, not 10.
Using
Processor architecture: AMDx64
system: x64 based PC
VS code version: 1.74.3
When creating a simple "Hello World" application I tried following this tutorial, and it didn't have any problems. It was when I followed the tutorial for importing external libraries that the problems ocured.
I’ve tried importing an external library, and used msys2 to install the files in the bin/include/lib folders for mingw64.
I set my include path to the include folder, and I’ve set my compiler to default. My JSON tasks document appears correct, and when I build the .exe file, it builds successfully… but it only builds an executable file compatible for windows eight
I ran the properties compatibility test, and this is the output I get
What exactly do I need to do in order to change the OS version to make it compatible to run on both the visual studio code terminal, and my system terminal as well?
the following is the output once I ran the build process
Starting build...
C:\msys64\mingw64\bin\cpp.exe -IC:\msys64\mingw64\include -fdiagnostics-color=always -g "D:\Documents\C++\VS_Code\FMT Import\FMTImport.cpp" -o "D:\Documents\C++\VS_Code\FMT Import\FMTImport.exe" -lfmt
>Build finished successfully.
edit:
config name:C:/msys64/mingw64/bin/g++.exe
compiler path: C:/msys64/mingw64/bin/g++.exe
intellisense mode: ${default}
edit 2:
running terminal attempts to build..
for g++
D:\Documents\C++\VS_Code\FMT Import>g++ -o FMTImport FMTImport.cpp
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\William\AppData\Local\Temp\ccshGhdE.o:FMTImport.cpp:(.text+0x8c): undefined reference to `fmt::v9::vprint(fmt::v9::basic_string_view<char>, fmt::v9::basic_format_args<fmt::v9::basic_format_context<fmt::v9::appender, char> >)'
collect2.exe: error: ld returned 1 exit status
for clang++
D:\Documents\C++\VS_Code\FMT Import>clang++ FMTImport.cpp -o FMTImport
FMTImport.cpp:1:10: fatal error: 'fmt/format.h' file not found
#include <fmt/format.h>

Is the scylladb build hardcoded to work only with gnu gcc?

The python configure.py contains a line
gcc_linker_output = subprocess.check_output(['gcc', '-###', '/dev/null', '-o', 't'], stderr=subprocess.STDOUT).decode('utf-8')
The comments before this line indicate scylladb uses a custom dynamic linker and references details about the ABI layout.
Is there code missing from the configure.py script which would enable building on a strict llvm environment, or is that not possible at this time?
I am building Scylladb on FreeBSD 13 which uses clang++ 13.0.0.
I am on branch master, commit 0efdc45d5981868b1b6, Setp 8, 2022.
I patched SCYLLA-VERSION-GEN to get past the date --utf and USAGE issues, and patched config.py with an entry to read ID from freebsd for the boost error message.
I run configure.py with
./configure.py --mode=release --compiler=clang++ --cflags=-I/usr/local/include
In fact ScyllaDB builds with clang. However its dependency Seastar is very dependent on Linux. If you want it to run on FreeBSD you'll have to port Seastar first (see reactor_backend.{cc,hh})

Automatic instrumentation with Score-P / Vampirtrace not working with gcc/g++

I have a simple helloworld.cpp to instrument with Score-P or Vampirtrace.
Installation of the performance/ tracing tools works fine. After compiling and running:
# score-p
scorep-g++ helloworld.cpp -o hello
export SCOREP_ENABLE_TRACING=true
export SCOREP_ENABLE_PROFILING=true
# vampirtrace
vtcxx -DVTRACE helloworld.cpp -o hello
# run
./hello
The created OTF files (OTF for vampirtrace/ OTF2 for Score-P) are more or less empty (no timeline data). I'm using Vampir to visualize the data.
More details:
I'm testing on Mac OS X (g++-8) and Xubuntu (g++-7; VirtualBox).
For Mac OS X I have installed brew install gcc.
For the instrumented Score-P version I also got a warning
[Score-P] src/measurement/profiling/scorep_profile_callpath.c:206: Warning: Master thread contains no regions.
but I can't find related issues/ help.
I also installed TAU and PDT for Vampirtrace, but nothing changed. By the way manual instrumentation works for Vampirtrace:
#include "vt_user.h"
...
VT_TRACER("name");
For Vampirtrace I also tested OpenMP instrumentation and this was working, but only that (no application tracing around).
For both environments I did not install Open MPI.
It would be great, if somebody has similar issues and could help.
PS: Later, I want to instrument an application with Poco::Threads. I only read about partial support for POSIX Threads.
Update
The problem is g++. I tried the same instrumentation with Intel icc and it worked.
The missing instrumentation with g++ is also possible with icc, if you add the parameter --nocompiler like
score-p --nocompiler icc helloworld.cpp -o hello
Update
I had to install missing packages. There are logging outputs for ./configure with hints. One of the following package solved it:
apt-get install llvm libwrap0-dev libclang-dev gcc-7-plugin-dev

NPM sqlite3 with sqlcipher support

I am trying to cross compile NPM Sqlite3 with sqlcipher support. I am using Ubuntu 16.04 to cross compile for linux armv7 based SOC(system on chip).
So I started with cross-compiling OpenSSL to build sqlcipher for arm. I successfully cross compiled sqlcipher to produce a static library (libsqlcipher.a).
Now I am trying to get the NodeJS side of the project. I need sqlite with sqlcipher support, compiled for arm. I am using SOC SDK to built till now.
I am using node v4.6.1 and npm v2.15.9 to cross compile. I made sure I have the same version installed on Ubuntu as the SOC.
The command I use to cross compile is as follows :
npm install sqlite3 --target_arch=arm --enable-static=yes --build-from-source --sqlite_libname=sqlcipher -fPIC --sqlite=home/onkar/Library/sqlcipher-master/.libs --verbose
I exported the location of the libsqlcipher.a to LDFLAGS. I get the following error when I try to cross compile. Can someone help me with this error?
/home/linuximage/sdk/sysroots/x86_64-angstromsdk-linux/usr/libexec/arm-angstrom-linux-gnueabi/gcc/arm-angstrom-linux-gnueabi/5.2.1/real-ld: error: /home/Library/sqlcipher-master/.libs/libsqlcipher.a(sqlite3.o): requires unsupported dynamic reloc R_ARM_THM_MOVW_ABS_NC; recompile with -fPIC
collect2: error: ld returned 1 exit status
node_sqlite3.target.mk:129: recipe for target 'Release/obj.target/node_sqlite3.node' failed
make: *** [Release/obj.target/node_sqlite3.node] Error 1
Please let me know if you require any additional information, I would be more than happy to provide you with the same.
Thanks,
Onkar
In the first instance, you should check if the -fPIC (position independent code) flag was correctly applied when the libsqlcipher.a file was originally created.
In your output above, it looks like the linker is using the file at:
/home/Library/sqlcipher-master/.libs/libsqlcipher.a
Run the command
objdump -r /home/Library/sqlcipher-master/.libs/libsqlcipher.a | more
... and check for a line close to the start of the output beginning with the text
RELOCATION RECORDS FOR
If you see this line, then the library doesn't contain position independent code.

ld: file not found: /usr/lib/crt1.o

When trying to compile Fortran using PGI on Mac OS X Sierra, I get the error
ld: file not found: /usr/lib/crt1.o
I found a workaround for older Mac OS X versions (http://www.pgroup.com/userforum/viewtopic.php?t=4578)
sudo ln -s /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/usr/lib/crt1.o /usr/lib/crt1.o
However, with Sierra, System Integrity Protection prevents writing in /usr/bin. How can I solve this problem?
I tried linking into /usr/local/bin/ (which is permitted), but then, how can I make sure the compiler searches for library in that path?
Installing just the Command Line Tools for Mac OS X solved the problem. Do this in your terminal:
xcode-select --install
Installing Lazarus on MacOS X :
worked for me
http://wiki.lazarus.freepascal.org/Installing_Lazarus_on_MacOS_X#Xcode_5.0.2B_compatibility_.28Mac_OS_X_10.8_and_10.9.29
Solution for command line programs:
The correct answer for me was as explained in this link:
https://medium.com/#kviat/free-pascal-3-0-2-linking-on-macos-sierra-c40706e86fda
After some googling I realized that most libraries were removed from
/usr/lib in macOS Sierra. However this case is handled in FPC, so we
just need to set internal compiler variable MacOSXVersionMin to 10.8
(or later). There is no standard compiler option for it, but after
some search in source code I found the solution: set the environment
variable MACOSX_DEPLOYMENT_TARGET:
You should give the deployment target of MacOS:
MACOSX_DEPLOYMENT_TARGET= XX.XX #for instance 10.15
Solution for generally:
Linking the necessary file to /usr/bin/crt* . As already stated, this linking will be prohibited by MacOs beginning from 10.10. But there is still a way to accomplish this linking procedure and it solves the problem.
1) Reboot the Mac and hold down Command + R keys simultaneously after you hear the startup chime, this will boot Mac OS X into Recovery Mode
2) When the “MacOS Utilities” / “OS X Utilities” screen appears, pull down the ‘Utilities’ menu at the top of the screen instead, and choose “Terminal”
3) Type the following command into the terminal then hit return:
csrutil disable; reboot
4) When you come back, run the command sudo mount -uw /
5) Just run the linking code you want to:
sudo ln -s /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/usr/lib/crt1.o /usr/lib/crt1.o
sources: http://osxdaily.com/2015/10/05/disable-rootless-system-integrity-protection-mac-os-x/
https://www.reddit.com/r/MacOS/comments/caiue5/macos_catalina_readonly_file_system_with_sip/
In my case the problem was actually an error on the PGI installation side. PGI seems to be well aware that newer versions of macOS do not have the /usr/lib/crt1.o and that you can't create files there anymore. But it is possible to setup correct environment variables for the PGI compilers and then the linker should use the correct path to the crt1.o.
This configuration should be done automatically during the installation of the PGI compiler suite by running the makelocalrc command and should generate the file /opt/pgi/osx86-64/$PGIVER/bin/localrc. But in my case this step failed silently.
Reasons for failure seem to be:
license agreement for XCode not (yet) accepted, although this error should leave you with a /opt/pgi/osx86-64/$PGIVER/bin/localrc.error, containing some details
XCode version not supported, which seems to leave you with nothing. This is what I got when I ran the makelocalrc script manually:
makelocalrc -x /opt/pgi/osx86-64/19.10
Error: Unsupported XCode version 11
In my case (PGI 19.10, macOS 10.15, XCode 11.2.1) I manually patched the /opt/pgi/osx86-64/19.10/bin/makelocalrc to not error out on XCode 11:
if test $xcodever -gt 11 ; then # <-- was "-gt 10"!
echo " Error: Unsupported XCode version " $xcodever
exit -1
fi
and then re-ran the script after which compilation with PGI compilers (both pgcc and pgfortran) worked:
sudo /opt/pgi/osx86-64/2019/bin/makelocalrc -x /opt/pgi/osx86-64/19.10
Your case may vary, but you might want to check for a /opt/pgi/osx86-64/$PGIVER/bin/localrc.error or the /opt/pgi/osx86-64/$PGIVER/bin/localrc itself and try to manually (re-) generate it if it is not there or if you upgraded XCode/macOS since the installation of the PGI compilers.