I've uploaded the project as a zip file so you can try it out.
https://dl.dropboxusercontent.com/u/35032740/ShareX/2015/11/Buggy.zip
I wanted to write a wrapper around the clipper library. The code compiles fine with cabal build, runs with cabal run but cabal repl produces this error:
Preprocessing executable 'Buggy' for Buggy-0.1.0.0...
GHCi, version 7.10.2: http://www.haskell.org/ghc/ :? for help
GHC runtime linker: fatal error: I found a duplicate definition for symbol
_ZNSt6vectorIN10ClipperLib8IntPointESaIS1_EE13_M_insert_auxEN9__gnu_cxx17__normal_iteratorIPS1_S3_EERKS1_
whilst processing object file
dist\build\Buggy\Buggy-tmp\wrapper.o
This could be caused by:
* Loading two different object files which export the same symbol
* Specifying the same object file twice on the GHCi command line
* An incorrect `package.conf' entry, causing some object to be
loaded twice.
ghc.exe: panic! (the 'impossible' happened)
(GHC version 7.10.2 for x86_64-unknown-mingw32):
loadObj "dist\\build\\Buggy\\Buggy-tmp\\wrapper.o": failed
Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug
For reference, here's the cabal file
-- Initial Buggy.cabal generated by cabal init. For further documentation,
-- see http://haskell.org/cabal/users-guide/
name: Buggy
version: 0.1.0.0
-- synopsis:
-- description:
-- license:
license-file: LICENSE
author: Luka Horvat
maintainer: lukahorvat9#gmail.com
-- copyright:
-- category:
build-type: Simple
-- extra-source-files:
cabal-version: >=1.10
executable Buggy
main-is: Main.hs
c-sources: clipper.cpp
, wrapper.cpp
-- other-modules:
-- other-extensions:
build-depends: base >=4.8 && <4.9
-- hs-source-dirs:
default-language: Haskell2010
extra-libraries: stdc++
Any ideas what the cause might be here?
I'm running Windows 10, 64bit.
I don't know the details of object file formats on Windows, so I'm guessing a bit.
Probably clipper.o and wrapper.o both define a weak symbol named _ZNSt6vectorIN10ClipperLib8IntPointESaIS1_EE13_M_insert_auxEN9__gnu_cxx17__normal_iteratorIPS1_S3_EERKS1_. (I see the same on Linux.) This probably came from a template instantiation (of vector). Weak symbols instruct the system linker to just pick any copy of the symbol if it encounters duplicates.
GHCi on Windows doesn't use the system linker, it has its own runtime linker that can load object files into itself while it runs. As a result it is generally not feature compatible with the system linker. Probably the runtime linker does not understand weak symbols, at least on Windows (https://ghc.haskell.org/trac/ghc/ticket/3333). From the error you got, we can assume that it treats them as regular symbols, and two regular symbols are not allowed to have the same name.
As a workaround, you may be able to build your C++ files with -fno-weak as described in https://stackoverflow.com/a/26454930/190376.
If that doesn't work, an alternative is to build your C++ files into a DLL, which you can have GHCi load using the system dynamic loader, avoiding this whole issue. On Linux this would look like
g++ wrapper.cpp clipper.cpp -shared -fPIC -o libclipper.so
ghci -L. -lclipper
though I imagine the details are different on Windows.
The specific error isn't what I'm used to seeing, but those backslashes say you're on Windows, and this otherwise looks like GHC bug #3242 which has been causing pain for years now. Good news: the cause was finally isolated two weeks ago. Bad news: the fix didn't make the deadline for 7.10.3, though at least the 8.0.1 milestone seems secure at this point.
Probably still worth posting your error text to that bug's thread; mine is only an educated guess, someone there will know for sure.
Related
I have an OCaml program that worked fine on Ubuntu 16 but when recompiled and run on Ubuntu 20 I get the following error:-
$ ocamldebug ./linearizer
OCaml Debugger version 4.08.1
(ocd) r
Loading program... done.
Time: 89534
Program end.
Uncaught exception: Sys_error "Illegal seek"
(ocd) b
Time: 89533 - pc: 624888 - module Netaccel_link
No source file for Netaccel_link.
I thought this was due to missing dev libraries but:-
$ sudo apt install libocamlnet-ocaml-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
libocamlnet-ocaml-dev is already the newest version (4.1.6-1build6).
0 upgraded, 0 newly installed, 0 to remove and 20 not upgraded.
What setup step am I missing on Ubuntu 20?
This looks like a regression bug in libocamlnet and you should report an issue there or, I am a bit pessimistic that you will get any response, you can try to debug the issue yourself.
The problem that you are facing has nothing to do with missing libraries (they will be reported during installation or, if the package is broken, end up in linker errors). It may result, however, from some misconfiguration of the system. If that is true, then you're lucky as you can fix it yourself.
I will give you some advice that might help you in debugging this issue. For more, please try using discuss.ocaml.org as a more suitable media (SO doesn't favor this kind of a discussion and we might get deleted by admins).
The illegal seek exception is thrown when the seek operation is applied on a non-regular file, aka ESPIPE Unix error. So check your inputs. It could be that what was previously regarded as a file in Ubuntu is now a pipe or a socket.
Try to use ltrace or strace to pinpoint the culprit e.g.,
ltrace ./linearizer
or, if it overwhelms you, try strace
strace ./linearizer
Instead of using ocamldebug you can use plain gdb. You can use gdb's interfaces to provide the path to the source code (though most likely it won't work since ocamlnet is not compiled with debug information). I believe that it will give you a more meaningful backtrace.
Instead of using the system installation try using opam. Install your dependencies with opam and try older versions as well as newer versions of the OCaml compiler. Also, try different versions of ocamlnet. Ideally, try to reproduce the environment that used to work for you.
When nothing else works, you can use objdump -d and look at the disassembly of your binary. OCaml is using a pretty readable and intuitive name mangling scheme (<module_name>__<function_name>_<uid>), so you can easily find the source code (search for <module_name>.ml file and look for the <function_name> there)
Finally, just use docker or any other container to run your application. Consider switching from ocamlnet to something more modern and supported.
On Linux and Mac, when using dlopen() to load a shared library that links to another library, if linking fails because of a missing symbol, you can get the name of the missing symbol with dlerror(). It says something like
dlopen failed: cannot locate symbol "foo"
On Windows, when using LoadLibrary() to load a DLL with a missing symbol, you can only get an error code from GetLastError() which for this type of issue will always be 127. How can I figure out which symbol is missing, or a more verbose error message from LoadLibrary() that explains why the function failed?
I figured out a way using the MSYS2 terminal. Other methods might work with GUI software.
A major caveat is that this can't be done in pure C/C++ and released for end users. It's for developers only, but it's better than nothing.
Install Debugging Tools for Windows by downloading the Windows SDK and unchecking everything except Debugging Tools.
I could be wrong, but it seems that installing this software installs a hook into the Windows kernel to allow LoadLibrary() to write verbose information to stderr.
Open the MSYS2 Mingw64 terminal as an administrator and run
'/c/Program Files (x86)/Windows Kits/10/Debuggers/x64/gflags.exe' -i main.exe +sls
This prints the following to the terminal to confirm that the registry has been changed.
Current Registry Settings for main.exe executable are: 00000002
sls - Show Loader Snaps
Use -sls instead of +sls if you need to undo, since I believe that the change takes place for all programs called main.exe in Windows globally, not just for your file.
Then running main.exe should print debug information to stderr, but since I'm debugging an -mwindows application, it's not working for me.
But for some reason, running the binary with MSYS2's gdb allows this debug information to be printed to stderr.
Install mingw-w64-x86_64-gdb with MSYS2 and run gdb ./main.exe and type run or r.
Search for a section similar to the following.
warning: 1ec8:43a0 # 764081125 - LdrpNameToOrdinal - WARNING: Procedure "foo" could not be located in DLL at base 0x000000006FC40000.
warning: 1ec8:43a0 # 764081125 - LdrpReportError - ERROR: Locating export "foo" for DLL "C:\whatever\plugin.dll" failed with status: 0xc0000139.
warning: 1ec8:43a0 # 764081125 - LdrpGenericExceptionFilter - ERROR: Function LdrpSnapModule raised exception 0xc0000139
Exception record: .exr 00000000050BE5F0
Context record: .cxr 00000000050BE100
warning: 1ec8:43a0 # 764081125 - LdrpProcessWork - ERROR: Unable to load DLL: "C:\whatever\plugin.dll", Parent Module: "(null)", Status: 0xc0000139
warning: 1ec8:43a0 # 764081171 - LdrpLoadDllInternal - RETURN: Status: 0xc0000139
warning: 1ec8:43a0 # 764081171 - LdrLoadDll - RETURN: Status: 0xc0000139
Great! It says Procedure "foo" could not be located in DLL so we have our missing symbol, just like in POSIX/UNIX's dlopen().
While the answer from Remy Lebeau is technically correct, determining the missing symbol from GetLastError() is still possible on a Windows platform. To understand what exactly is missing, understanding the terminology is critical.
Symbol:
When a DLL is compiled, it's functions are referenced by symbols.
These symbols directly relate to the functions name (the symbols are
represented by visible and readable strings), its return type, and
it's parameters. The symbols can actually be read directly through a
text editor although difficult to find in large DLLs.DLL Symbols - C++ Forum
To have a missing symbol implies that a function within cannot be found. If this error occurs prior to using GetProcAddress(), then it's possible that any number of functions cannot be loaded due to missing prerequisites. This means it is possible that a library that you are attempting to load also requires a library that the first cannot load. These levels of dependency may go on for an unknown number of layers, but the only answer that GetLastError() can determine is that there was a missing symbol. One such method is by using Dependency Walker to determine the missing library the first library requires. Once all required libraries are available and can be found by that library (which can be its own can of worms), that library can be loaded via LoadLibrary().
I am using a Window 32-bit machine to compile an R package developed using Rcpp and compiled with Rtools 3.4 in RStudio 1.0.28. I keep getting an error about the # signs within the 32-bit external dll (NYCgeo.dll):
thefile.o:thefile.cpp:(.text+0x913): undefined reference to `_imp__NYCgeo#8'
collect2.exe: error: ld returned 1 exit status
Sure enough, when I opened the 32-bit NYCgeo.dll in a text editor, I found #8 suffix. This is weird because when I developed the 64-bit version, the 64-bit NYCgeo.dll did not contain #8 suffix and I did not have any errors. Anyway, I read about the --kill-at command and was wondering where I would include it. I tried RStudio's Configure Build Tools settings as well as my makevars.win.in file but had no luck.
Response to #Dirk
Updated title as requested.
I am compiling the package from within RStudio using Rtools so I assumed it might have something to do with RStudio's Project Options.
I have spent the past week checking existing documentation. This post, this post, and this post describe the issue I am having. My issue is that I do not know where to specify either "--kill-at" or "--add-stdcall-alias"
The whole point of my package is to leverage NYC Dept of City Planning's geocoding software. I did not "just throw" the binary NYCgeo.dll "into the mix." In fact, my 64-bit version of the package works fine. My issue is with developing the 32-bit version... specifically, the presence of an #8 suffix in the NYCgeo.dll binary which is causing an error.
NYCgeo.dll is a C binary. I am not using Visual Studio.
The previous question you mentioned dealt with creating Makevars files for the 64-bit version of my package (thanks again, #Coatless for providing useful information). The 64-bit NYCgeo.dll binary did not contain an #8 suffix.
UPDATE:
I tried to create a better title for this question. The question pertains to creating an R package which utilizes functionality from another piece of software... in my case, geocoding software. Specifically, the issue I experienced is that the 32-bit version of the geocoding software has a decorated dll files while the 64-bit version does not. A decorated binary contains # symbols which trigger an error during compiling. My task was to devise a way to demangle (not sure if that is a real word) the 32-bit dll but leave the 64-bit dll alone.
Many thanks.
Gretchen
The rJava package was incredibly helpful in understanding how to deal with decorated binaries.
I created a def file named NYCgeo.def and saved it in my src directory:
LIBRARY NYCGEO.DLL
EXPORTS
NYCgeo#8
I then updated my Makevars.win.in file which is also in my src directory:
GBAT_PATH = #GBAT_PATH#
GBAT_DLL = #GBAT_DLL#
PKG_LIBS = -L"$(GBAT_PATH)/Bin" -l$(GBAT_DLL)
PKG_CPPFLAGS = -I"$(GBAT_PATH)/Include"
ifeq "${R_ARCH}" "/i386"
$(SHLIB): $(OBJECTS) NYCGEO.a
NYCGEO.a: NYCGEO.def
$(DLLTOOL) -k -d NYCGEO.def -l NYCGEO.a -D "$(GBAT_PATH)/Bin/$(GBAT_DLL)" $(DT_ARCH)
endif
I am now able to compile the package on both 32-bit and 64-bit machines running Windows.
I start saying that i am a newbie in programming and then i am not sure i will be able to explain well my problem.
I had some c++ code i wrote, this code are loaded and used by some R functions.
To compile the code i used the following:
R CMD SHLIB MyCode.cpp
and i loaded the library in R with
dyn.load("MyCOde.so")
Sometimes i built also an R package and i was able to load it into R.
If i do all these stuff on a Mac with mountain lion everything work fine, but now that i switched to mavericks, i have some problems. The R CMD SHLIB MyCode.cpp command works but when i used dyn.load("MyCOde.so") i get the following text:
Errore in dyn.load(paste(dir_function, "MyCOde.so", sep = "")) :
unable to load shared object 'MyCOde.so':
dlopen(MyCOde.so, 6): Symbol not found: __ZNSt8ios_base4InitC1Ev
Referenced from: MyCOde.so
Expected in: flat namespace
in MyCOde.so
Moreover if i try to load the package in R, i get the following
ld: warning: directory not found for option '-L/usr/local/lib/gcc/x86_64-apple-darwin13.0.0/4.8.2'
ld: library not found for -lquadmath
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [MyCode.so] Error 1
Can someone helps me?
Based on the helpful website of:
thecoatlessprofessor
Type this into your terminal shell:
curl -O http://r.research.att.com/libs/gfortran-4.8.2-darwin13.tar.bz2
sudo tar fvxz gfortran-4.8.2-darwin13.tar.bz2 -C /
This will create what you need to resume compiling as before.
Since it starts to work I can publish the answer for such a cases.
When you change the compiler and standard libraries - please note that different libraries have different implementation and different standard support. Changing the basement of your system might require total rebuild of your system with the new C++ standard library.
Your libraries are not the exception. So if have the errors in your linker like this:
warning: directory not found for option
'-L/usr/local/lib/gcc/x86_64-apple-darwin13.0.0/4.8.2'
apply next algorithm:
Check whether the directory /usr/local/lib/gcc/x86_64-apple-darwin13.0.0/4.8.2 still exists. I bet it is not.
Check if you still have the libstdc++ from the missed compiler? Usually if you upgrade the same compiler and the C++ standard library ABI does not change everything should continue to work. If the ABI changed or you switch standard C++ library and compiler - you face the massive system rebuild.
Recompile your library and apps with the new C++ standard library and compiler.
I have just downloaded clang 3.3 (homebrew) from the LLVM web page to my mac (OS X 10.8.4), but get this compiler error when using std=c++11 stdlib=libc++:
In file included from /usr/include/c++/v1/string:434:
In file included from /usr/include/c++/v1/algorithm:594:
In file included from /usr/include/c++/v1/memory:590:
In file included from /usr/include/c++/v1/typeinfo:61:
/usr/include/c++/v1/exception:146:5: error: an attribute list cannot appear here
_LIBCPP_NORETURN friend void rethrow_exception(exception_ptr);
^~~~~~~~~~~~~~~~
/usr/include/c++/v1/__config:190:28: note: expanded from macro '_LIBCPP_NORETURN'
# define _LIBCPP_NORETURN [[noreturn]]
^~~~~~~~~~~~
It seems that I also need another libc++ (even though it was said that it was 100% complete on MAC ...), but I cannot find any. Any help appreciated. Just for your info:
> clang++ -v
clang version 3.3 (tags/RELEASE_33/final)
Target: x86_64-apple-darwin12.4.0
Thread model: posix
And, yes, I googled it and found this: http://comments.gmane.org/gmane.comp.compilers.llvm.bugs/24138 claiming it's resolved in libc++ trunk ???
Okay, as suggested by Howard, I've downloaded tip-of-the-trunk libc++ into /opt/local/share/libcxx, but have trouble building it. The manual says to cd libcxx/lib, export TRIPLE=-apple-, and run ./buildit. I presume this implies bash (I'm usually a tcsh user, so I moved my .tcshrc, got a new shell and started bash). I did that and the compilations worked, but the library build failed. Apparently ./buildit doesn't see $TRIPLE=-apple-, as it picks the wrong LDSHARED_FLAG (not that on line 81, but that on line 103, which is to be used if $TRIPLE is not set), even though echo $TRIPLE yields -apple- as it should. When I add the statement echo TRIPLE = $TRIPLE at the top of buildit, it reports nothing. How come? What is wrong here?
The failure was that because the wrong LDSHARED_FLAG was picked the loading didn't work (ld complaint about the unknown option -soname which, I think, makes sense under linux). I don't know why buildit (a #! /bin/sh file) didn't pick up the TRIPLE environment variable (it did pick up several unwanted ones such as CXX and CC). I now simply added TRIPLE=-apple- at the top of that file and it did built the library. However, the loader spitted out several warnings all of which were of the form
ld: warning: direct access in ___cxa_bad_typeid to global weak symbol typeinfo for std::bad_typeid means the weak symbol cannot be overridden at runtime. This was likely caused by different translation units being compiled with different visibility settings.
But most importantly, it works (the compilation at least, I have yet to test the library). I have one final question. The advice was to use -I and -L to tell the compiler about the whereabouts of this version. Is it not possible to put it into the usual place /usr/include/c++/v1/? Note that Xcode has its version somewhere else anyway and I had put in a symbolic link (/usr/include/c++/v1/) to that one to get my homebrew clang 3.2 working (after the some Xcode update). What about the library? Can I also put it in a standard place?
Here is the home page of libc++:
http://libcxx.llvm.org
You can download the tip-of-trunk libc++ from there. You can tell clang to point to your download with -nostdinc++ -I<path-to-libc++>/include. You can also tell clang to link to your tip-of-trunk libc++ with -L<path-to-libc++>/lib and export DYLD_LIBRARY_PATH=<path-to-libcxx>/lib. The directions are all on the libc++ home page.
Xcode is the easiest way to get clang + libc++. But if you want the very latest, this is the place to go.
Congratulations!
Don't worry about the ld warning. It is a harmless ld bug that will be fixed in a future release. I see it on 10.8.4 too and it doesn't hurt anything.
The libc++ headers no longer live at /usr/include/c++/v1. Xcode has migrated them into itself. Having libc++ headers at /usr/include/c++/v1 from older installs has been a source of confusion and bugs. I regularly use -nostdinc++ -I to point to the libc++ headers I want (I often have several versions going at the same time), and that works well for me.
It is possible for you to replace your /usr/lib/libc++.1.dylib with that you have built. I do not recommend doing this. I have to sometimes to do a proper test, but I always do so very carefully because sometimes this causes me to have to reboot onto a backup disk and restore my /usr/lib to its original state. If you do go this route, it is a very good idea to have a backup of the original /usr/lib/libc++.1.dylib very handy.
I recommend instead -L on the command line, and export DYLD_LIBRARY_PATH=<path-to-libcxx>/lib in the shell. More than one person (including myself) has gotten their computer into a really nasty place by not following this advice.
If you run testit (under test/), all you need is DYLD_LIBRARY_PATH in that shell. The testit script is set up to point to the right places without an install.
Also I recommend figuring out why you had to modify buildit. No one else is seeing that behavior. printenv on your command line may help in this endeavor.
libc++ is updated often. We try to keep tip-of-trunk always in a shippable state.