When compiling following program with Xcode 10 GM:
#include <iostream>
#include <string>
#include <variant>
void hello(int) {
std::cout << "hello, int" << std::endl;
}
void hello(std::string const & msg) {
std::cout << "hello, " << msg << std::endl;
}
int main(int argc, const char * argv[]) {
// insert code here...
std::variant< int, std::string > var;
std::visit
(
[]( auto parameter )
{
hello( parameter );
},
var
);
return 0;
}
I get the following error:
main.cpp:27:5: Call to unavailable function 'visit': introduced in macOS 10.14
However, if I change min deployment target to macOS 10.14, the code compiles fine and it works, even though I am running macOS 10.13.
Since std::visit is function template, and should not depend on OS version (which I proved by running the code on lower version of mac than actually supported), should this be considered as bug and reported to Apple or is this expected behaviour?
The same happens when compiling for iOS (iOS 12 is minimally expected).
All std::variant functionality that might throw std::bad_variant_access is marked as available starting with macOS 10.14 (and corresponding iOS, tvOS and watchOS) in the standard header files. This is because the virtual std::bad_variant_access::what() method is not inline and thus defined in the libc++.dylib (provided by the OS).
There are several workarounds (all technically undefined behaviour), ordered by my personal preference:
1) Grab into the Implementation
std::visit only throws if one of the variant arguments is valueless_by_exception. Looking into the implementation gives you the clue to use the following workaround (assuming vs is a parameter pack of variants):
if (... && !vs.valueless_by_exception() ) {
std::__variant_detail::__visitation::__variant::__visit_value(visitor, vs...);
} else {
// error handling
}
Con: Might break with future libc++ versions. Ugly interface.
Pro: The compiler will probably yell at you when it breaks and the workaround can be easily adapted. You can write a wrapper against the ugly interface.
2) Suppress the Availability Compiler Error ...
Add _LIBCPP_DISABLE_AVAILABILITY to the project setting Preprocessor Macros ( GCC_PREPROCESSOR_DEFINITIONS)
Con: This will also suppress other availability guards (shared_mutex, bad_optional_access etc.).
2a) ... and just use it
It turns out that it already works in High Sierra, not only Mojave (I've tested down to 10.13.0).
In 10.12.6 and below you get the runtime error:
dyld: Symbol not found: __ZTISt18bad_variant_access
Referenced from: [...]/VariantAccess
Expected in: /usr/lib/libc++.1.dylib
in [...]/VariantAccess
Abort trap: 6
where the first line unmangles to _typeinfo for std::bad_variant_access. This means the dynamic linker (dyld) can't find the vtable pointing to the what() method mentioned in the introduction.
Con: Only works on certain OS versions, you only get to know at startup time if it does not work.
Pro: Maintains original interface.
2b) ... and provide your own exception implemention
Add the following lines one of your project source files:
// Strongly undefined behaviour (violates one definition rule)
const char* std::bad_variant_access::what() const noexcept {
return "bad_variant_access";
}
I've tested this for a standalone binary on 10.10.0, 10.12.6, 10.13.0, 10.14.1 and my example code works even when causing a std::bad_variant_access to be thrown, catching it by std::exception const& ex, and calling the virtual ex.what().
Con: My assumption is that this trick will break when using RTTI or exception handling across binary boundaries (e.g. different shared object libraries). But this is only an assumption and that's why I put this workaround last: I have no idea when it will break and what the symptoms will be.
Pro: Maintains original interface. Will probably work on all OS versions.
This happens because std::visit throws an bad_variant_access exception in cases described here and since the implementation of that exception depends on an newer version of libc++ you are required to use versions of iOS and macOS that ship this new version (macOS 10.14 and iOS 12).
Thankfuly, there is a implementation path available for when c++ exceptions are turned off which doesn't depend on the newer libc++ so if possible you can use that option.
P.S.
About the case where you increased the minimum deployment target to 10.14 and were still able to run the program normally on 10.13 I'm guessing you would run into problems at the point that this new exception would be triggered (since the exception method which relies on a newer version of libc++ would not be resolved).
Here's another alternative (that won't be palatable for some). If you're already using Boost, then you can use Boost.Variant2 when targeting iOS.
#if MACRO_TO_TEST_FOR_IOS_LT_11
#include <boost/variant2/variant.hpp>
namespace variant = boost::variant2;
#else
#include <variant>
namespace variant = std;
#endif
Then you can use variant::visit in your code.
I'm still working out the kinks to test for the iOS target version (and if we're targeting iOS at all). That's why I used MACRO_TO_TEST_FOR_IOS_LT_11 above, as a placeholder.
Similarly you can also use abseil-cpp libraries to seamlessly use std::variant where it is enabled, and abseil::variant where it isn't.
Abseil is an open-source collection of C++ code designed to augment the C++ standard library.
Add
CXXFLAGS += -D_LIBCPP_DISABLE_AVAILABILITY
to your Makefile.
See some of the other posts to see details on pros and cons of this, but this will get the code to compile and run.
Even though templates generally come from headers, doesn't mean the runtime target doesn't matter. Those templates are part of a wider library, and they compile to code that must still be compatible with the rest of that library. It makes sense for the whole standard library to be of one, single version, and it makes sense for that version to be the one that'll work on the target machine. Can you imagine the chaos that would ensue otherwise?
Some of the others here have given some low-level, practical reasons why in this particular case that version unity is important. Personally I think it's best to forget about implementation details like "templates go in headers" in situations like this; you shouldn't need to care about it, plus you risk making abstraction-breaking assumptions for little benefit. Just code to contract and you'll be fine.
Related
I have a library of code I'm working on upgrading from x86 to x64 for a Windows application.
Part of the code took advantage of MSVC inline assembly blocks. I'm not looking to go through and interpret the assembly but I am looking to keep functionality from this part of the application.
Can I compile the functions using the inline assembly using GCC to make a DLL and link that to the rest of the library?
EDIT 1:(7/7/21) The flexibility with which compiler the project uses is open and I am currently looking into using Clang for use with MSVC.(also the Intel C++ compiler as another possibility) As stated in the first sentence it is a Windows application that I want to keep on Windows and the purpose of using another compiler is due to me 1.) not wanting to rewrite the large amount of assembly and 2.) because I know that MSVC does not support x64 inline assembly. So far clang seems to be working with a couple issues of how it declares comments inside of the assembly block and a few commands. The function is built around doing mathematical operations on a block of data, in what was supposed to be as fast as possible when it was developed but now that it works as intended I'm not looking to upgrade just maintain functionality. So, any compiler that will support inline assembly is an option.
EDIT 2:(7/7/21) I forgot to mention in the first edit, I'm not necessarily looking to load the 32-bit DLL into another process because I'm worried about copying data into an out of shared memory. I've done a similar solution for another project but the data set is around 8 MB and I'm worried that slow copy times for the function would cause the time constraint on the math to cause issues in the runtime of the application.(slow, laggy, and buffering are effects I'm trying to avoid.) I'm not trying to make it any faster but it definitely can't get any slower.
In theory, if you manage to create a plain C interface for that DLL (all exported symbols from DLL are standard C functions) and don't use memory management functions across "border" (no mixed memory management) then you should be able to dynamically load that DLL from another another (MSVC) process and call its functions, at least.
Not sure about statically linking against it... probably not, because the compiler and linker must go hand in hand (MSVC compiler+MSVC linker or GCC compiler+GCC linker) . The output of GCC linker is probably not compatible with MSVC at least regarding name mangling.
Here is how I would structure it (without small details):
Header.h (separate header to be included in both DLL and EXE)
//... remember to use your preferred calling convention but be consistent about it
struc Interface{
void (*func0)();
void (*func1)(int);
//...
};
typedef Interface* (*GetInterface)();
DLL (gcc)
#include "Header.h"
//functions implementing specific functionality (not exported)
void f0)(){/*...*/}
void f1)(int){/*...*/}
//...
Interface* getInterface(){//this must be exported from DLL (compiler specific)
static Interface interface;
//initialize functions pointers from interface with corresponding functions
interface.func0 = &f0;
interface.func1 = &f1;
//...
return &interface;
}
EXE (MSVC)
#include "Header.h"
int main(){
auto dll = LoadLibrary("DLL.dll");
auto getDllInterface = (GetInstance)GetProcAddress(dll, "getInterface");
auto* dllInterface = getDllInterface();
dllInterface->func0();
dllInterface->func1(123);
//...
return 0;
}
I want to use Boost.Filesystem together with -fno-exceptions. According to the Boost.Filesystem documentation it states that it supports the BOOST_NO_EXCEPTIONS macro.
However, the following snippet:
#define BOOST_NO_EXCEPTIONS
#include <boost/filesystem.hpp>
int main() {}
compiled with:
g++ -fno-exceptions boost_test.cpp
gives the error:
/.../boost/filesystem/operations.hpp: In constructor
'boost::filesystem::filesystem_error::filesystem_error(const string&,
boost::system::error_code)':
/.../boost/filesystem/operations.hpp:84:16: error:
exception handling disabled, use -fexceptions to enable
catch (...) { m_imp_ptr.reset(); }
I compile using gcc 5 and boost version 1.57 on Mac OSX (also tested on similar ubuntu setups).
I am wondering whether my understanding of BOOST_NO_EXCEPTIONS is right in that it should cover the usage of -fno-exceptions or whether it's simply there for the boost::throw_exception part?
Well, "no" is the obvious answer here, g++ cannot deal with the filesystem_error class. There's a humdinger in boost/filesystem/config.hpp:
// throw an exception ----------------------------------------------------------------//
//
// Exceptions were originally thrown via boost::throw_exception().
// As throw_exception() became more complex, it caused user error reporting
// to be harder to interpret, since the exception reported became much more complex.
// The immediate fix was to throw directly, wrapped in a macro to make any later change
// easier.
#define BOOST_FILESYSTEM_THROW(EX) throw EX
This macro is used extensively in libs/filesystem/src/operations.cpp to throw exceptions. This is a show-stopper.
Fwiw, your sample program only appears to compile properly in clang and MSVC++, they only complain in their back-end about having to emit exception handling code, g++ does it in its front-end. No complaint from clang/msvc++ for this sample code since that exception handling code was already emitted previously, back when the boost libraries were built.
Which demonstrates another severe problem with your approach, you probably originally built boost without -fno-exceptions in effect. Not good.
This is what Boost.Filesystem the documentation says:
All exceptions thrown by the Filesystem Library are implemented by calling boost::throw_exception(). Thus exact behavior may differ depending on BOOST_NO_EXCEPTIONS at the time the filesystem source files are compiled.
In my understanding, it doesn't really say it support BOOST_NO_EXCEPTIONS
And when I did a egrep -r BOOST_NO_EXCEPTIONS under the filesystem directory, I found nothing
And after I read the source code, it supports my guessing. There are many places using try { ... } catch(...) in the code. You can also tell from the error message you got. Here is an example:
filesystem_error(
const std::string & what_arg, const path& path1_arg,
const path& path2_arg, system::error_code ec)
: system::system_error(ec, what_arg)
{
try
{
m_imp_ptr.reset(new m_imp);
m_imp_ptr->m_path1 = path1_arg;
m_imp_ptr->m_path2 = path2_arg;
}
catch (...) { m_imp_ptr.reset(); }
}
And if you read this, the semantic of BOOST_NO_EXCEPTIONS is actually not disabling exceptions, but:
forwarding all exceptions to a user-defined non-template version of
boost::throw_exception.
After experiencing crashes when introducing nested calls of std::async in my real program, I was able to reproduce the problem in the following minimum example. It crashes often, but not always. Do you see anything what goes wrong, or is it a compiler or standard library bug? Note that the problem remains if get() calls to the futures are added.
#include <future>
#include <vector>
int main (int, char *[])
{
std::vector<std::future<void>> v;
v.reserve(100);
for (int i = 0; i != 100; ++i)
{
v.emplace_back(std::async(std::launch::async, [] () {
std::async(std::launch::async, [] { });
}));
}
return 0;
}
I observe two different kinds of crashes: (in about every fifth run)
Termination with "This application has requested the Runtime to terminate it in an unusual way."
Termination after throwing an instance of 'std::future_error', what(): Promise already satisfied.
Environment:
Windows 7
gcc version 4.8.2 (i686-posix-dwarf-rev3, Built by
MinGW-W64 project), as provided by Qt 5.3.2
Command line call: g++ -std=c++11 -pthread futures.cpp
Compiled and run on two independent machines
Option -pthread?
Could it be that in my environment for some reason the option -pthread is silently not taken into account? I observe the same behavior with and without that option.
Since this answer is still "unanswered," after talking with some people from Lounge<C++>, I think I can say that it's pretty obvious from the comments that this is due to an implementation error either on MinGW/MinGW-w64's or pthread's part at the time. Using gcc 4.9.1, MinGW-W64, the problem does not appear anymore. In fact, the program above appears to compile and run correctly even on a version earlier than 4.8.2 with POSIX threading.
I myself am not an expert, my guess is that the exact trip-up happens when the program appears to try to write to the same promise twice, which, I think, should be a big no-no, as an std::async should write its result only once (again, I'm not sure if I'm right here, and other comments and edits will most likely clarify).
Also, this may be a related problem: std::future exception on gcc experimental implementation of C++0x
Here is some C++ code illustrating my problem with a minimal expample:
// uncomment the next line, to make it hang up:
//#define BOOST_DATE_TIME_POSIX_TIME_STD_CONFIG //needed for nanosecond support of boost
#include <boost/thread.hpp>
void foo()
{
while(true);
}
int main(int noParameters, char **parameterArray)
{
boost::thread MyThread(&foo);
if ( MyThread.timed_join( boost::posix_time::seconds(1) ) )
{
std::cout<<"\nDone!\n";
}
else
{
std::cerr<<"\nTimed out!\n";
}
}
As long as I don't turn on the nanosecond support everthing works as expected, but as soon as I uncomment the #define needed for the nanosecond support in boost::posix_time the program doesn't get past the if-statement any more, just as if I had called join() instead of timed_join().
Now I've already figured out, that this happens because BOOST_DATE_TIME_POSIX_TIME_STD_CONFIG changes the actual data representation of the timestamps from a single 64bit integer to 64+32 bit. A lot boost stuff is completely implemented inside the headers but the thread methods are not and because of that they cannot adapt to the new data format without compiling them again with the apropriate options. Since the code is meant to run on an external server, compiling my own version of boost is not an option and neither is turning off the nanosecond support.
Therefore my question is as follows: Is there a way to pass on a value (on the order of seconds) to timed_join() without using the incompatible 96bit posix_time methods and without modifying the standard boost packages?
I'm running on Ubuntu 12.04 with boost 1.46.1.
Unfortunately I don't think your problem can be cleanly solved as written. Since the library you're linking against was compiled without nanosecond support, by definition you violate the one-definition rule if you happen to enable nanosecond support for any piece that's already compiled into the library binary. In this case, you're enabling it across the function calls to timed_join.
The obvious solution is to decide which is less painful to give up: Building your own boost, or removing nanosecond times.
The less obvious "hack" that may or may not totally work is to write your own timed_join wrapper that takes a thread object and an int representing seconds or ms or whatever. Then this function is implemented in a source file with nothing else and that does not enable nanosecond times for the specific purpose of calling into the compiled boost binary. Again I want to stress that if at any point you fail to completely segregate such usages you'll violate the one definition rule and run into undefined behavior.
Our project uses a few boost 1.48 libraries on several platforms, including Windows, Mac, Android, and IOS.
We are able to consistently get the IOS version of the project to crash (nontrivially but reliably) when using IOS, and
from our investigation we see that ~thread_data_base is being called on the thread's thread_info while its thread is still running.
This seems to happen as a result of the smart pointer reaching a zero count, even though it is obviously still
in scope in the thread_proxy function which creates it and runs the requested function in the thread.
This seems to happen in various cases - the call stack is not identical between crashes, though there are a few
variations which are common.
Just to be clear - this often requires running code which is creating hundreds of threads, though there are
never more than about 30 running simultaneously. I have "been lucky" and got it very very early in the
run also, but that's rare.
I created a version of the destructor which actually catches the code red-handed:
in libs/thread/src/pthread/thread.cpp:
thread_data_base::~thread_data_base()
{
boost::detail::thread_data_base* const thread_info=detail::get_current_thread_data();
void *void_thread_info = (void *) thread_info;
void *void_this = (void *) this;
// is somebody destructing the thread_data other than its own thread?
// (remember that its own which should no longer point to it anyway,
// because of the call to detail::set_current_thread_data(0) in thread_proxy)
if (void_thread_info) { // == void_this) {
__builtin_trap();
}
}
I should note that (as seen from the commented-out code) I had previously checked to see that void_thread_info == void_this because I
was only checking for the case where the thread's current thread_info was killing itself.
I have also seen cases where the value returned by get_current_thread_data is non-zero and
different from "this", which is really weird.
Also when I first wrote that version of the code, I wrote:
if (((void*)thread_info) == ((void*)this))
and at run-time I got some very weird exception that said I something about a virtual function table
or something like that - I don't remember. I decided that it was trying to call "==" for this object type
and was unhappy with that, so I rewrote as above, putting the conversions to void * as separate
lines of code. That in itself is quite suspicious to me. I am not one to run to rush to blame compilers, but...
I should also note that when we did catch this happening the trap, we saw the destructor for
~shared_count appear twice consecutively on the stack in Xcode source. Very doubleweird.
We tried to look at the disassembly, but couldn't make much out of it.
Again - it looks like this is always a result of the shared_count which seems to be owned by
the shared_ptr which owns the thread_info reaching zero too early.
Update: it seems that it is possible to get into situations which reach the above trap without the situation doing any harm. Since fixing the issue (see answer) I have seen it happen, but always after thread_info->run() has finished executing. Don't yet understand how...but it's working.
Some additional info:
I should note that the boost.sh from Pete Goodliffe (and modified by others) that is commonly used to compile boost for IOS
has the following note in the header:
: ${EXTRA_CPPFLAGS:="-DBOOST_AC_USE_PTHREADS -DBOOST_SP_USE_PTHREADS"}
# The EXTRA_CPPFLAGS definition works around a thread race issue in
# shared_ptr. I encountered this historically and have not verified that
# the fix is no longer required. Without using the posix thread primitives
# an invalid compare-and-swap ARM instruction (non-thread-safe) was used for the
# shared_ptr use count causing nasty and subtle bugs.
#
# Should perhaps also consider/use instead: -BOOST_SP_USE_PTHREADS
I use those flags, but to no avail.
I found the following which is very tantalizing - it looks like they had the same issue in std::thread:
http://llvm.org/bugs/show_bug.cgi?format=multiple&id=12730
That was suggestive of using an alternate implementation inside boost for arm processors which seems also to directly address this issue:
spinlock_gcc_arm.hpp
The version included with boost 1.48 uses outdated arm assembly.
I took the updated version from boost 1.52, but I'm having trouble compiling it.
I get the following error:
predicated instructions must be in IT block
I found a reference to what looks to be a similar use of this instruction here:
https://zeromq.jira.com/browse/LIBZMQ-414
I was able to use the same idea to get the 1.52 code to compile by modifying
the code as follows (I inserted an appropriate IT instruction)
__asm__ __volatile__(
"ldrex %0, [%2]; \n"
"cmp %0, %1; \n"
"it ne; \n"
"strexne %0, %1, [%2]; \n"
BOOST_SP_ARM_BARRIER :
"=&r"( r ): // outputs
"r"( 1 ), "r"( &v_ ): // inputs
"memory", "cc" );
But in any case, there are ifdefs in this file which look for the arm architecture, which is not defined that way in my environment. After I simply edited the file so that only ARM 7 code
was left, the compiler complains about the definition of BOOST_SP_ARM_BARRIER:
In file included from ./boost/smart_ptr/detail/spinlock.hpp:35:
./boost/smart_ptr/detail/spinlock_gcc_arm.hpp:39:13: error: instruction requires a CPU feature not currently enabled
BOOST_SP_ARM_BARRIER :
^
./boost/smart_ptr/detail/spinlock_gcc_arm.hpp:13:32: note: expanded from macro 'BOOST_SP_ARM_BARRIER'
# define BOOST_SP_ARM_BARRIER "dmb"
Any ideas??
Figured this out. It turns out that the boost.sh script that I mention in the question chose the incorrect boost flag to address this problem - instead of BOOST_SP_USE_PTHREADS (and the other flag there with it, BOOST_AC_USE_PTHREADS) it turns out that what is needed on IOS is BOOST_SP_USE_SPINLOCK. This ends up giving pretty much the identical solution used in the std::thread issue referred to in the question.
If you are compiling for any modern IOS device which uses ARM 7, but using an older boost (we are using 1.48), you need to copy the file spinlock_gcc_arm.hpp from a more recent boost (like 1.52). That file is #ifdef'd for the different arm architectures, but it is not clear to me that the defines it is looking for are defined in the IOS compile environment using the script. So you can either edit the file (violent but effective) or invest some time to figure out how to make this tidy and correct.
In any case, you may need to insert the extra assembly instruction that I did above in the question:
"it ne; \n"
I have not yet gone back to see if I can delete that now that I have my compile environment working problem.
However, we're not done yet. The code used in boost for this option includes, as discussed, ARM assembly language instructions. The ARM chips support two instruction sets which can't be mixed in a given module (not sure of the scope, but evidently file by file is an acceptable granularity when compiling). The instructions used in boost for this locking include non-Thumb instructions, but IOS by default uses the Thumb instruction set. The boost code, aware of the instruction set issue, checks to see that you have arm enabled but not thumb, but by default in IOS, thumb is on.
Getting the compiler to generate non-thumb ARM code depends on which compiler you are using in IOS - Apple's LLVM or LLVM GCC. GCC is deprecated, and Apple's LLVM is the default when you use XCode.
For the default Clang + Apple LLVM 4.1, you need to compile using the -mno-thumb flag. Also any files in your IOS app which use any part of boost which uses smart pointers will also have to be compiled using -mno-thumb.
To compile boost like this, I think you can just add -mno-thumb to the EXTRA_CPP_FLAGS in the script. (I modified the user-config.jam directly while experimenting and haven't yet gone back to clean up.)
For your app, in Xcode you need to select your target, then go into the Build Phases tab, and there select Compile sources. There you have the option of adding compile flags, so for each relevant file (which includes boost), add the -mno-thumb flag. You can do this directly in project.pbxproj also where each file has
settings = { COMPILER_FLAGS = ""; };
you just change this to
settings = { COMPILER_FLAGS = "-mno-thumb"; };
But there's a little more. You also have to modify the darwin.jam file in the tools/build/v2/tools directory. In boost 1.48, there is a code that says:
case arm :
{
options = -arch armv6;
}
This has to be modified to
case arm :
{
options = -arch armv7 ;
}
Finally, in the boost.sh script, in the function writeBjamUserConfig(), you should remove the references to -arch armv6.
If somebody knows how to do this a little more generally and cleanly, I'm sure we'd all benefit. For now, this is where I've gotten to, and I hope that this will help other IOS boost threads users. I hope that the various variants on the boost.sh IOS script out there will be updated. I plan to add some more links to this answer later.
Update: For a great article which describes the issue on the processor level,
see here:
http://preshing.com/20121019/this-is-why-they-call-it-a-weakly-ordered-cpu
Enjoy!
I use boost.asio, boost.thread, boost.smart_ptr etc. on iOS platform, the app always crash when run in release mode, which throws signal sigabrt. The crash call stack is :
__stack_chk_fail
boost::asio::detail::completion_handle
boost::asio::detail::task_ios_service_operation::complete
boost::asio::detail::task_io_service::do_run_one
boost::asio::detail::task_ios_service::run
boost::asio::io_service::run
![when create a asio work with creating new thread and io_service][1]
When trying to solve the problem, I found the following articles:
[boost-thread-threads-not-starting-on-the-iphone-ipad-in-release-build][2]
[The issue of spin_lock and thumb on iOS][3]
Then I try to add -mno-thumb to my project compile flag, and the problem occured in release mode is gone.
However, a new bug bring out : EXC_ARM_DA_ALIGN, which crashed at where I try to convert network data to host-endian.
As[this article][4] says, the ARM instructions strict that the memory data must be aligned.
And follow the article [Exc_arm_da_align][5], I fix it by using memcpy for the data convert, instead of directly converting from the pointer.
[1]: http://i.stack.imgur.com/3ijF4.png
[2]: http://stackoverflow.com/questions/4201262/boost-thread-threads-not-starting-on-the-iphone-ipad-in-release-builds/4245821#4245821
[3]: http://groups.google.com/group/boost-list/browse_thread/thread/7dc1e80659182ab3
[4]: https://brewx.qualcomm.com/bws/content/gi/common/appseng/en/knowledgebase/docs/kb95.html
[5]: http://www.cnblogs.com/unionfind/archive/2013/02/25/2932262.html