The C++20 draft, [except.handle]/12 says: (emphasis added)
Exceptions thrown in destructors of objects with thread storage duration or in constructors of namespace-scope objects with thread storage duration are not caught by a function-try-block on the initial function of the thread.
and [except.terminate]/1.10 says (emphasis added)
In some situations exception handling is abandoned for less subtle
error handling techniques. [Note 1: These situations are:
[..]
(1.10) when execution of the initial function of a thread exits via an exception [..]
What is "the initial function of a thread"? Is it the main function? Is it the implementation of std::invoke as specified in [thread.thread.constr]/6? or something else?
I asked someone for that, and the answer was that the initial function of a thread is the first function runned by this thread; and the first function runned by a thread is always the implementation of std::invoke. Is this true?
If that's not true, what is the actual definition of the "initial function of a thread" in the context of the above standard wording?
I asked someone for that, and the answer was that the initial function of a thread is the first function runned by this thread; and the first function runned by a thread is always the implementation of std::invoke. Is this true?
This is not meaningfully true.
The phrase "initial function" has no particular definition in the standard, so it's meant to be taken literally: the function initially run by a thread. And the constructor of std::thread/jthread does say that:
The new thread of execution executes
invoke(decay-copy(std::forward<F>(f)), decay-copy(std::forward<Args>(args))...)
with the calls to decay-copy being evaluated in the constructing thread.
However, the "as if" rule still applies. A particular implementation does not specifically have to use std::invoke; it only must behave "as if" it did. And while users are allowed to provide specializations for standard library class templates with user-defined types, they are not allowed to specialize standard library function templates, or overload them outside of very specific circumstances. So there's no way for a user to (validly) interfere with the meaning of std::invoke.
As such, an implementation doesn't specifically have to use std::invoke at all. It could use behavior equivalent to it.
More importantly... it wouldn't matter.
Your program can't change what happens after the call to the function you give the thread. Since you cannot affect that code, any behavior around the "initial function" wording applies to the only thing you can control: the behavior of the function you give it
So if the function you give thread emits an exception, that will necessarily trigger the clause about the behavior of an "initial function of the thread" which doesn't catch an exception. There is no way to cause something below that function call to catch or otherwise interfere with such clauses.
So for all meaningful purposes, "initial function of the thread" is "whatever function you pass that gets called by thread/jthread's constructor".
In invoke(decay-copy(std::forward<F>(f)), decay-copy(std::forward<Args>(args))...
Where f is callable and your initial function.
Related
The C++ Standard (c++14 and also the current working draft) states that
Except where explicitly specified in this document, it is
implementation-defined which functions in the C++ standard library may
be recursively reentered.
Except of std::recursive_mutex I couldn't find any type or function in the standard where it is explicitly specified that it can be recursively reentered safely. Even for std::recursive_mutex the standard states that:
It is unspecified how many levels of ownership may be acquired by a
single thread.
My questions for C++17:
Which types or functions, defined by the C++ standard, have to be implemented in a way that they can be recursively reentered?
Reentrant code may not call non-reentrant functions (see rules for reentrancy below). With this in mind, is it correct to summarize that C ++ code can only be safely reentrant if no C++ standard types or functions are used?
Please note, that reentrancy and thread-safety are orthogonal concepts:
In computing, a computer program or subroutine is called reentrant if
multiple invocations can safely run concurrently on a single processor
system, where a reentrant procedure can be interrupted in the middle
of its execution and then safely be called again ("re-entered") before
its previous invocations complete execution. The interruption could be
caused by an internal action such as a jump or call, or by an external
action such as an interrupt or signal, unlike recursion, where new
invocations can only be caused by internal call.
This definition of reentrancy differs from that of thread-safety in
multi-threaded environments. (...) Conversely, thread-safe code does
not necessarily have to be reentrant (...)
Rules for reentrancy:
Reentrant code may not hold any static or global non-constant data.
Reentrant code may not modify itself.
Reentrant code may not call non-reentrant computer programs or routines.
Which types or functions, defined by the C++ standard, have to be implemented in a way that they can be recursively reentered?
Reentrancy is a pre-requisite for asynchronous signal safety. Therefore functions which are specified as signal-safe must be implemented as reentrant.
[support.signal]
An evaluation is signal-safe unless it includes one of the following:
a call to any standard library function, except for plain lock-free atomic operations and functions explicitly identified as signal-safe;
A plain lock-free atomic operation is an invocation of a function f from [atomics], such that:
f is the function atomic_is_lock_free(), or
f is the member function is_lock_free(), or
f is a non-static member function invoked on an object A, such that A.is_lock_free() yields true, or
f is a non-member function, and for every pointer-to-atomic argument A passed to f, atomic_is_lock_free(A) yields true.
As such, those plain lock-free atomic opreations must be reentrant.
I just read this article about the actual reasons behind the current boost::mutex implementation and noticed the following phrase:
Block-scope statics have the additional problem of a potential race
condition on "the first time through", which can lead to the
destructor being run multiple times on popular compilers, which is
undefined behaviour — compilers often use the equivalent of a call to
atexit in order to ensure that destruction is done in the reverse
order of construction, and the initialization race that may cause the
constructor to be run twice may also cause the destructor to be
registered twice
Is it true? Should I really check whether another thread already inside this object's destructor via atomic operations or something like this? Should I do it even in C++11 - C++14? Because as far as I know there's no more "constructor for the same local object with static storage duration can be called simultaneously from several threads" problem since C++11 -- it requires that another threads should wait for the constructor's completion. Am I right?
It looks like this article was written pre C++11, it says amongst other things:
[...] next version of the C++ Standard, scheduled to be released in 2009.[...]
and this was the case pre C++11, it was unspecified what happened in this case since threading was not part of memory model pre C++11.
This changed in C++11 and the draft C++11 standard section 6.7 Declaration statement says (emphasis mine):
The zero-initialization (8.5) of all block-scope variables with static
storage duration (3.7.1) or thread storage duration (3.7.2) is
performed before any other initialization takes place. [...] Otherwise
such a variable is initialized the first time control passes through
its declaration; such a variable is considered initialized upon the
completion of its initialization. If the initialization exits by
throwing an exception, the initialization is not complete, so it will
be tried again the next time control enters the declaration. If
control enters the declaration concurrently while the variable is
being initialized, the concurrent execution shall wait for completion
of the initialization. [...]
Pre C++11 we have to treat the static local variable just like we treat any other critical section. We can find a excellent description of the situation pre C++11 in the post C++ scoped static initialization is not thread-safe, on purpose!.
The C++11 standard says:
30.6.6 Class template future
(3) "The effect of calling any member function other than the destructor,
the move-assignment operator, or valid on a future object for which
valid() == false is undefined."
So, does it mean that the following code might encounter undefined behaviour?
void wait_for_future(std::future<void> & f)
{
if (f.valid()) {
// what if another thread meanwhile calls get() on f (which invalidates f)?
f.wait();
}
else {
return;
}
}
Q1: Is this really a possible undefined behaviour?
Q2: Is there any standard compliant way to avoid the possible undefined behaviour?
Note that the standard has an interesting note [also in 30.6.6 (3)]:
"[Note: Implementations are encouraged
to detect this case and throw an object of type future_error with an
error condition of future_errc::no_state. —endnote]"
Q3: Is it ok if I just rely on the standard's note and just use f.wait() without checking f's validity?
void wait_for_future(std::future<void> & f)
{
try {
f.wait();
}
catch (std::future_error const & err) {
return;
}
}
EDIT: Summary after receiving the answers and further research on the topic
As it turned out, the real problem with my example was not directly due to parallel modifications (a single modifying get was called from a single thread, the other thread called valid and wait which shall be safe).
The real problem was that the std::future object's get function was accessed from a different thread, which is not the intended use case! The std::future object shall only be used from a single thread!
The only other thread that is involved is the thread that sets the shared state: via return from the function passed to std::async or calling set_value on the related std::promise object, etc.
More: even waiting on an std::future object from another thread is not intended behaviour (due to the very same UB as in my example#1). We shall use std::shared_future for this use case, having each thread its own copy of an std::shared_future object. Note that all these are not through the same shared std::future object, but through separate (related) objects!
Bottom line:
These objects shall not be shared between threads. Use a separate (related) object in each thread.
A normal std::future is not threadsafe by itself. So yes it is UB, if you call modifying functions from multiple threads on a single std::future as you have a potential race condition. Though, calling wait from multiple threads is ok as it's const/non-modifying.
However, if you really need to access the return value of a std::future from multiple threads you can first call std::future::share on the future to get a std::shared_future which you can copy to each thread and then each thread can call get. Note that it's important that each thread has its own std::shared_future object.
You only need to check valid if it is somehow possible that your future might be invalid which is not the case for the normal usecases(std::async etc.) and proper usage(e.g.: not callig get twice).
Futures allow you to store the state from one thread and retrieve it from another. They don't provide any further thread safety.
Is this really a possible undefined behaviour?
If you have two threads trying to get the future's state without synchronisation, yes. I've no idea why you might do that though.
Is there any standard compliant way to avoid the possible undefined behaviour?
Only try to get the state from one thread; or, if you genuinely need to share it between threads, use a mutex or other synchronisation.
Is it ok if I just rely on the standard's note
If you known that the only implementations you need to support follow that recommendation, yes. But there should be no need.
and just use f.wait() without checking f's validity?
If you're not doing any weird shenanigans with multiple threads accessing the future, then you can just assume that it's valid until you've retrieved the state (or moved it to another future).
I wanted to know how appropriate its to use std::async in performance oriented code. Specifically
Is there any penalty in catching the exception from worker thread to main thread?
How are the values returned from worker to main?
Are the input arguments passed by ref actually never get copied or not?
I am planning to pass a heavy session object to a thread or write std::async.
bool fun(MySession& sessRef);
MySession sess;
auto r = std::async(&fun, sess);
EDIT:
I am planning to use it with GCC 4.9.1 and VS2013 both since the application is platform agnostic. However most deployments will be *nix based so atleast GCC should be performant.
We can't tell exactly "how is std::async implemented", since you're not referring to a particular toolchain that provides that implementation actually.
1. Is there any penalty in catching the exception from worker thread to main thread?
Define "Penalty" by which means exactly? That can't be answered unless you clarify about your concerns/requirements.
Usually there shouldn't be any penalty, by just catching the exception in the thread, that created the throwing one. It's just about the exception may be provided to the creating thread via the join(), and this causes some cost for keeping that particular exception through handling of join().
2. How are the values returned from worker to main?
To cite what's the c++ standards definition saying about this point:
30.6.8 Function template async
4 Returns: An object of type future<typename result_of<typename decay<F>::type(typename decay<Args>::type...)>::type> that refers to the shared state created by this call to async.
3. Are the input arguments passed by ref actually never get copied or
not?
That point is answered in detail here: Passing arguments to std::async by reference fails. As you see, the default case they are copied.
According to #Yakk's comment, it might be possible to pass these parameters via std::ref to avoid operating on copies, but take references.
how is std::async implemented
I can tell only for the c++ standards requirements, how it should be implemented, unless you're referring to a particular toolchain, that tries to provide a proper implementation of std::async.
C99 offers the _Exit function, which exits "immediately", although it does may close file descriptors. Unix/POSIX extends this behavior by mandating the closing of all fd's without flushing (and offers the synonym _exit).
Will these functions call destructors for static objects when called from a C++ program? Does the C++ standard make any guarantees about _Exit?
(Inspired by this question; I suddenly wondered what happens in the typical fork-exec-_exit idiom in C++.)
First, no form of program exit will automatically call destructors for heap objects (implied in ISO/IEC 14882:1998(E) 12.4.10).
Calling exit() will not call destructors for objects with automatic duration, as it does not return through their enclosing scopes (3.6.1.4). However, destructors for static objects will be called, in reverse order of construction (18.3.8).
Calling abort() does not call any destructors for any type of object, nor does it call atexit() registered functions (18.3.3). The C++ standard copy I have here is a bit dated and does not mention _exit or _Exit directly, but I'd imagine that, if present, they should behave the same - that is, not calling any destructors. In particular, in the C99 standard, _Exit() skips atexit handlers (it is implementation defined whether stream buffers are flushed, open streams are closed, or temporary files removed).
Further note that abort() can be cancelled by trapping signal SIGABRT (ISO/IEC 9899:1999 (E) 7.20.4.1.2 - I only have C99 here but I expect it would be the same in the version referenced by C++). _Exit() cannot.
On a more practical note, on most unix implementations of abort() and _exit(), abort() raises a SIGABRT while _exit() simply calls an operating system call to terminate the process immediately. This means that the main differences are:
You can specify an exit code for _exit()
abort() may be trapped by a signal handler
Depending on system configuration, OS, and ulimits, abort() may result in a core dump or similar
In a fork()/exec() pattern, _exit() would probably be preferable, to avoid the possibility of core dump.
It simply doesn't exist in standard C++, so there are no guarantees.
It is planned for inclusion in C++0x. That specifies (§18.5):
The function _Exit(int status) has
additional behavior in this
International Standard:
— The program is terminated without
executing destructors for objects of
automatic, thread, or static storage
duration and without calling functions
passed to atexit() (3.6.3).
Followup:
ISO approved C++0x on August 12, 2011.
Technically, _Exit is not defined by the C++ standard, so you can't even call it from a 100% portable C++ program. The C++03 standard incorporates by reference the C89 standard (aka C90 or ANSI C), whereas _Exit is only defined in the newer C99 standard. I'm not sure which version of C the upcoming C++0x standard incorporates, but I would guess that it's based on C99.
In any case, though, here are the relevant clauses from the relevant language standards:
_Exit is not guaranteed to close file descriptors. From C99 §7.20.4.4/2 (emphasis mine):
The _Exit function causes normal program termination to occur and control to be returned to the host environment. No functions registered by the atexit function or signal handlers registered by the signal function are called. The status returned to the host environment is determined in the same way as for the exit function (7.20.4.3). Whether open streams with unwritten buffered data are flushed, open streams are closed, or temporary files are removed is implementation-defined.
Recall that implementation-defined means that the implementation (that is, the compiler toolchain and runtime environment) can choose to do whatever it wants, but it must document what it does.
From C++03 §3.6.3/1:
Destructors (12.4) for initialized objects of static storage duration (declared at block scope or at namespace scope) are called as a result of returning from main and as a result of calling exit (18.3). These objects are destroyed in the reverse order of the completion of their constructor or of the completion of their dynamic initialization. If an object is initialized statically, the object is destroyed in the same order as if the object was dynamically initialized. For an object of array or class type, all subobjects of that object are destroyed before any local object with static storage duration initialized during the construction of the subobjects is destroyed.
§3.6.3/4:
Calling the function
void abort();
declared in <cstdlib> terminates the program without executing destructors for objects of automatic or static storage duration and without calling the functions passed to atexit().
Practically, in most implementations, global object destructors are implemented via atexit, so what you will see is that _Exit will not call the destructors for global objects, although this behavior is not guaranteed (since _Exit and C++ are not guaranteed to both exist in the same language).
Note that while C++ does not specify _Exit and C99 leaves it implementation-defined whether it flushes buffers, POSIX requires that it not flush buffers (since this would break the main usage of _exit/_Exit, i.e. handling failure of execve after fork). As POSIX does not align itself with C++ standards or defer to them on anything, I think it's very unlikely that a future version of the C++ standard would try to change any of this. It will probably either leave _Exit unspecified or specify that it's implementation-defined.
C++0x defines a new function called std::quick_exit that terminates a process without calling any destructors. Just checked, g++-4.4.5 already provides it.
There is an interesting analysis here in relation with concurrency and object destruction. As far as I know, destructors will not be called. There is nothing about it in the current standard.
Calling of static destructors is defined in terms of atexit. _exit (or _Exit) is defined not to run atexit handlers. So static destructors should not be called by any implementation.
Automatic destructors are not even called when calling exit().
So any sane definition of _Exit semantics for C++ would not run destructors.
I did a quick test with gcc on Mac OS and my destructors didn't get called.
struct A
{
~A()
{
puts("A::~A");
}
};
A globalA;
int main()
{
A localA;
_exit(0); // or _Exit(0)
return 0;
}
exit(0) on the other hand calls globalA's destructor.
fork(), exec(), and _exit() are all defined by POSIX and they pre-date C99's _Exit() by many years. Programs that use fork/exec/_exit are not portable to every system that supports C++.
With regard to _exit() specifically, it is an operating system call that (under POSIX) will close files and terminate the process directly (but not necessarily quickly). This would bypass any C++ mechanisms for calling destructors.
Even with _Exit() or similar being provided by C++0x, I doubt if there would be much reason to use that in conjunction with fork. It likely just provides broader portability for a "quick-exit" in other contexts. That functionality is already covered by _exit() if you are using the POSIX API.
Program termination is addressed in C++2003 section [3.6.3]. It says that static objects are destructed implicitly when main() returns and when exit() is called. It also says that such objects are NOT destructed when abort() is called. _exit() isn't addressed in the C++ 2003 standard, but the fact that it is meant to bypass language-specific cleanup is described in the POSIX documentation. That effect is further substantiated by what is stated and by what is NOT stated in the C++ standard.