Working on a toy project that I started to answer an SO question I'm getting flooded by a g++ warning that I don't understand.
format.hpp:230: warning: dereferencing pointer ‘<anonymous>’
does break strict-aliasing rules
searching on the internet I've got the impression that this could be a g++ bug; is it really a bug and if yes is there any workaround for it? The full source code is too big for inclusion but is available here. Here is the part where the warning is triggered...
template<typename T>
class ValueWrapper : public ValueWrapperBase
{
public:
T x;
ValueWrapper(const T& x) : x(x) {}
virtual std::string toString(const Field& field) const
{
return Formatter<T>().toString(x, field);
}
private:
// Taboo
ValueWrapper(const ValueWrapper&);
ValueWrapper& operator=(const ValueWrapper&);
};
typedef std::map<std::string, ValueWrapperBase *> Env;
class Dict
{
private:
Env env;
public:
Dict() {}
virtual ~Dict()
{
for (Env::iterator i=env.begin(), e=env.end(); i!=e; ++i)
delete i->second;
}
template<typename T>
Dict& operator()(const std::string& name, const T& value)
{
Env::iterator p = env.find(name);
if (p == env.end())
{
env[name] = new ValueWrapper<T>(value);
}
else
{
ValueWrapperBase *vw = new ValueWrapper<T>(value);
delete p->second;
p->second = vw;
}
return *this;
}
const ValueWrapperBase& operator[](const std::string& name) const
{
Env::const_iterator p = env.find(name);
if (p == env.end())
throw std::runtime_error("Field not present");
return *(p->second);
}
private:
// Taboo
Dict(const Dict&);
Dict& operator=(const Dict&);
};
Line 230 is p->second = vw;.
I get the warning for every instantiation of the template method operator(), always about line 230.
EDIT
Apparently the bug is about the use of map iterators that can generate inline code that confuses the optimizer. Rewriting a section avoiding using iterators I got shorter code that also compiles cleanly without warnings.
template<typename T>
Dict& operator()(const std::string& name, const T& value)
{
ValueWrapperBase *vw = new ValueWrapper<T>(value);
ValueWrapperBase *& p(env[name]);
delete p;
p = vw;
return *this;
}
As far as I can tell this actually stems from code in map and not from your code itself.
According to http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42032 and http://gcc.gnu.org/bugzilla/show_bug.cgi?id=43978 which both deal with maps and are very similar to each other, that there are absolutely some cases where it warns incorrectly because it loses track of the dynamic types of the objects. They equally state that there are some cases where it warms properly.
Also they indicate that the warning is shushed in 4.5 until they can implement it properly.
Finally, did you try rewriting your method as follows to see if it helps the warning in 4.3/4.4?
template<typename T>
Dict& operator()(const std::string& name, const T& value)
{
ValueWrapperBase *vw = new ValueWrapper<T>(value);
delete env[name];
env[name] = new ValueWrapper<T>(value);
return *this;
}
-fno-strict-aliasing (see http://gcc.gnu.org/onlinedocs/gcc-4.2.4/gcc/Optimize-Options.html#index-fstrict_002daliasing-572) turns off gcc's strict aliasing optimisations, and (presumably) with it the warning.
See also What is the strict aliasing rule? and http://cellperformance.beyond3d.com/articles/2006/06/understanding-strict-aliasing.html.
I've seen this "error" before and decided that it's often meaningless. I don't see anything wrong with your code. You might try your luck with newer versions of GCC--I seem to recall seeing this pop up somewhere around 4.3-4.4.
Edit: I said this warning/error is "often" meaningless. Not "usually." I absolutely do not advocate simply ignoring or disabling warnings just because they are annoying, but in this code, and in some of my own code, there is no apparent problem despite GCC's complaint.
Related
Revisiting lifetime extension in C++, I found out that there are some patterns that break "decomposability" of C++ expressions. For example, the following two blocks are a valid C++ code:
class NonMovable {
public:
NonMovable(NonMovable&&) = delete;
NonMovable(const NonMovable&) = delete;
NonMovable();
int Value() const;
};
template <class T>
const T& identity(const T& x) {
return x;
}
template <class T>
class A {
public:
explicit A(const T& value) : value_(value) {}
const T& GetValue() const {
return value_;
}
private:
const T& value_;
};
Correct usage:
int main() {
int retcode = identity(
identity(/*tmp1*/ A(/*tmp2*/ NonMovable{}).GetValue())).Value();
// tmp1 and tmp2 end their lifetimes here:
// their full-expression is the whole previous line
return retcode;
}
But if we decompose the first expression in main, it becomes invalid:
int main() {
auto&& a_obj = /*tmp1*/ A(/*tmp2*/ NonMovable{});
// tmp2 lifetime ends here
// oops! dereferencing dangling reference:
int retcode = identity(
identity(a_obj.GetValue())).Value();
return retcode;
// tmp1 lifetime ends here
}
My question is:
Is it possible to disable the second kind of usage?
P.S.: I'm not really sure if the second main introduces UB, because I've tested it with clang -Wlifetime, and it doesn't complain. But I still believe it is UB. In real life I've came across a similar behaviour: the code broke, emmiting UBSan warnings and segfaults if I decomposed a single expression into two separate ones.
P.P.S.: those identitys don't really matter much, if I understand object lifetimes correctly (which I now doubt)
Your analysis is correct. Without lifetime extension, all temporaries are destroyed at the end of the "full expression", i.e. the ; at the end of the line. So when you say
int retcode = A(NonMovable{}).GetValue().Value();
(comments and identity calls removed for clarity) then everything is okay; the NonMovable object is still alive at the time you ask for its value.
On the other hand, when you say
auto&& a_obj = A(NonMovable{});
then the NonMovable is destroyed at the end of the line, and the A object will be holding a dangling reference. (As an aside, auto&& just lifetime-extends the temporary A here -- you may as well just use plain auto)
My question is: Is it possible to disable the second kind of usage?
Not really, at least as far as I know. You could add a deleted A(NonMovable&&) constructor, but this would also prevent "correct" usage as in the first example. The is exactly the same issue that occurs with std::string_view (and will occur with std::span in C++20) -- essentially, your A class has reference semantics, but is referring to a temporary which has been destroyed.
So by using collective mind, in the comments under the question we've managed to come up with the following implementation of A, which might be applicable to some use cases (but not std::span or std::string_view usage):
struct Dummy;
template <class T>
class A {
public:
explicit A(const T& value) : value_(value) {}
template <class TDummy = Dummy>
const T& GetValue() const& {
static_assert(!std::is_same_v<TDummy, Dummy>,
"Stop and think, you're doing something wrong!"
"And in any case, don't use std::move on this class!");
}
const T& GetValue() && {
return value_;
}
private:
const T& value_;
};
Now, if one tries to compile the following code, he will get a descriprive error message:
int main() {
auto&& a_obj = A(NonMovable{});
// will not compile:
int retcode = identity(
identity(a_obj.GetValue())).Value();
return retcode;
}
The reason is that decltype((a_obj)) == A<NonMovable>&, so it binds to the method that produces a compile time error.
It satisfies my use cases, but, sadly, this is not a universal solution -- it depends on what one wants from class A.
I am moving my project from VS2015 to VS2017, which of course does not go smoothly.
I am seeing strange compiler error that can be reproduced by the following code:
struct MoveOnly
{
MoveOnly() {}
MoveOnly(const MoveOnly&) = delete;
MoveOnly& operator = (const MoveOnly&) = delete;
MoveOnly(MoveOnly&&) = default;
MoveOnly& operator = (MoveOnly&&) = default;
bool operator == (const MoveOnly& rhs)const{return false;}
};
struct Hasher
{
size_t operator()(const MoveOnly&)const{return 0;}
};
std::vector < std::unordered_map<MoveOnly, int, Hasher> > test;
test.emplace_back();
I can successfully compile this code with all compilers (gcc 7.2, clang 5.0.0, icc 18, as well as MSVC 2015). Please follow this link to see the test:
https://godbolt.org/g/uSqwDJ. On MSVC 2017 (19.10.25017), however there is an error that is caused by the compiler trying to reference deleted copy constructor of MoveOnly type. This error does not make much sense to me, because there is no reason to copy anything here instead of moving. /std:c++14, /std:c++17, /std:c++latest do not help. Also the fact the gcc and clang handle the code correctly makes me suspicious about msvc 2017 compiler.
Update:
After Yakk found what the problem is, I tried using other containers in place of unordered_map and the code only compiles with vector.
So the problem seems to be this:
static_assert( noexcept(std::unordered_map<MoveOnly, int, Hasher>( std::declval<std::unordered_map<MoveOnly, int, Hasher>&&>())), "");
Your compiler doesn't think std::unordered_map<MoveOnly, int, Hasher> can be noexcept move-constructed.
Then, the paranoid std::vector implementation that MSVC 2017 ships with falls back on copying elements to generate a strong exception guarantee on vector resize.
Copying is obviously not possible.
Now the same is true of std::unordered_map<int, int> -- MSVC thinks that moving it also risks throwing exceptions; I believe nothing you can do with the key or hash type can possibly make the move constructor of unordered_map exception-safe.
There is no good reason for unordered_map(unordered_map&&) to not be noexcept. I am uncertain if the standard permits it or mandates it, or if it is a bug in the compiler; if the standard mandates it to be noexcept(false), then the standard has a defect.
You can work around this by storing a vector of unique_ptrs. Or write an exception-eating value_ptr with fewer changes to your code:
template<class T>
struct value_ptr {
std::unique_ptr<T> raw;
value_ptr() noexcept(true)
{
try {
raw = std::make_unique<T>();
} catch (...) {}
}
template<class T0, class...Ts,
std::enable_if_t<!std::is_same<value_ptr, std::decay_t<T0>>{}, bool> =true
>
value_ptr(T0&& t0, Ts&&...ts) noexcept(true)
{
try {
raw=std::make_unique<T>( std::forward<T0>(t0), std::forward<Ts>(ts)... )
} catch(...) {}
}
value_ptr(value_ptr&& o)noexcept(true)=default;
value_ptr(value_ptr const& o)noexcept(true)
{ try {
if (o.raw)
raw = std::make_unique<T>(*o.raw);
}catch(...){}
}
value_ptr& operator=(value_ptr&& o)noexcept(true)=default;
value_ptr& operator=(value_ptr const& o)noexcept(true)
{ try {
if (o.raw)
raw = std::make_unique<T>(*o.raw);
}catch(...){}
return *this;
}
T* operator->() const { return raw.get(); }
T& operator*() const { return *raw; }
explicit operator bool() const { return (bool)raw; }
};
Here is my test harness:
template<class M>
void test_M() {
static_assert( noexcept(M( std::declval<M&&>())), "");
std::vector < M > test;
test.emplace_back();
}
void foo()
{
using M0=value_ptr<std::unordered_map<MoveOnly, int, Hasher>>;
using M1=std::unique_ptr<std::unordered_map<MoveOnly, int, Hasher>>;
test_M<M0>();
test_M<M1>();
}
Clang 3.9 extremely reuses memory used by temporaries.
This code is UB (simplified code):
template <class T>
class my_optional
{
public:
bool has{ false };
T value;
const T& get_or_default(const T& def)
{
return has ? value : def;
}
};
void use(const std::string& s)
{
// ...
}
int main()
{
my_optional<std::string> m;
// ...
const std::string& s = m.get_or_default("default value");
use(s); // s is dangling if default returned
}
We have tons of code something like above (my_optional is just a simple example to illustrate it).
Because of UB all clang compiler since 3.9 starts to reuse this memory, and it is lawful behavior.
The question is: how to detect such dangling references at compile time or with something like sanitizer at runtime? No clang sanitizer can detect them.
Upd. Please do not answer: "use std::optional". Read carefully: question is NOT about it.
Upd2. Please do not answer: "your code design is bad". Read carefully: question is NOT about code design.
You can detect misuses of this particular API by adding an additional overload:
const T& get_or_default(T&& rvalue) = delete;
If the argument given to get_or_default is a true rvalue, it will be chosen instead, so compilation will fail.
As for detecting such errors at runtime, try using Clang's AddressSanitizer with use-after-return (ASAN_OPTIONS=detect_stack_use_after_return=1) and/or use-after-scope (-fsanitize-address-use-after-scope) detection enabled.
You could try out lvalue_ref wrapper from Explicit library. It prevents the unwanted binding to a temporary in one declaration, like:
const T& get_or_default(lvalue_ref<const T> def)
{
return has ? value : def.get();
}
That is an interesting question. The actual cause of the dangling ref is that you use an rvalue reference as if it was an lvalue one.
If you have not too much of that code, you can try to throw an exception that way:
class my_optional
{
public:
bool has{ false };
T value;
const T& get_or_default(const T&& def)
{
throw std::invalid_argument("Received a rvalue");
}
const T& get_or_default(const T& def)
{
return has ? value : def;
}
};
That way, if you pass it a ref to a temporary (which is indeed an rvalue), you will get an exception, that you will be able to catch or at least will give a soon abort.
Alternatively, you could try a simple fix by forcing to return a temporary value (and not a ref) if you were passed an rvalue:
class my_optional
{
public:
bool has{ false };
T value;
const T get_or_default(const T&& def)
{
return get_or_default(static_cast<const T&>(def));
}
const T& get_or_default(const T& def)
{
return has ? value : def;
}
};
Another possibility would be to hack the Clang compiler to ask it to detect whether the method is passed an lvalue or an rvalue, by I am not enough used to those techniques...
-edit- I narrowed it down. Reproducible: Why does passing this object in C break my code?
My app is not working properly after i made a change. I got a warning in msvc but not in gcc.
Heres a repo of the warning.
warning C4190: 'fnA' has C-linkage specified, but returns UDT 'Test' which is incompatible with C
#include <type_traits>
template<class T>
class Test{
T t;
};
typedef Test<int> A;
//static_assert(std::is_pod<A>::value, "Not a POD"); //fails in msvc 2010
static_assert(sizeof(A) == sizeof(int), "");
static_assert(sizeof(A) == sizeof(void*), "");
extern "C" {
A fnA(A v) { return v; }
}
int main() {
A a;
fnA(a);
}
AFAIK there should be no reason why i can't use Test as a return value; This may not be the problem but this may be the problem. I can't figure out the problem but i get runtime oddities that i can not reproduce (both gcc and msvc). I suspected the problem would be MyString is corrupted but it appears that isnt the case which made me quite confused. Heres my wrapper. MyString is >8bytes and i need to hand this off to C code which returns everything by int unfortunately. Which is why i put the static assert to see if the class size is what i expected. Now that size/splicing is handled, i'm still completely like WTF!?! Why does that warning exist and is there anything i can possibly do to fix it?
Even writing class Test{ T t; }; causes the warning however struct fixes it. struct with private breaks it and i need t to be private.
OK! After I removed the constructors in WrappedPointer and changed the class to struct (which makes lhs public). It runs perfectly in GCC and MSVC. Changing struct WrappedPointer to class WrappedPointer breaks my code. WTF!?! This is a debug build too, not optimized. Why on earth does changing the keyword struct to class break the code!?! WTF!?! BUT that change doesnt break gcc. Using non default constructors break gcc...
template <class T>
struct WrappedPointer {
//private:
T* lhs;
public:
void SetLHS(T*v) { lhs=v; }
//WrappedPointer(){}
//WrappedPointer(T*value) : lhs(value){}
//WrappedPointer(const WrappedPointer&v) : lhs(v.lhs){}
T* operator->() const { return lhs; }
T* operator*() const { return lhs; }
template<class TT>
bool operator==(TT t) const { return *lhs==t; }
template<class TT>
bool operator!=(TT t) const { return *lhs!=t; }
bool operator==(int v) const { myassert2(v==0); return lhs==0; }
bool operator!=(int v) const { myassert2(v==0); return lhs!=0; }
bool operator==(const WrappedPointer&t) const { return *lhs==*t; }
bool operator!=(const WrappedPointer&t) const { return *lhs!=*t; }
}
typedef WrappedPointer<MyString> String;
//typedef MyString* String;
static_assert(sizeof(String) == sizeof(int), "");
static_assert(sizeof(String) == sizeof(void*),"");
The external "C" marks the function to have C linkage and disables name mangling. Now the issue in your function is that the argument is a template (the typedef only creates an alias in the current translation unit, A is still Test<int> for all purposes), and that name must be mangled.
extern "C" {
A fnA(A v) { return v; }
}
This says that fnA is a C function. C does not have object-oriented programming, or templates. The function uses both. Therefore, it cannot link as a C function.
Classes are not valid in C.
Templates are not valid in C.
Templated classes are certainly not valid in C.
Pick your language and stick to it!
I was implementing (for training purpose) a Bubble Sort template function:
template<typename iterInput,
typename predicate>
void BubbleSort(iterInput first1,iterInput last1,predicate func)
{
bool swapped(false);
do
{
swapped = false;
iterInput begin = first1;
iterInput beginMinus = first1;
++begin;
for (;begin != last1; begin++,beginMinus++)
{
if (func(*beginMinus,*begin) )
{
std::swap(*beginMinus,*begin);
swapped = true;
}
}
}
while(swapped);
}
When I have realized that this function will not work for class with no assignment operator, like this one (forgive me for the bad name):
class NoCopyable
{
public:
explicit NoCopyable(int value) : value_(value) {}
NoCopyable(const NoCopyable& other) : value_(other.value_) {}
~NoCopyable() {}
bool operator<(const NoCopyable& other) { return value_ < other.value_; }
void setValue(int value) { value_ = value; }
std::ostream& print(std::ostream& os) const { return os << value_; }
private:
NoCopyable& operator=(const NoCopyable& other);
int value_;
};
std::ostream& operator<<(std::ostream& os, const NoCopyable& obj)
{
return obj.print(os);
}
struct PrintNoCopyable
{
void operator()(const NoCopyable& noCopyable) { std::cout << noCopyable << '\n'; }
};
The compiler raises this error Error 1 error C2248: 'NoCopyable::operator =' : cannot access private member declared in class 'NoCopyable'
So, I have slightly modify the code using instead of the std::swap function my version of the swap function, here is the code:
template<typename T1,
typename T2>
void noAssignmentSwap(T1& t1,T2& t2)
{
T1 temp(t1);
t1.~T1();
new (&t1) T1(t2);
t2.~T2();
new (&t2) T2(temp);
}
The code compiles and gives the right result. However I am not completely sure, I remember a Sutter's article that suggest you to avoid playing with the objects life time. The article just warns you by playing with fire without actually giving you any real reason. I can see problem in exception safety if the copy constructor of T1 or T2 can throw. However there is the same problem in the standard version if the assignment operator is allowed to throw.
Here the question, can you see any possible drawbacks in this version of swap?
Cheers
Apart from anything else, if a class does not have an assignment operator, its designer probably did not intend it to be swapped. If they did that, they probably disabled copy construction too, so your new swap function still won't work.
As for your assertion that Standard Library containers do not need assignment - that is true so long as you don't want to actually do anything useful with them. Does this code compile for you?
#include <vector>
using namespace std;
struct A {
private:
void operator=( const A &);
};
int main() {
vector <A> v;
v.push_back( A() );
v[0] = A(); // assignment needed here
}
I think it won't.
The difference is that when the assignment operator fails, you still have the same number of objects.
If you destroy one object and fail to create a new one, one object is lost! If it was part of a container, the container's state is probably also invalid.
You need a copy ctor instead of an assignment operator, but the two are sufficiently similar that at least in a typical case, you'll have both or you'll have neither. IOW, I don't think this generally accomplishes much.
I'd class it right along side the xor-swap trick: interesting, but generally useless.
It might be confusing to a future maintainer of the code.