Following code writes into invalid area in memory but there's no compilation error.
int _tmain(int argc, _TCHAR* argv[])
{
char* s1 = new char[10];
for(int i=0;i<20;i++) s1[i]='a';
cout << s1 << endl;
return 0;
}
In runtime the code terminates with return code 0 printing 20 a's then some garbage before it met 0 but I assume this is very dangerous as it could contaminate / illegally access other area in memory.
Is there any way such mistake can be detected in compile time? Or at least a runtime exception raised pointing straight into s1[i]='a' line?
Setting /RTCs and /GS flag as recommended in other posts did not help.
Running Visual Studio code analysis (ANALZYE -> Run Code Analysis) gives no result either.
There are list of tools third party tools posted in here: C++ static code analysis tool on Windows but I'm hoping there's a way this can be detected by Visual Studio alone?
According to http://msdn.microsoft.com/en-us/library/8dbf701c.aspx, /GS (Buffer Security Check) is carried out in run-time, not in compile type.
/RTCs (according to http://msdn.microsoft.com/en-us/library/8wtf2dfz.aspx) controls run-time checks. So neither of these two switches were designed to do static analysis of your code. That's, they are not supposed to detect your problem at compile time.
I think static code analysis is still in research stage in general, I'd be surprised that VS 2012 would provide full fledged support.
Another possibility is that the specific types of error you are trying to detect is an array-out-of-bound error. Buffer-overrun may not be the right keyword to do search.
Related
Using Visual Studio 2019 Professional on Windows 10 x64. I have several C++ DLL projects, some of which are multi-threaded. I'm using CRITICAL_SECTION objects for thread safety.
In DLL1:
CRITICAL_SECTION critDLL1;
InitializeCriticalSection(&critDLL1);
In DLL2:
CRITICAL_SECTION critDLL2;
InitializeCriticalSection(&critDLL2);
When I use critDLL1 with EnterCriticalSection or LeaveCriticalSection everything is fine in both _DEBUG or NDEBUG mode. But when I use critDLL2, I get an access violation in 'ntdll.dll' in NDEBUG (though not in _DEBUG).
After popping up message boxes in NDEBUG mode, I was eventually able to track the problem down to the first use of EnterCriticalSection.
What might be causing the CRITICAL_SECTION to fail in one project but work in others? The MSDN page was not helpful.
UPDATE 1
After comparing project settings of DLL1 (working) and DLL2 (not working), I've accidentally got DLL2 working. I've confirmed this by reverting to an earlier version (which crashes) and then making the project changes (no crash!).
This is the setting:
Project Properties > C/C++ > Optimization > Whole Program Optimization
Set this to Yes (/GL) and my program crashes. Change that to No and it works fine. What does the /GL switch do and why might it cause this crash?
UPDATE 2
The excellent answer from #Acorn and comment from #RaymondChen, provided the clues to track down and then resolve the issue. There were two problems (both programmer errors).
PROBLEM 1
The assumption of Whole Program Optimzation (wPO) is the MSVC compiler is compiling "the whole program". This is an incorrect assumption for my DLL project which internally consumes a 3rd party library and is in turn consumed by an external application written in Delphi. This setting is set to Yes (/GL) by default but should be No. This feels like a bug in Visual Studio, but in any case, the programmer needs to be aware of this. I don't know all the details of what WPO is meant to do, but at least for DLLs meant to be consumed by other applications, the default should be changed.
PROBLEM 2
Serious programmer error. It was a call into a 3rd party library, which returned a 128-byte ASCII code which was the error:
// Before
// m_config::acSerial defined as "char acSerial[21]"
(void) m_pLib->GetPara(XPARA_PRODUCT_INFO, &m_config.acSerial[0]);
EnterCriticalSection(&crit); // Crash!
// After
#define SERIAL_LEN 20
// m_config::acSerial defined as "char acSerial[SERIAL_LEN+1]"
//...
char acSerial[128];
(void) m_pLib->GetPara(XPARA_PRODUCT_INFO, &acSerial[0]);
strncpy(m_config.acSerial, acSerial, max(SERIAL_LEN, strlen(acSerial)));
EnterCriticalSection(&crit); // Works!
The error, now obvious, is that the 3rd party library did not copy the serial number of the device into the char* I provided...it copied 128 bytes into my char* stomping over everything contiguous in memory after acSerial. This wasn't noticed before because m_pLib->GetPara(XPARA_PRODUCT_INFO, ...) was one of the first calls into the 3rd party library and the rest of the contiguous data was mostly NULL at that point.
The problem was never to do with the CRITICAL_SECTION. My thanks for Acorn and RaymondChen ... sanity has been restored to this corner of the universe.
If your program crashes under WPO (an optimization that assumes that whatever you are compiling is the entire program), it means that either the assumption is incorrect or that the optimizer ends up exploiting some undefined behavior that previously didn't (without the optimization applied), even if the assumption is correct.
In general, avoid enabling optimizations unless you are really sure you know you meet their requirements.
For further analysis, please provide a MRE.
I have a class where I define a circular buffer like so:
class cTest
{
public:
boost::circular_buffer<std::vector<std::pair<double, double>>> circDat;
cTest() : circDat(1000)
{
}
};
I then create a stl vector of type cTest
std::vector<cTest> vC;
Afterwards I try to fill the vector like this:
for (unsigned int i = 0; i < 4; ++i)
{
cTest obj;
vC.push_back(obj);
}
While this works in Debug mode, in Release, it crashes (sometimes, when I run with from Visual Studio, I get a Heap Corruption message). The boost documentation mentions, that in Debug mode, the uninitialized memory is filled with '0xcc'. I assume, the error I get, has its root in uninitialized memory. But I am not really sure, how to fix this problem.
If I use pointers, it seems to work:
std::vector<cTest*> vC;
for (unsigned int i = 0; i < 4; ++i)
{
cTest* obj = new cTest;
vC.push_back(obj);
}
But I still do not know, what the problem with the first version is. If anyone does know, I'd appreciate the help.
Edit:
I've tried to create a minimal, reproducable code but failed. It also seemed to crash randomly, not really correlating to the lines added/removed. I then stumbled on the /GL flag in Visual Studio 2015.
After turning the /GL flag off (in the GUI project - in the library project it can stay on), I've been unable to recreate the crash. I do not know, if this is really a fix. But it seems like there was a similar problem present in Visual Studio 2010:
crash-in-program-using-openmp-x64-only
Edit2:
I've managed to pull together a minimal working example. The code can be downloaded here:
https://github.com/davidmarianovak/crashtest
You need Boost (I used 1.60) and QT5 (I used 5.6.3). Build GoAcquire in Release (/GL is active in Visual Studio). Afterwards, build GoGUI in Release (activate /GL and use 'standard' for link-time code generation). After you've built it, run it and it should crash.
The crash can be avoided by changing this in 'GoInterface.hpp' line 22:
void fillGraphicsViews(std::vector<cSensorConstruct> vSens);
to
void fillGraphicsViews(std::vector<cSensorConstruct> &vSens);
But I do not really believe that it is the problem. Can anyone tell me, what I'm doing wrong? I'm using Visual Studio 2015 for this.
I bet you're forgetting about iterator/reference invalidation. So the problem is not with the code shown.
Iterator invalidation rules
This makes sense since you report that pointers seem to work: the pointers stay the same even if push_back causes reallocation.
Simply don't hold on to references/iterators to vector elements when you don't know that they're going to stay valid.
If your vector has a known maximum size, you could "cheat" by reserving the capacity ahead of time:
static constexpr size_t MAX_BUFFERS = 100;
std::vector<cTest> vC;
vC.reserver(MAX_BUFFERS); // never more
And then perhaps guard the invariant:
assert(vC.size() < MAX_BUFFERS);
vC.push_back(obj);
I have a C++ client to a C++/CLI DLL, which initializes a series of C# dlls.
This used to work. The code that is failing has not changed. The code that has changed is not called before the exception is thrown. My compile environment has changed, but recompiling on a machine with an environment similar to my old one still failed. (EDIT: as we see in the answer this is not entirely true, I was only recompiling the library in the old environment, not the library and client together. The client projects had been upgraded and couldn't easily go back.)
Someone besides me recompiled the library, and we started getting memory management issues. The pointer passed in as a String must not be in the bottom 64K of the process's address space. I recompiled it, and all worked well with no code changes. (Alarm #1) Recently it was recompiled, and memory management issues with strings re-appeared, and this time they're not going away. The new error is Unhandled Exception: System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
I'm pretty sure the problem is not located where I see the exception, the code didn't change between the successful and failing builds, but we should review that to be complete. Ignore the names of things, I don't have much control over the design of what it's doing with these strings. And sorry for the confusion, but note that _bridge and bridge are different things. Lots of lines of code missing because this question is already too long.
Defined in library:
struct Config
{
std::string aye;
std::string bee;
std::string sea;
};
extern "C" __declspec(dllexport) BridgeBase_I* __stdcall Bridge_GetConfiguredDefaultsImplementationPointer(
const std::vector<Config> & newConfigs, /**< new configurations to apply **/
std::string configFolderPath, /**< folder to write config files in **/
std::string defaultConfigFolderPath, /**< folder to find default config files in **/
std::string & status /**< output status of config parse **/
);
In client function:
GatewayWrapper::Config bridge;
std::string configPath("./config");
std::string defaultPath("./config/default");
GatewayWrapper::Config gwtransport;
bridge.aye = "bridged.dll";
bridge.bee = "1.0";
bridge.sea = "";
configs.push_back(bridge);
_bridge = GatewayWrapper::Bridge_GetConfiguredDefaultsImplementationPointer(configs, configPath, defaultPath, status);
Note that call to library that is crashing is in the same scope as the vector declaration, struct declaration, string assignment and vector push-back
There are no threading calls in this section of code, but there are other threads running doing other things. There is no pointer math here, there are no heap allocations in the area except perhaps inside the standard library.
I can run the code up to the Bridge_GetConfiguredDefaultsImplementationPointer call in the debugger, and the contents of the configs vector look correct in the debugger.
Back in the library, in the first sub function, where the debugger don't shine, I've broken down the failing statement into several console prints.
System::String^ temp
List<CConfig^>^ configs = gcnew List<CConfig ^>((INT32)newConfigs.size());
for( int i = 0; i< newConfigs.size(); i++)
{
std::cout << newConfigs[i].aye<< std::flush; // prints
std::cout << newConfigs[i].aye.c_str() << std::flush; // prints
temp = gcnew System::String(newConfigs[i].aye.c_str());
System::Console::WriteLine(temp); // prints
std::cout << "Testing string creation" << std::endl; // prints
std::cout << newConfigs[i].bee << std::flush; // crashes here
}
I get the same exception on access of bee if I move the newConfigs[i].bee out above the assignment of temp or comment out the list declaration/assignment.
Just for reference that the std::string in a struct in a vector should have arrived at it's destination ok
Is std::vector copying the objects with a push_back?
std::string in struct - Copy/assignment issues?
http://www.cplusplus.com/reference/vector/vector/operator=/
Assign one struct to another in C
Why this exception is not caught by my try/catch
https://stackoverflow.com/a/918891/2091951
Generic AccessViolationException related questions
How to handle AccessViolationException
Programs randomly getting System.AccessViolationException
https://connect.microsoft.com/VisualStudio/feedback/details/819552/visual-studio-debugger-throws-accessviolationexception
finding the cause of System.AccessViolationException
https://msdn.microsoft.com/en-us/library/ms164911.aspx
Catching access violation exceptions?
AccessViolationException when using C++ DLL from C#
Suggestions in above questions
Change to .net 3.5, change target platform - these solutions could have serious issues with a large mult-project solution.
HandleProcessCorruptedStateExceptions - does not work in C++, this decoration is for C#, catching this error could be a very bad idea anyway
Change legacyCorruptedStateExceptionsPolicy - this is about catching the error, not preventing it
Install .NET 4.5.2 - can't, already have 4.6.1. Installing 4.6.2 did not help. Recompiling on a different machine that didn't have 4.5 or 4.6 installed did not help. (Despite this used to compile and run on my machine before installing Visual Studio 2013, which strongly suggests the .NET library is an issue?)
VSDebug_DisableManagedReturnValue - I only see this mentioned in relation to a specific crash in the debugger, and the help from Microsoft says that other AccessViolationException issues are likely unrelated. (http://connect.microsoft.com/VisualStudio/feedbackdetail/view/819552/visual-studio-debugger-throws-accessviolationexception)
Change Comodo Firewall settings - I don't use this software
Change all the code to managed memory - Not an option. The overall design of calling C# from C++ through C++/CLI is resistant to change. I was specifically asked to design it this way to leverage existing C# code from existing C++ code.
Make sure memory is allocated - memory should be allocated on the stack in the C++ client. I've attempted to make the vector be not a reference parameter, to force a vector copy into explicitly library controlled memory space, did not help.
"Access violations in unmanaged code that bubble up to managed code are always wrapped in an AccessViolationException." - Fact, not a solution.
but it was the mismatch, not the specific version that was the problem
Yes, that's black letter law in VS. You unfortunately just missed the counter-measures that were built into VS2012 to turn this mistake into a diagnosable linker error. Previously (and in VS2010), the CRT would allocate its own heap with HeapAlloc(). Now (in VS2013), it uses the default process heap, the one returned by the GetProcessHeap().
Which is in itself enough to trigger an AVE when you run your app on Vista or higher, allocating memory from one heap and releasing it from another triggers an AVE at runtime, a debugger break when you debug with the Debug Heap enabled.
This is not where it ends, another significant issue is that the std::string object layout is not the same between the versions. Something you can discover with a little test program:
#include <string>
#include <iostream>
int main()
{
std::cout << sizeof(std::string) << std::endl;
return 0;
}
VS2010 Debug : 32
VS2010 Release : 28
VS2013 Debug : 28
VS2013 Release : 24
I have a vague memory of Stephen Lavavej mentioning the std::string object size reduction, very much presented as a feature, but I can't find it back. The extra 4 bytes in the Debug build is caused by the iterator debugging feature, it can be disabled with _HAS_ITERATOR_DEBUGGING=0 in the Preprocessor Definitions. Not a feature you'd quickly want to throw away but it makes mixing Debug and Release builds of the EXE and its DLLs quite lethal.
Needless to say, the different object sizes seriously bytes when the Config object is created in a DLL built with one version of the standard C++ library and used in another. Many mishaps, the most basic one is that the code will simply read the Config::bee member from the wrong offset. An AVE is (almost) guaranteed. Lots more misery when code allocates the small flavor of the Config object but writes the large flavor of std::string, that randomly corrupts the heap or the stack frame.
Don't mix.
I believe 2013 introduced a lot of changes in the internal data formats of STL containers, as part of a push to reduce memory usage and improve perf. I know vector became smaller, and string is basically a glorified vector<char>.
Microsoft acknowledges the incompatibility:
"To enable new optimizations and debugging checks, the Visual Studio
implementation of the C++ Standard Library intentionally breaks binary
compatibility from one version to the next. Therefore, when the C++
Standard Library is used, object files and static libraries that are
compiled by using different versions can't be mixed in one binary (EXE
or DLL), and C++ Standard Library objects can't be passed between
binaries that are compiled by using different versions."
If you're going to pass std::* objects between executables and/or DLLs, you absolutely must ensure that they are using the same version of the compiler. It would be well-advised to have your client and its DLLs negotiate in some way at startup, comparing any available versions (e.g. compiler version + flags, boost version, directx version, etc.) so that you catch errors like this quickly. Think of it as a cross-module assert.
If you want to confirm that this is the issue, you could pick a few of the data structures you're passing back and forth and check their sizes in the client vs. the DLLs. I suspect your Config class above would register differently in one of the fail cases.
I'd also like to mention that it is probably a bad idea in the first place to use smart containers in DLL calls. Unless you can guarantee that the app and DLL won't try to free or reallocate the internal buffers of the other's containers, you could easily run into heap corruption issues, since the app and DLL each have their own internal C++ heap. I think that behavior is considered undefined at best. Even passing const& arguments could still result in reallocation in rare cases, since const doesn't prevent a compiler from diddling with mutable internals.
You seem to have memory corruption. Microsoft Application Verifier is invaluable in finding corruption. Use it to find your bug:
Install it to your dev machine.
Add your exe to it.
Only select Basics\Heaps.
Press Save. It doesn't matter if you keep application verifier open.
Run your program a few times.
If it crashes, debug it and this time, the crash will point to your problem, not just some random location in your program.
PS: It's a great idea to have Application Verifier enabled at all times for your development project.
I'm seeing something weird happening with Visual Studio 2015 Community. Code that worked perfectly in VS2012 crashes at startup when ported to VS2015, before main is invoked: the classic symptoms of some static initialization mess. I have a few static variables, but used properly, with the standard "Construct On First Use" pattern, e.g.:
const int& i()
{
static int *v = new int(1);
return *v;
}
The code above causes an assert from the runtime while initializing the program (see image http://i.stack.imgur.com/nsQGS.png). Pressing retry simply quits the program: no call stack, no information whatsoever.
The code below however works flawlessly:
const int& i()
{
static int *v = nullptr;
if (v == nullptr)
v = new int(1);
return *v;
}
It seems to me that VS2015 is doing what looks like some (illegal) optimization (even in a debug build), by executing the static variable initialization code during program initialization, and not the first time that the function is called, as the C++ standard requires.
I tried several variations of the code above: class methods, free functions, different objects (std::vector, cv::Mat), always with the same result: the static pointer has to be initialized to null or the program doesn't start.
So... am I missing something or is VS2015 actually messing up badly?
UPDATE:
I spent some time trying to find the minimal setup that shows the problem, and this is what I got:
create a project using the "CLR empty project" template.
add a cpp file, containing just the offending function and main(). Main can be empty, it doesn't matter: the bug occurs BEFORE it is even reached.
add 'main' as entry point (or you get a link error).
The x86 version works, but the x64 doesn't.
For comparison, a project with the identical code but created as "Win32 console application" doesn't have the problem, even after adding the /CLR option. I can see differences in the vcxproj files, but none justifies the error, although one or more of them clearly IS the cause.
Is there a way to upload a zip with the project?
Well, #bogdan got it right. My project had a mix of settings that was neither /SUBSYSTEM:CONSOLE nor /SUBSYSTEM:WINDOWS. Once I fixed that, everything started to work as expected. My test projects had the same problem, I blame Microsoft for not having a clear "CLR windows app" C++ template any more in VS2015 (they want to push you to use C# for that, which makes sense most of the times, but not always).
I found this page particularly useful in explaining all the different settings that have to be consistent (it's not just /SUBSYSTEM...).
I wish I could mark #bogdan's comment as answer, but I can't see anything to do that.
Thanks Bogdan!
I'm having a very strange problem when trying to run RapidXml 1.13 under Windows CE 6.0 compiled with Visual Studio 2005. I have an extremely small program that fails to run:
#include <rapidxml.hpp>
using namespace rapidxml;
int _tmain(int argc, _TCHAR* argv[])
{
xml_document<char> doc;
return 0;
}
It compiles fine with 0 errors and 0 warnings (at W3). However, when I run or debug the program, I get an access violation exception:
First-chance exception at 0x000110d4 in RapidXml_Test.exe:
0xC0000005: Access violation writing location 0x0001fb48.
The debugger then points to this line (1366 in rapidxml.hpp) as the culprit (the open brace):
template<class Ch = char>
class xml_document: public xml_node<Ch>, public memory_pool<Ch>
{
public:
//! Constructs empty XML document
xml_document()
: xml_node<Ch>(node_document)
------->{
}
...
If anyone has any clue what the problem could be I'd greatly appreciate it. I have much more complicated code working in my build and run-time environment so I don't suspect anything there. I'm also fairly confident it's not a project setting. I assume at this point that RapidXml's use of templates is somehow confusing the Windows CE VC++ compiler. I don't know what else it could be.
Thanks in advance!
I found the solution. RapidXML allocates its own pool of memory once its loaded. Problem is, I think it allocates it on the stack and I was getting a stack overflow! (How serendipitous that the problem with my first question here actually WAS a stack overflow). Anyways, reducing the size of the pool solved my problem.