After some days of test I figured out that the runtime patching mechanism patch_functions.cc is not safe to use in a production environment.
It seems to work well in a VS2010 project except for HeapAlloc() and HeapFree() but cannot be used in a VS2015 project due to some unresolved problems Open Issues.
the windows readme describes this alternative way to use tcmalloc:
An alternative to all the above is to statically link your application
with libc, and then replace its malloc with tcmalloc. This allows you
to just build and link your program normally; the tcmalloc support
comes in a post-processing step. This is more reliable than the above
technique (which depends on run-time patching, which is inherently
fragile), though more work to set up. For details, see
https://groups.google.com/group/google-perftools/browse_thread/thread/41cd3710af85e57b
Unfortunately the provided lik is urechable, seems that google had closed the group.
Could someone explain me how to do this?
I assume it suggests to write your own malloc which uses tcmalloc.
So you have to define and link your own one (by creating or using an .c aka translation unit) and write something like this
#ifdef __cplusplus
extern "C" {
#endif
#include <stdlib.h>
void* malloc(size_t size) {
return tcmalloc(size);
}
//Also define a free if memory which has been allocated by tcmalloc
//needs to be freed by a special function
// Like
/*
void free(void* ptr) {
if (ptr) {
tcfree(ptr);
}
}
*/
#ifdef __cplusplus
}
#endif
Problem is: Depending on your building system or linker, it may nag about double symbols aka references. Then you have to somehow exclude libcs malloc, or change the libc by yourself.
I stumbled on this as well and believe I found a way to have it working.
First, in windows\config.h, you have to replace
#undef WIN32_OVERRIDE_ALLOCATORS
by
#define WIN32_OVERRIDE_ALLOCATORS
Then, and it's the most important thing, you have to make sure of two things:
windows\patch_functions.cc is not compiled and linked
windows\overridde_functions.cc is compiled and linked
At first, I omitted step 2 and got a barely functioning DLL where some memory allocations would get freed and overridden apparently at random.
In my case, making sure of both steps was just a matter of ensuring only windows\override_functions.cc is included in my libtcmalloc VS2017 project.
Related
I have a library of code I'm working on upgrading from x86 to x64 for a Windows application.
Part of the code took advantage of MSVC inline assembly blocks. I'm not looking to go through and interpret the assembly but I am looking to keep functionality from this part of the application.
Can I compile the functions using the inline assembly using GCC to make a DLL and link that to the rest of the library?
EDIT 1:(7/7/21) The flexibility with which compiler the project uses is open and I am currently looking into using Clang for use with MSVC.(also the Intel C++ compiler as another possibility) As stated in the first sentence it is a Windows application that I want to keep on Windows and the purpose of using another compiler is due to me 1.) not wanting to rewrite the large amount of assembly and 2.) because I know that MSVC does not support x64 inline assembly. So far clang seems to be working with a couple issues of how it declares comments inside of the assembly block and a few commands. The function is built around doing mathematical operations on a block of data, in what was supposed to be as fast as possible when it was developed but now that it works as intended I'm not looking to upgrade just maintain functionality. So, any compiler that will support inline assembly is an option.
EDIT 2:(7/7/21) I forgot to mention in the first edit, I'm not necessarily looking to load the 32-bit DLL into another process because I'm worried about copying data into an out of shared memory. I've done a similar solution for another project but the data set is around 8 MB and I'm worried that slow copy times for the function would cause the time constraint on the math to cause issues in the runtime of the application.(slow, laggy, and buffering are effects I'm trying to avoid.) I'm not trying to make it any faster but it definitely can't get any slower.
In theory, if you manage to create a plain C interface for that DLL (all exported symbols from DLL are standard C functions) and don't use memory management functions across "border" (no mixed memory management) then you should be able to dynamically load that DLL from another another (MSVC) process and call its functions, at least.
Not sure about statically linking against it... probably not, because the compiler and linker must go hand in hand (MSVC compiler+MSVC linker or GCC compiler+GCC linker) . The output of GCC linker is probably not compatible with MSVC at least regarding name mangling.
Here is how I would structure it (without small details):
Header.h (separate header to be included in both DLL and EXE)
//... remember to use your preferred calling convention but be consistent about it
struc Interface{
void (*func0)();
void (*func1)(int);
//...
};
typedef Interface* (*GetInterface)();
DLL (gcc)
#include "Header.h"
//functions implementing specific functionality (not exported)
void f0)(){/*...*/}
void f1)(int){/*...*/}
//...
Interface* getInterface(){//this must be exported from DLL (compiler specific)
static Interface interface;
//initialize functions pointers from interface with corresponding functions
interface.func0 = &f0;
interface.func1 = &f1;
//...
return &interface;
}
EXE (MSVC)
#include "Header.h"
int main(){
auto dll = LoadLibrary("DLL.dll");
auto getDllInterface = (GetInstance)GetProcAddress(dll, "getInterface");
auto* dllInterface = getDllInterface();
dllInterface->func0();
dllInterface->func1(123);
//...
return 0;
}
I am trying for two weeks to create a DLL to which I can pass strings and get back strings. But still no success.
I tried this on Dev-C++(TDM-GCC 4.9.2) and visual studio community 2015. I searched a lot about this and tried almost every sample code I found but I have no success.
I have to use this DLL with MetaTrader Terminal 4.
Here is a one sample code, which I used. This code compiles successfully but when I send a string to this, from MT4, I get an access violation error.
#ifndef MYLIB_HPP
#define MYLIB_HPP
#include <string>
#ifdef MYLIB_EXPORTS
#define MYLIB_API __declspec(dllimport)
#else
#define MYLIB_API __declspec(dllexport)
#endif
bool MYLIB_API test(const std::string& str);
#endif
bool MYLIB_API MyTest(const std::string& str)
{
return (str == "Hi There");
}
If you do share a C++ string between a DLL and another executable, both need to have been compiled with the same tool-chain. This is because std::string is defined in header only. So, if the DLL and executable use different string headers, they may well be binary incompatible.
If you want to make sure that things do work with different tool-chains, stick to NULL terminated C strings.
You have just experienced one of the MQL4 tricks,the MQL4 string is not a string but a struct thus #import on MQL4 side will make MT4 to inject that, not matching your DLL C-side expectations and the access-violation error is straightforward, as your C-side code tried to access the MQL4 territories...
First rule to design API/DLL: READ the documentation very carefully.
Yes, one may object, that the MQL4 doc is somewhat tricky to follow, but thus more double the Rule#1, read the documentation very, very, very carefully as some important design facts are noted almost hidden in some not very predictable chapters or somewhere in explanations of ENUM tables, compiler directives, pragma-s side-notes et al.
Second rule: design API/DLL interface so as to allow smooth integration
MQL4 has changed the rules somewhere about Build 670+. Good news is, the MetaQuotes has announced, that there will be no further investment on their side into MT4 further developlments, so the MT4-side of the DLL/API integration will hopefully stop from further creeping.
Given your statement, that you design the DLL/API, try to design future-proof specification -- use block of uchar[]-s instead of "interpretations"-sensitive string, pass both inputs and outputs by-reference and return just some form of int aReturnCODE = myDLL_FUNC( byRefParA, byRefParB, byRefRESULT ); and your efforts will result in clean code, better portability among 3rd party language-wrappers and will also minimise your further maintenance costs.
Most likely, your code and the one you're linking against have been compiled with a different ABI for std::string, i.e. the string used by the library has a different memory layout (and sizeof) than the one you're compiling with.
I once ran into this problem when linking against the hdf5 library and using gcc. In this case, the problem could be solved by reverting to a previous ABI, as explained here.
However, the problem also occurred with clang, when such a solution was not available. Thus, to make this all working I had to avoid using std::string in any calls to the library (hdf5 in my case) that was compiled with the different ABI, and instead make do with the hdf5 interface using const char*.
I have the problem with passing by reference std::string to function in dll.
This is function call:
CAFC AFCArchive;
std::string sSSS = std::string("data\\gtasa.afc");
AFCER_PRINT_RET(AFCArchive.OpenArchive(sSSS.c_str()));
//AFCER_PRINT_RET(AFCArchive.OpenArchive(sSSS));
//AFCER_PRINT_RET(AFCArchive.OpenArchive("data\\gtasa.afc"));
This is function header:
#define AFCLIBDLL_API __declspec(dllimport)
AFCLIBDLL_API EAFCErrors CAFC::OpenArchive(std::string const &_sFileName);
I try to debug pass-by-step through calling the function and look at _sFileName value inside function.
_sFileName in function sets any value(for example, t4gs..\n\t).
I try to detect any heap corruption, but compiler says, that there is no error.
DLL has been compiled in debug settings. .exe programm compiled in debug too.
What's wrong?? Help..!
P.S. I used Visual Studio 2013. WinApp.
EDIT
I have change header of func to this code:
AFCLIBDLL_API EAFCErrors CAFC::CreateArchive(char const *const _pArchiveName)
{
std::string _sArchiveName(_pArchiveName);
...
I really don't know, how to fix this bug...
About heap: it is allocated in virtual memory of our process, right? In this case, shared virtual memory is common.
The issue has little to do with STL, and everything to do with passing objects across application boundaries.
1) The DLL and the EXE must be compiled with the same project settings. You must do this so that the struct alignment and packing are the same, the members and member functions do not have different behavior, and even more subtle, the low-level implementation of a reference and reference parameters is exactly the same.
2) The DLL and the EXE must use the same runtime heap. To do this, you must use the DLL version of the runtime library.
You would have encountered the same problem if you created a class that does similar things (in terms of memory management) as std::string.
Probably the reason for the memory corruption is that the object in question (std::string in this case) allocates and manages dynamically allocated memory. If the application uses one heap, and the DLL uses another heap, how is that going to work if you instantiated the std::string in say, the DLL, but the application is resizing the string (meaning a memory allocation could occur)?
C++ classes like std::string can be used across module boundaries, but doing so places significant constraints on the modules. Simply put, both modules must use the same instance of the runtime.
So, for instance, if you compile one module with VS2013, then you must do so for the other module. What's more, you must link to the dynamic runtime rather than statically linking the runtime. The latter results in distinct runtime instances in each module.
And it looks like you are exporting member functions. That also requires a common shared runtime. And you should use __declspec(dllexport) on the entire class rather than individual members.
If you control both modules, then it is easy enough to meet these requirements. If you wish to let other parties produce one or other of the modules, then you are imposing a significant constraint on those other parties. If that is a problem, then consider using more portable interop. For example, instead of std::string use const char*.
Now, it's possible that you are already using a single shared instance of the dynamic runtime. In which case the error will be more prosaic. Perhaps the calling conventions do not match. Given the sparse level of detail in your question, it's hard to say anything with certainty.
I encountered similar problem.
I resolved it synchronizing Configuration Properties -> C / C++ settings.
If you want debug mode:
Set _DEBUG definition in Preprocessor Definitions in both projects.
Set /MDd in Code Generation -> Runtime Library in both projects.
If you want release mode:
Remove _DEBUG definition in Preprocessor Definitions in both projects.
Set /MD in Code Generation -> Runtime Library in both projects.
Both projects I mean exe and dll project.
It works for me especially if I don't want to change any settings of dll but only adjust to them.
C++ class constructor can be inlined or not be inlined. However, I found a strange situation where only inline class constructor can avoid Visual Studio memory crash. The example is as follows:
dll.h
class _declspec(dllexport) Image
{
public:
Image();
virtual ~Image();
};
class _declspec(dllexport) Testimage:public Image
{
public:
Testimage();
virtual ~Testimage();
};
typedef std::auto_ptr<Testimage> TestimagePtr;
dll.cpp
#include "dll.h"
#include <assert.h>
Image::~Image()
{
std::cout<<"Image is being deleted."<<std::endl;
}
Image::Image()
{
}
Testimage::Testimage()
{
}
Testimage::~Testimage()
{
std::cout<<"Geoimage is being deleted."<<std::endl;
}
The dll library is compiled as a dynamic library, and it is statically linked to the C++ runtime library (Multi-threaded Debug (/MTd)). The executable program that runs the library is as follows:
int main()
{
TestimagePtr my_img(new Testimage());
return 0;
}
The executable program will invoke the dll library and it also statically links the runtime library. The problem I have is that when running the executable program the following error message appears:
However, when the class constructor in dll is inlined as the following codes show:
class _declspec(dllexport) Image
{
public:
Image();
virtual ~Image();
};
class _declspec(dllexport) Testimage:public Image
{
public:
Testimage()
{
}
virtual ~Testimage();
};
The crash will disappear. Could someone explain the reason behind? Thanks! By the way, I am using VC2010.
EDIT: The following situation also trigger the same crash
.
Situation 1
int main()
{
//TestimagePtr my_img(new Testimage());
Testimage *p_img;
p_img = new Testimage();
delete p_img;
return 0;
}
it is statically linked to the C++ runtime library (Multi-threaded Debug (/MTd)
This is a very problematic scenario in versions of Visual Studio prior to VS2012. The issue is that you have more than one version of the CRT loaded in your process. One used by your EXE, another used by the DLL. This can cause many subtle problems, and not so subtle problems like this crash.
The CRT has global state, stuff like errno and strtok() cannot work properly when that global state is updated by one copy of the CRT and read back by another copy. Relevant to your crash, a hidden global state variable is the heap that the CRT uses to allocate memory from. Functions like malloc() and ::operator new use that heap.
This goes wrong when objects are allocated by one copy of the CRT and released by another. The pointer that's passed to free() or ::operator delete belongs to the wrong heap. What happens next depends on your operating system. A silent memory leak in XP. In Vista and up, you program runs with the debug version of the memory manager enabled. Which triggers a breakpoint when you have a debugger attached to your process to tell you that there's a problem with the pointer. The dialog in your screenshot is the result. It isn't otherwise very clear to me how inlining the constructor could make a difference, the fundamental issue however is that your code invokes undefined behavior. Which has a knack for producing random outcomes.
There are two approaches available to solve this problem. The first one is the simple one, just build both your EXE and your DLL project with the /MD compile option instead. This selects the DLL version of the CRT. It is now shared by both modules and you'll only have a single copy of the CRT in your process. So there is no longer a problem with having one module allocating and another module releasing memory, the same heap is used.
This will work fine to solve your problem but can still become an issue later. A DLL tends to live a life of its own and may some day be used by another EXE that was built with a different version of the CRT. The CRT will now again not be shared since they'll use different versions of the DLL, invoking the exact same failure mode you are seeing today.
The only way to guarantee that this cannot happen is to design your DLL interface carefully. And ensure that there will never be a case where the DLL allocates memory that the client code needs to release. That requires giving up on a lot of C++ goodies. You for example can never write a function that returns a C++ object, like std::string. And you can never allow an exception to cross the module boundary. You are basically down to a C-style interface. Note how COM addresses this problem by using interface-based programming techniques and a class factory plus reference counting to solve the memory management problem.
VS2012 has a counter-measure against this problem, it has a CRT version that allocates from the default process heap. Which solves this particular problem, not otherwise a workaround for the global state issue for other runtime functions. And adds some new problems, a DLL compiled with /MT that gets unloaded that doesn't release all of its allocations now causes an unpluggable leak for example.
This is an ugly problem in C++, the language fundamentally misses an ABI specification that addresses problems like this. The notion of modules is entirely missing from the language specification. Being worked on today but not yet completed. Not simple to do, it is solved in other languages like Java and the .NET languages by specifying a virtual machine, providing a runtime environment where memory management is centralized. Not the kind of runtime environment that excites C++ programmers.
I tried to reproduce your problem in VC2010 and it doesn't crash. It works with a constructor inline or not. Your problem is probably not in what you write here.
Your project is too hard to open as it seams to have its file pathes set in absolute, probably because generated with CMake. (So the files are not found by the compiler).
The problem I see in your code is that you declare the exported classes with _declspec(dllexport) directly written.
You should have a #Define to do this, and the value should be _declspec(dllimport) when read from the exe compilation. Maybe the problem comes from that.
This is driving me nuts. I am using some 3rd-party code in a Windows .lib that, in debug mode, is causing an error similar to the following:
Run-Time Check Failure #2 - Stack around the variable 'foo' was corrupted.
The error is thrown when either the object goes out of scope or is deleted. Simply allocating one of these objects and then deleting it will throw the error. I therefore think the problem is either in one of the many constructors/destructors but despite stepping through every line of code I cannot find the problem.
However, this only happens when creating one of these objects in a static library. If I create one in my EXE application, the error does not appear. The 3rd-party code itself lives in a static lib. For example, this fails:
**3RDPARTY.LIB**
class Foo : public Base
{
...
};
**MY.LIB**
void Test()
{
Foo* foo = new Foo;
delete foo; // CRASH!
}
**MY.EXE**
void Func()
{
Test();
}
But this will work:
**3RDPARTY.LIB**
class Foo : public Base
{
...
};
**MY.EXE**
void Func()
{
Foo* foo = new Foo;
delete foo; // NO ERROR
}
So, cutting out the 'middle' .lib file makes the problem go away and it is this weridness that is driving me mad. The EXE and 2 libs all use the same CRT library. There are no errors linking. The 3rd-party code uses inheritance and there are 5 base classes. I've commented out as much code as I can whilst still getting it to build and I just can't see what's up.
So if anyone knows why code in a .lib would act differently to the same code in a .exe, I would love to hear it. Ditto any tips for tracking down memory overwrites! I am using Visual Studio 2008.
One possibility is that it's a calling convention mismatch - make sure that your libraries and executables are all set to use the same default calling convention (usually __cdecl). To set that, open up your project properties and go to Configuration Properties > C/C++ > Advanced and look at the Calling Convention option. If you call a function with the wrong calling convention, you'll completely mess up the stack.
OK, I tracked the problem down and it's a cracker, if anyone's interested. Basically, my .LIB, which exhibited the problem. had defined _WIN32_WINNT as 0x0501 (Windows 2000 and greater), but my EXE and the 3rd-party LIB had it defined as 0x0600 (Vista). Now, one of the headers included by the 3rd-party lib is sspi.h which defines a structure called SecurityFunctionTable which includes the following snippet:
#if OSVER(NTDDI_VERSION) > NTDDI_WIN2K
// Fields below this are available in OSes after w2k
SET_CONTEXT_ATTRIBUTES_FN_W SetContextAttributesW;
#endif // greater thean 2K
Th cut a long story short, this meant a mismatch in object sizes between the LIBs and this was causing the Run-Time Check Failure.
Class!
Is your .lib file linked against the library's .lib? I assume from your example that you are including the header with the declaration of the destructor; without it, deleting such a type is allowed but can result in UB (in a bizarre manner contrary to the general rule that something must be defined before used). If the .lib files aren't linked together, it's possible that a custom operator delete or destructor is having some weird linking issues, and while that shouldn't happen, you never can quite tell if it won't.
Without seeing more code, it's hard to give you a firm answer. However, for tracking down memory overwrites, I recommend using WinDbg (free from Microsoft, search for "Debugging Tools for Windows").
When you have it attached to your process, you can have it set breakpoints for memory access (read, write, or execute). It's really powerful overall, but it should especially help you with this.
The error is thrown when either the object goes out of scope or is deleted.
Whenever I've run into this it had to do with the compiled library using a different version of the C++ runtime than the rest of the application.