I am developing on SAM3X8E, I am using the GCC C++ compiler and atmel studio 7. I am trying to include the stdio library and I keep getting the error:
"undefined reference to end" in sbrk
following this post:
https://community.atmel.com/forum/how-solve-linker-error-sbrkc11-error-undefined-reference-end
I explicitly added end to the heap section of the linker script in the heap section via,
end = sheap;
I'm not sure what's going on, sbrk should see the linker variable.
I solved this problem.
Add _sbrk function.
extern int _sheap;
caddr_t _sbrk( int incr )
{
static unsigned char *heap = NULL;
unsigned char *prev_heap;
if (heap == NULL) {
heap = (unsigned char *) &(_sheap);
}
prev_heap = heap;
heap += incr;
return (caddr_t) prev_heap;
}
Related
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 6 months ago.
Improve this question
What's the problem of my memory management? Because it causes crash that I show in a comment in the code below ("A memory block could not be found when trying to free."). I know that my memory management is not thread safe because I use global variables g_numBlocks and g_blocks that can cause risk when using multiple threads.
Since my memory management code seems too complex, can anyone suggest a stable and better "Memory Management for C++" to avoid memory leaks.
The code that contains bug
#include "emc-memory.h" // <-- Declare the functions MALLOC() and FREE() from other library.
#include <vector>
int main() {
printf("HERE(1)\n");
{
std::vector<string> paths = { // <-- Problem, 'std::vector' & 'string' use internal malloc/free & operator new/delete that are overwritten with my own custom memory management.
"/foo/bar.txt",
"/foo/bar.",
"/foo/bar",
"/foo/bar.txt/bar.cc",
"/foo/bar.txt/bar.",
"/foo/bar.txt/bar",
"/foo/.",
"/foo/..",
"/foo/.hidden",
"/foo/..bar",
};
} // <-- It crashes here, error in FREE(): "A memory block could not be found when trying to free.".
printf("HERE(2)\n"); // The reason I know it crashes above is this line is not evaluated, only "HERE(1)" is printed. I'm using [RelWithDebInfo] with blurry debugging info.
return 0;
}
Compilers:
[Visual Studio 2015] [Debug]: No problem.
[Visual Studio 2015] [RelWithDebInfo]: No problem.
[GCC 12.1.0 x86_64-w64-mingw32] [Debug]: No problem.
[GCC 12.1.0 x86_64-w64-mingw32] [RelWithDebInfo]: Broken which means there's a bug.
In "emc-memory.h" in other library .so file
extern const char* __file;
extern int __line;
#define new (__file = __FILE__, __line = __LINE__, 0) ? 0 : new
enum MEMORYBLOCKTYPE {
MEMORYBLOCKTYPE_MALLOC,
MEMORYBLOCKTYPE_NEW,
};
void *MALLOC(size_t size, MEMORYBLOCKTYPE type);
void *REALLOC(void *block, size_t newSize);
void FREE(void *block, MEMORYBLOCKTYPE type);
#define malloc(size) ((__file = __FILE__, __line = __LINE__, 0) ? 0 : MALLOC(size, MEMORYBLOCKTYPE_MALLOC))
#define realloc(block, newSize) REALLOC(block, newSize)
#define free(block) FREE(block, MEMORYBLOCKTYPE_MALLOC)
In "emc-memory.cpp" in other library .so file
I use this code in a link to override the operator new & delete: https://codereview.stackexchange.com/questions/7216/custom-operator-new-and-operator-delete
typedef unsigned long long BlockId; // The reason it's 64-bit is a memory block can be freed and reallocated multiple times, which means that there can be a lot of ids.
BlockId g_blockId = 0;
BlockId newBlockId() {
return g_blockId++;
}
struct Block {
const char *file;
int line;
const char *scope;
char *hint;
size_t size;
BlockId id; // That id is used for comparison because it will never be changed but the block pointer can be changed.
void *block;
MEMORYBLOCKTYPE type;
};
bool g_blocks_initialized = false;
int g_numBlocks;
Block **g_blocks;
void *MALLOC(size_t size, MEMORYBLOCKTYPE type) {
if (g_blocks_initialized == false) {
g_blocks_initialized = true;
_initializeList(g_numBlocks, g_blocks);
}
Block *b = (Block *)malloc(sizeof(*b));
b->file = __file ; __file = nullptr;
b->line = __line ; __line = 0;
b->scope = __scope; __scope = nullptr;
b->hint = allocateMemoryHint(__hint);
b->size = size;
b->id = newBlockId();
b->block = malloc(size);
b->type = type;
_addListItem(g_numBlocks, g_blocks, b);
return b->block;
}
void FREE(void *block, MEMORYBLOCKTYPE type) {
if (block == nullptr) {
return; // 'free' can free a nullptr.
}
for (int i = 0; i < g_numBlocks; i++) {
Block *b = g_blocks[i];
if (b->block == block) {
if (b->type != type) {
switch (type) {
case MEMORYBLOCKTYPE_MALLOC: EMC_ERROR("The memory block type must be MALLOC."); break;
case MEMORYBLOCKTYPE_NEW: EMC_ERROR("The memory block type must be NEW."); break;
default: EMC_ERROR("Error"); break;
}
}
_removeListItem(g_numBlocks, g_blocks, b);
freeMemoryHint(b->hint); b->hint = nullptr;
SAFE_FREE(b->block);
SAFE_FREE(b);
return;
}
}
EMC_ERROR("A memory block could not be found when trying to free.\n\nExamples:\n - Calling free(pointer) where pointer was not set to zero after it's been called twice, the solution was to use SAFE_FREE(). And if possible, replace any free() with SAFE_FREE(). For example, see Lexer::read0() on the original line \"free(out.asIdentifier);\".\n - If an 'Engine' object is destroyed before destroying a Vulkan object then it can cause this error (It can happen with 'Release' or 'RelWithDebInfo' configuration but not with 'Debug' configuration), that problem happened to me and I stuck there for hours until I realized it.");
}
I would humbly suggest that without a very clear reason to think otherwise the best memory management for GCC C++ is the out-of-the-box default memory management for GCC C++.
That would mean your best solution would have been to do nothing or as it is now strip out your overrides of the global operators.
You may find in some area of a system the default memory management is sub-optimal but in 2022 the default options are very effective and if you find a general purpose strategy that is better it's a publishable paper.
However your question tells us nothing about the application in question or your motivations for thinking you should even try to change the memory management let alone give advice on what to.
Sure you can add a global allocation mutex to block memory management and make it thread-safe. I will be surprised if that doesn't turn out to more than throw away whatever advantage you're hoping to gain.
Not sure where SAFE_FREE is coming from.
If you see in MALLOC function, they use c runtime malloc. Meaning that if you want to free the block you need to use the corresponding free() function.
Make sure that SAFE_FREE is indeed using the c runtime free with the correct parameters.
I have previously, here, been shown that C++ functions aren't easily represented in assembly. Now I am interested in reading them one way or another because Callgrind, part of Valgrind, show them demangled while in assembly they are shown mangled.
So I would like to either mangle the Valgrind function output or demangle the assembly names of functions. Anyone ever tried something like that? I was looking at a website and found out the following:
Code to implement demangling is part of the GNU Binutils package;
see libiberty/cplus-dem.c and include/demangle.h.
Has anyone ever tried something like that? I want to demangle/mangle in C.
My compiler is gcc 4.x.
Use the c++filt command line tool to demangle the name.
Here is my C++11 implementation, derived from the following page:
http://gcc.gnu.org/onlinedocs/libstdc++/manual/ext_demangling.html
#include <cxxabi.h> // needed for abi::__cxa_demangle
std::shared_ptr<char> cppDemangle(const char *abiName)
{
int status;
char *ret = abi::__cxa_demangle(abiName, 0, 0, &status);
/* NOTE: must free() the returned char when done with it! */
std::shared_ptr<char> retval;
retval.reset( (char *)ret, [](char *mem) { if (mem) free((void*)mem); } );
return retval;
}
To make the memory management easy on the returned (char *), I'm using a std::shared_ptr with a custom lambda 'deleter' function that calls free() on the returned memory. Because of this, I don't ever have to worry about deleting the memory on my own, I just use it as needed, and when the shared_ptr goes out of scope, the memory will be free'd.
Here's the macro I use to access the demangled type name as a (const char *). Note that you must have RTTI turned on to have access to 'typeid'
#define CLASS_NAME(somePointer) ((const char *) cppDemangle(typeid(*somePointer).name()).get() )
So, from within a C++ class I can say:
printf("I am inside of a %s\n",CLASS_NAME(this));
This is a slight variation on Dave's version above. This is a unique_ptr version with a little bit of checking on the return type, though it looks like you could just ignore that, but somehow that just seems unclean.
auto cppDemangle (const char *abiName)
{
//
// This function allocates and returns storage in ret
//
int status;
char *ret = abi::__cxa_demangle(abiName, 0 /* output buffer */, 0 /* length */, &status);
auto deallocator = ( [](char *mem) { if (mem) free((void*)mem); } );
if (status) {
// 0: The demangling operation succeeded.
// -1: A memory allocation failure occurred.
// -2: mangled_name is not a valid name under the C++ ABI mangling rules.
// -3: One of the arguments is invalid.
std::unique_ptr<char, decltype(deallocator) > retval(nullptr, deallocator);
return retval;
}
//
// Create a unique pointer to take ownership of the returned string so it
// is freed when that pointers goes out of scope
//
std::unique_ptr<char, decltype(deallocator) > retval(ret, deallocator);
return retval;
}
I'm trying to migrate an old project from Borland BCC 5 to Qt 5.6.3 with Microsoft MSVC 2015 64bit compiler.
The following code is a model of original work. It works as expected in Visual Studio 2015 and Borland BCC 5, However, when I try to run it in Qt 5.6.3 with Qt Creator 3.6.1. It throws an error as "*hcID" is not accessible.
#include <stdlib.h>
typedef void (*ProcFoo) (void *pvData, unsigned long ulBarTag);
static char* gpcID = NULL;
typedef struct FooType
{
ProcFoo pfFoo; ///< pointer to Foo callback function
unsigned long ulBarTag; ///< base id
} FooType;
FooType* pFoo = NULL;
void RegisterAsFoo(unsigned long ulBarTag, ProcFoo pfFoo)
{
pFoo = (FooType*)malloc(sizeof(FooType));
pFoo->ulBarTag = ulBarTag;
pFoo->pfFoo = pfFoo;
}
void GetBarTag(unsigned long *pulBarTag, unsigned long ulBarTag)
{
*pulBarTag = ulBarTag;
}
int main()
{
char s[] = "Some Value";
gpcID = s;
char ** hcID = NULL;
RegisterAsFoo((unsigned long)&gpcID, (ProcFoo)GetBarTag);
pFoo->pfFoo(&hcID, pFoo->ulBarTag);
// after the execution, the hcID is still NULL in Qt 5.6.3, however,
// it is valid in Visual Studio 2015 and Borland BCC 5.
char* result = (*hcID);
return 1;
}
Thanks if you have any comments or ideas!
The problem is in the
(unsigned long)
cast which works for 32 bit.
Use
(unsigned long long)
instead. You need to do the replacement all over the place.
Otherwise, get rid of the int to pointer conversion or use a 32 bit compiler instead
I apologize for this rather localized question, but I was hoping to get someone else's view on it to make sure I'm not doing something obviously wrong.
I believe I've run into a bug in either the Visual C++ Runtime library, or somewhere in Microsoft's implementation of std::stringstream. The issue only manifests itself with a combination of when:
imbue() is called to change the locale on a stringstream, and
A custom global operator new is used which returns a pointer offset from the base address returned by malloc() used to allocate the block.
I've been able to reproduce this with the following minimal test case:
#include <sstream>
static void *localMalloc(size_t bytes)
{
unsigned char *ptr = static_cast<unsigned char *>( malloc(bytes + 64) );
ptr += 64;
printf("malloc of %d bytes: %ph\n", bytes, ptr);
return ptr;
}
void *operator new(size_t bytes) { return localMalloc(bytes); }
void *operator new[](size_t bytes) { return localMalloc(bytes); }
void operator delete(void *ptr) throw() { /* do nothing */ }
void operator delete[](void *ptr) throw() { /* do nothing */ }
struct DecimalSeparator : std::numpunct<char>
{
char do_decimal_point() const
{
return '.';
}
};
int main()
{
std::stringstream ss;
ss.imbue(std::locale(std::locale(), new DecimalSeparator));
ss << 5; // <-- is_block_type_valid(header->_block_use) assertion failure here
return 0;
}
If either:
The ptr += 64; line in localMalloc() or
The ss.imbue() call in main()
are commented out, the code works as expected, and the assertion does not happen.
I have attempted to step into the code using the debugger as much as I possibly can, but I am currently unable to pinpoint the location at where the code fails in the STL as Visual Studio dumps me into raw disassembly mode after stepping out of basic_stringbuf::overflow() making debugging nearly impossible. As far as I can tell, I haven't seen any invalid memory writes outside of the memory being allocated, so I'm not completely sure where the CRT is checking the heap or why it thinks the pointer is invalid.
Note that operator delete is intentionally ignoring the free in this test case for brevity. It makes no difference if free() is properly called on the memory block or not.
So far, tests on the following compilers and platforms have resulted in:
VS2015 Update 2: fails
VS2015 Update 3: fails
VS2015 Update 3 + KB3165756: fails
clang-703.0.31 on OSX: OK
Does anyone see anything odd here that I missed?
This is more than likely not a bug, but instead your version of delete is not being called, and instead the Visual Studio's debug runtime library's version of the global delete is called. Having two or more versions of the global delete operator in the same program is undefined behavior.
From this reference (Global replacements), the behavior is stated to be undefined when this occurs.
From the C++ ISO standard:
3.7.4 Dynamic storage duration [basic.stc.dynamic]
//...
ยง2 The library provides default definitions for the global allocation
and deallocation functions. Some global allocation and deallocation
functions are replaceable (18.6.1). A C++ program shall provide at
most one definition of a replaceable allocation or deallocation
function.
Running with Visual Studio 2015 using the release version Visual Studio runtime library does not produce this error, and the replacement global delete does in fact get called.
Visual Studio 2015 online compiler results.
Relating to a previous question of mine
I've successfully interposed malloc, but calloc seems to be more problematic.
That is with certain hosts, calloc gets stuck in an infinite loop with a possible internal calloc call inside dlsym. However, a basic test host does not exhibit this behaviour, but my system's "ls" command does.
Here's my code:
// build with: g++ -O2 -Wall -fPIC -ldl -o libnano.so -shared Main.cc
#include <stdio.h>
#include <dlfcn.h>
bool gNanoUp = false;// global
// Function types
typedef void* (*MallocFn)(size_t size);
typedef void* (*CallocFn)(size_t elements, size_t size);
struct MemoryFunctions {
MallocFn mMalloc;
CallocFn mCalloc;
};
MemoryFunctions orgMemFuncs;
// Save original methods.
void __attribute__((constructor)) __nano_init(void) {
fprintf(stderr, "NANO: init()\n");
// Get address of original functions
orgMemFuncs.mMalloc = (MallocFn)dlsym(RTLD_NEXT, "malloc");
orgMemFuncs.mCalloc = (CallocFn)dlsym(RTLD_NEXT, "calloc");
fprintf(stderr, "NANO: malloc() found #%p\n", orgMemFuncs.mMalloc);
fprintf(stderr, "NANO: calloc() found #%p\n", orgMemFuncs.mCalloc);
gNanoUp = true;
}
// replacement functions
extern "C" {
void *malloc(size_t size) {
if (!gNanoUp) __nano_init();
return orgMemFuncs.mMalloc(size);
}
void* calloc(size_t elements, size_t size) {
if (!gNanoUp) __nano_init();
return orgMemFuncs.mCalloc(elements, size);
}
}
Now, When I do the following, I get an infinite loop followed by a seg fault, eg:
% setenv LD_PRELOAD "./libnano.so"
% ls
...
NANO: init()
NANO: init()
NANO: init()
Segmentation fault (core dumped)
However if I comment out the calloc interposer, it almost seems to work:
% setenv LD_PRELOAD "./libnano.so"
% ls
NANO: init()
NANO: malloc() found #0x3b36274dc0
NANO: calloc() found #0x3b362749e0
NANO: init()
NANO: malloc() found #0x3b36274dc0
NANO: calloc() found #0x3b362749e0
<directory contents>
...
So somethings up with "ls" that means init() gets called twice.
EDIT
Note that the following host program works correctly - init() is only called once, and calloc is successfully interposed, as you can see from the output.
// build with: g++ test.cc -o test
#include <stdio.h>
#include <stdlib.h>
int main(int argc, char* argv[]) {
void* p = malloc(123);
printf("HOST p=%p\n", p);
free(p);
char* c = new char;
printf("HOST c=%p\n", c);
delete c;
void* ca = calloc(10,10);
printf("HOST ca=%p\n", ca);
free(ca);
}
% setenv LD_PRELOAD "./libnano.so"
% ./test
NANO: init()
NANO: malloc() found #0x3b36274dc0
NANO: calloc() found #0x3b362749e0
HOST p=0x601010
HOST c=0x601010
HOST ca=0x601030
I know I am a bit late (6 years). But I wanted to override calloc() today and faced a problem because dlsym() internally uses calloc(). I solved it using a simple technique and thought of sharing it here:
static unsigned char buffer[8192];
void *calloc(size_t nmemb, size_t size)
{
if (calloc_ptr == NULL) // obtained from dlsym
return buffer;
init(); // uses dlsym() to find address of the real calloc()
return calloc_ptr(len);
}
void free(void *in)
{
if (in == buffer)
return;
free_ptr(in);
}
buffer satisfies the need of dlsym() till the real calloc() has been located and my calloc_ptr function pointer initialized.
With regard to __nano_init() being called twice: You've declared the function as a constructor, so it's called when the library is loaded, and it's called a second time explicitly when your malloc() and calloc() implementations are first called. Pick one.
With regard to the calloc() interposer crashing your application: Some of the functions you're using, including dlsym() and fprintf(), may themselves be attempting to allocate memory, calling your interposer functions. Consider the consequences, and act accordingly.
Using dlsym based hooking can result in crashes, as dlsym calls back into the memory allocator. Instead use malloc hooks, as I suggested in your prior question; these can be installed without actually invoking dlsym at all.
You can get away with a preliminary poor calloc that simply returns NULL. This actually works on Linux, YMMV.
static void* poor_calloc(size_t nmemb, size_t size)
{
// So dlsym uses calloc internally, which will lead to infinite recursion, since our calloc calls dlsym.
// Apparently dlsym can cope with slightly wrong calloc, see for further explanation:
// http://blog.bigpixel.ro/2010/09/interposing-calloc-on-linux
return NULL; // This is a poor implementation of calloc!
}
You can also use sbrk to allocate the memory for "poor calloc".