I have tried to check programmatically how big an array I can allocate but my code does not seem to check it. How to make it faster? In the end I would like to get an exception.
#include "stdafx.h"
#include "iostream"
using namespace std;
int ASCENDING = 1, DESCENDING = 2;
int tworzTablice(int rozmiar, char* t){
try{
t = new char[rozmiar];
delete []t;
}catch (std::bad_alloc& e){
tworzTablice(rozmiar - 1,t);
return -1;
}
return rozmiar;
}
int f(long p, long skok){
char* t;
try{
while(true){
t = new char[p];
delete []t;
p = p + skok;
}
}
catch (std::bad_alloc& ba){
p = tworzTablice(p-1, t);
cout<<"blad";
}
return p;
}
int main(){
cout<<f(0, 100000000)<<endl;;
cout<<"koniec"<<endl;
system("pause");
return 0;
}
As I noted, there is a way to query the OS in order to determine the maximal size of heap-allocated memory, but I can't for the heck of it remember its name.
However, you can easily find out yourself. However, you should use malloc/free instead of new/delete in order to avoid the unnecessary initialisation of all cells;
#include <cstdlib>
#include <cstdio>
size_t maxMem() {
static size_t size = 0;
if (!size) {
size_t m = 0;
for (void* p = 0; (p = malloc(1<<m)); m++)
free(p);
while (m) {
size_t const testSize = size + (1<<(--m));
if (void* const p = malloc(testSize)) {
size = testSize;
free(p);
}
}
}
return size;
}
int main() {
// forgive me for using printf, but I couldn't remember how to hex-format in std::cout
printf("%u (hex %X)\n",int(maxMem()),int(maxMem()));
}
On my 64 bit machine I get
2147483647 (hex 7FFFFFFF)
while on another 32 system I get
2140700660 (hex 7F987FF4)
You can then go ahead and new an array of that size if you really have to. Note however, that this is the largest consecutive chunk you can request. The total memory your process might allocate is larger and depends on the installed RAM and the reserved swap space.
Allocating all available memory is probably a bad idea, but if you really want to:
vector<char*> ptrs;
int avail;
try {
while (true)
ptrs.push_back(new char[1000]);
}
catch (bad_alloc& b)
{
avail = ptrs.size() * 1000;
for (int i = 0; i < ptrs.size(); i++)
delete[] ptrs[i];
}
Related
When running this code I get an error as shown in the image below.
I've tried running it on GCC compiler and it worked fine. But when running it on Visual Studio on Windows this error appeared:
Debug Error!
Program: C:\Users\yudab\source\repos\Project2\Debug\Project2.exe
HEAP CORRUPTION DETECTED: after Normal block (#153) at 0x014FD2E0.
CRT detected that the application wrote to memory after end of heap buffer.
After some testing it seems as the error only appears after trying to delete the second word.
#include <cstring>
#include <string>
#pragma warning(disable : 4996)
#include <iostream>
using namespace std;
void delStr(char**& lexicon, int& lexSize, char word[]);
void printAll(char** lexicon, int lexSize);
void retract2dArr(char**& arr, int& size);
int main() {
char** lexicon = new char* [3];
lexicon[0] = new char[6]{ "hello" };
lexicon[1] = new char[5]{ "test" };
lexicon[2] = new char[6]{ "world" };
int size = 3;
char removeTest[5] = { "test" }; //The first word I want to remove from the list
char removeWorld[6] = { "world" }; //The second word I want to remove from the list
printAll(lexicon, size); //First prints the entire list
delStr(lexicon, size, removeTest); //Removes the first word
delStr(lexicon, size, removeWorld); //Removes the second word
printAll(lexicon, size); //Prints the list after deleting the words
return 0;
}
void delStr(char**& lexicon, int& lexSize, char word[]) {
bool toDelete = false;
for (int i = 0; i < lexSize; i++) {
if (strcmp(lexicon[i], word) == 0) {
toDelete = true;
for (; i < lexSize - 1; i++) {
strcpy(lexicon[i], lexicon[i + 1]);
}
}
}
if (toDelete == true) {
delete[] lexicon[lexSize - 1];
retract2dArr(lexicon, lexSize);
}
return;
}
void printAll(char** lexicon, int lexSize) {
for (int i = 0; i < lexSize; i++) {
cout << lexicon[i];
if (i != lexSize - 1) {
cout << " ";
}
}
cout << endl;
return;
}
void retract2dArr(char**& arr, int& size) {
size--;
char** newArr = new char* [size];
for (int i = 0; i < size; i++) {
*(newArr + i) = *(arr + i);
}
printAll(newArr, size);
delete[] arr;
arr = newArr;
return;
}
You can't strcpy one string to another
if (strcmp(lexicon[i], word) == 0) {
toDelete = true;
for (; i < lexSize - 1; i++) {
strcpy(lexicon[i], lexicon[i + 1]);
}
}
As length will be different for each strings.
Example:
lexicon[0] = new char[6]{ "hello" };
lexicon[1] = new char[5]{ "test" }; // length is 4
lexicon[2] = new char[6]{ "world" }; // length is 5
3rd string won't fit in 2nd string, it causes out of bound access.
As kiran Biradar pointed out, the strcpy is to blame here. Although instead of copying each word in the lexicon to the memory allocated for the previous word, it would probably be better to simply move the pointers back withing the lexicon array.
Try something like this for your delStr function:
void delStr(char**& lexicon, int& lexSize, char word[]) {
for (int i = 0; i < lexSize; i++) {
if (strcmp(lexicon[i], word) == 0) {
delete[] lexicon[i];
for (; i < lexSize - 1; i++) {
lexicon[i] = lexicon[i + 1];
}
retract2dArr(lexicon, lexSize);
}
}
}
P.S. You didnt need to use a toDelete flag, you could call teh retract2dArr function within the first if.
I get segmentation faults when I use the =-operator to copy a struct that contains a std::vector to uninitialized memory.
The critical code looks like that:
template<typename T>
ComponentContainer
{
T* buffer;
size_t capacity;
size_t m_size;
public:
ComponentContainer();
~ComponentContainer();
size_t size();
void resize(size_t size);
T & operator[](size_t index);
};
template<typename T>
void ComponentContainer<T>::resize(size_t newSize)
{
if(this->m_size >= newSize)
{
this->m_size = newSize;
}
else
{
if(this->capacity < newSize)
{
const size_t newCapacity = capacity*2;
T* newBuffer = (T*)malloc(newCapacity*sizeof(T));
for(size_t i = 0; i<m_size; i++)
{
// checks if this->buffer[i] is valid intialized memory
if(pseudo_checkIfElementIsInitialized(i))
{
// when this is uncommented no segfault happens
//new (&newBuffer[i]) T();
newBuffer[i] = this->buffer[i]; // <- segfault happens here
}
}
this->capacity = newCapacity;
free(this->buffer);
this->buffer = newBuffer;
}
this->m_size = newSize;
}
}
The T-type is a struct with a std::vector of structs when I get the segfault.
I suspect that the std::vector =-operator uses somehow the left side variable newBuffer[i] and the segmentation fault happens since newBuffer[i] is not initialized.
Objects will be created only with in-placement new with the function T & operator[](size_t index). The malloc should only allocate the memory without initializing anything.
I tried to write a simple example but that hasn't worked out so well:
#include <iostream>
#include <vector>
struct Hello
{
Hello()
{
std::cout << "constructor" << std::endl;
}
~Hello()
{
std::cout << "destructor" << std::endl;
}
std::vector<double> v = std::vector<double>(1);
};
int main()
{
Hello* buffer = (Hello*)malloc(1*sizeof(Hello));
char* noise = (char*)buffer;
for(size_t i = 0; i<sizeof(Hello); i++)
{
noise[i] = 100;
}
auto tmp = Hello();
tmp.v[0] = 6.6;
//new (&buffer[0]) Hello();
buffer[0] = tmp;
std::cout << buffer[0].v[0] << std::endl;
return 0;
}
It works fine without segfault. I assume that is because the uninitialized memory was just by chance ok for the std::vector =-operation.
So
a) is that theory correct
and if yes
b) how to solve this problem without using a default constructor (T()) for every class that i use as T for my ComponentContainer
Well, yeah. You can't assign to an object that doesn't exist.
Uncomment the line that fixes it!
If you can't default construct, then copy construct:
new (&newBuffer[i]) T(this->buffer[i]);
And if you can't do that, then, well, you know the rest.
The malloc should only allocate the memory without initializing anything.
Is it possible that you've underestimated the weight of this statement? You don't just get memory then decide whether or not to initialise it with some values. You have to actually create objects before using them; this is not optional. You're programming C++, not manipulating bits and bytes on a tape :)
I have a small example program here for the particle photon that has a memory bug that I cannot figure out.
What it does: loads up a buffer with small string chunks, converts that large buffer back into a string. Then it creates a bunch of objects that are only wrappers for small chunks of buffer. It does this repetitively, and I don't allocate any new memory after the setup(), yet the memory goes down slowly until it crashes.
main.cpp
includes, variable declarations
#include "application.h" //needed when compiling spark locally
#include <string>
#include <unordered_map>
#include "dummyclass.h"
using namespace std;
SYSTEM_MODE(MANUAL);
char* buffer;
unordered_map<int, DummyClass*> store;
string alphabet;
unsigned char alphabet_range;
unsigned char state;
int num_chars;
static const unsigned char STATE_INIT = 0;
static const unsigned char STATE_LOAD_BUFFER = 1;
static const unsigned char STATE_PREP_FOR_DESERIALIZE = 2;
static const unsigned char STATE_FAKE_DESERIALIZE = 3;
static const unsigned char STATE_FINISH_RESTART = 4;
delete objects helper function
bool delete_objects()
{
Serial.println("deleting objects in 'store'");
for(auto iter = store.begin(); iter != store.end(); iter++)
{
delete iter->second;
iter->second = nullptr;
}
store.clear();
if(store.empty())
return true;
else
return false;
}
set up function, allocates memory, initial assignments
void setup()
{
Serial.begin(9600);
Serial1.begin(38400);
delay(2000);
buffer = new char[9000];
alphabet = string("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789~!##$^&*()_-?/><[]{}|");
alphabet_range = alphabet.length() - 1;
state = STATE_INIT;
num_chars = 0;
}
loop function, gets run over and over
void loop()
{
switch(state){
case STATE_INIT: {
strcpy(buffer, "");
state = STATE_LOAD_BUFFER;
delay(1000);
break;
}
case STATE_LOAD_BUFFER: {
if(num_chars < 6000){
string chunk;
for(char i = 0; i < 200; i++){
int index = rand() % alphabet_range;
chunk.append(alphabet.substr(index, 1));
num_chars++;
}
strcat(buffer, chunk.c_str());
}
else{
num_chars = 0;
state = STATE_PREP_FOR_DESERIALIZE;
}
delay(500);
break;
}
case STATE_PREP_FOR_DESERIALIZE: {
Serial.println("\nAttempting to delete current object set...");
delay(500);
if(delete_objects())
Serial.println("_delete_objects succeeded");
else {
Serial.println("_delete_objects failed");
break;
}
state = STATE_FAKE_DESERIALIZE;
delay(1000);
break;
}
case STATE_FAKE_DESERIALIZE: {
string buff_string(buffer);
if(buff_string.length() == 0){
Serial.println("Main:: EMPTY STRING CONVERTED FROM BUFFER");
}
int index = 0;
int key = 1;
while(index < buff_string.length())
{
int amount = (rand() % 50) + 5;
DummyClass* dcp = new DummyClass(buff_string.substr(index, amount));
store[key] = dcp;
index += amount;
key++;
}
state = STATE_FINISH_RESTART;
delay(1000);
break;
}
case STATE_FINISH_RESTART: {
state = STATE_INIT;
break;
}
}
}
dummyclass.h
very minimal, constructor just stores a string in a character buffer. this object is just a wrapper.
using namespace std;
class DummyClass {
private:
char* _container;
public:
DummyClass(){
}
DummyClass(string input){
_container = new char[input.length()];
strcpy(_container, input.c_str());
}
~DummyClass(){
delete _container;
_container = nullptr;
}
char* ShowMeWhatYouGot(){
return _container;
}
};
EDIT:
This is a real problem that I am having, I'm not sure why it is getting downvoted. Help me out here, how can I be more clear? I'm reluctant to shrink the code since it imitates many aspects of a much bigger program that it is modeling simply. I want to keep the structure of the code in place in case this bug is an emergent property.
Always account for the string terminator:
DummyClass(string input){
_container = new char[input.length()];
strcpy(_container, input.c_str());
}
Allocates one too few bytes to hold the input string and terminator that is then copied into it. The \0that's appended at the end is overwriting something, which is most likely metadata required to re-integrate the alloced memory fragment back into the heap successfully. I'm actually surprised it didn't crash...
It probably doesn't happen every allocation (only when you overflow into a new 8 byte aligned chunk), but once is enough :)
So, after some testing, I'd like to give a shout out to Russ Schultz who commented the right answer. If you want to post a solution formally, I would be happy to mark it as correct.
The memory bug is caused by allocating the char buffer _container without considering the null terminating character, meaning I am loading in a string that is too big. (not entirely sure why this causes a bug and doesn't throw an error?)
On a different site however, I also received this piece of advice:
string chunk;
for(char i = 0; i < 200; i++){
int index = rand() % alphabet_range;
chunk.append(alphabet.substr(index, 1));
// strcat(buffer, alphabet.substring(index, index + 1));
num_chars++;
}
This loop looks suspect to me. You are depending on the string append method to grow chunk as needed, but you know you are going to run that loop 200 times. Why not use the string reserve method to just allocate that much space? I bet that this chews up a lot of memory with each new char you append calling realloc, potentially fragmenting memory.
This ended up not being the solution, but it might be good to know.
I have a recursive function that requires me to create a new array every time the function is called. The function also requires the array that was previously created:
void myFunc(int* prevArray)
{
int newSize;
//do some calculations to find newSize
int* newArray;
newArray = new int[newSize];
//do some calculations to fill newArray
//check some stopping condition
myFunc(newArray);
}
This function leaks memory, but I can't avoid that by adding
delete[] newArray;
since I can only add that after calling the function again. How can I solve this?
You can solve this by making use of dynamic memory allocation.
// allocate initial size
const int INITIAL_SIZE = 5;
int *myArray = malloc(sizeof(int) * INITIAL_SIZE));
int myFunc(int *aArray, int numAllocated) {
int numElements = calculateNewSize();
if (numElements != numAllocated) {
// allocate new size
realloc(aArray, (numElements * sizeof(int));
}
return numElements;
}
Now you can call myFunc like this:
int numElements;
numElements = myFunc(myArray, numElements);
When your done using myFunc don't forget to free the memory
free(myArray);
Try something like
void myFunc(int* prevArray)
{
int newSize;
...newArray = new int[newSize];
myFunc(newArray);
delete[] newArray;
}
or better yet use std::unique_ptr to control the newArray memory. In this way you will follow the rule of thumb regarding dynamic memory - that it should have one owner, responsible for both allocating and freeing it.
You might just use a vector and swap the new result into the final result.
#include <iostream>
#include <vector>
struct X { ~X() { std::cout << "Destruction\n"; } };
void recursive(unsigned n, std::vector<X>& result) {
// Put new_result in a scope for destruction
{
std::vector<X> new_result(1);
// Do something
// The previous result is no longer needed
std::swap(result, new_result);
}
// Next recursion
if(n) {
std::cout << "Call\n";
recursive(--n, result);
}
}
int main() {
std::vector<X> result(1);
std::cout << "Call\n";
recursive(3, result);
return 0;
}
This is a follow-up to Critique my heap debugger from yesterday. As suggested by bitc, I now keep metadata about the allocated blocks in a separate handwritten hashtable.
The heap debugger now detects the following kinds of errors:
memory leaks (now with more verbose debugging output)
illegal pointers passed to delete (that also takes care of double deletes)
wrong form of delete (array vs. non-array)
buffer overflows
buffer underflows
Feel free to discuss and thanks in advance!
#include <cstdio>
#include <cstdlib>
#include <cstring>
#include <new>
namespace
{
// I don't want to #include <algorithm> for a single function template :)
template <typename T>
void my_swap(T& x, T& y)
{
T z(x);
x = y;
y = z;
}
typedef unsigned char byte;
const byte CANARY[] = {0x5A, 0xFE, 0x6A, 0x8D,
0x5A, 0xFE, 0x6A, 0x8D,
0x5A, 0xFE, 0x6A, 0x8D,
0x5A, 0xFE, 0x6A, 0x8D};
bool canary_dead(const byte* cage)
{
bool dead = memcmp(cage, CANARY, sizeof CANARY);
if (dead)
{
for (size_t i = 0; i < sizeof CANARY; ++i)
{
byte b = cage[i];
printf(b == CANARY[i] ? "__ " : "%2X ", b);
}
putchar('\n');
}
return dead;
}
enum kind_of_memory {AVAILABLE, TOMBSTONE, NON_ARRAY_MEMORY, ARRAY_MEMORY};
const char* kind_string[] = {0, 0, "non-array memory", " array memory"};
struct metadata
{
byte* address;
size_t size;
kind_of_memory kind;
bool in_use() const
{
return kind & 2;
}
void print() const
{
printf("%s at %p (%d bytes)\n", kind_string[kind], address, size);
}
bool must_keep_searching_for(void* address)
{
return kind == TOMBSTONE || (in_use() && address != this->address);
}
bool canaries_alive() const
{
bool alive = true;
if (canary_dead(address - sizeof CANARY))
{
printf("ERROR: buffer underflow at %p\n", address);
alive = false;
}
if (canary_dead(address + size))
{
printf("ERROR: buffer overflow at %p\n", address);
alive = false;
}
return alive;
}
};
const size_t MINIMUM_CAPACITY = 11;
class hashtable
{
metadata* data;
size_t used;
size_t capacity;
size_t tombstones;
public:
size_t size() const
{
return used - tombstones;
}
void print() const
{
for (size_t i = 0; i < capacity; ++i)
{
if (data[i].in_use())
{
printf(":( leaked ");
data[i].print();
}
}
}
hashtable()
{
used = 0;
capacity = MINIMUM_CAPACITY;
data = static_cast<metadata*>(calloc(capacity, sizeof(metadata)));
tombstones = 0;
}
~hashtable()
{
free(data);
}
hashtable(const hashtable& that)
{
used = 0;
capacity = 3 * that.size() | 1;
if (capacity < MINIMUM_CAPACITY) capacity = MINIMUM_CAPACITY;
data = static_cast<metadata*>(calloc(capacity, sizeof(metadata)));
tombstones = 0;
for (size_t i = 0; i < that.capacity; ++i)
{
if (that.data[i].in_use())
{
insert_unsafe(that.data[i]);
}
}
}
hashtable& operator=(hashtable copy)
{
swap(copy);
return *this;
}
void swap(hashtable& that)
{
my_swap(data, that.data);
my_swap(used, that.used);
my_swap(capacity, that.capacity);
my_swap(tombstones, that.tombstones);
}
void insert_unsafe(const metadata& x)
{
*find(x.address) = x;
++used;
}
void insert(const metadata& x)
{
if (2 * used >= capacity)
{
hashtable copy(*this);
swap(copy);
}
insert_unsafe(x);
}
metadata* find(void* address)
{
size_t index = reinterpret_cast<size_t>(address) % capacity;
while (data[index].must_keep_searching_for(address))
{
++index;
if (index == capacity) index = 0;
}
return &data[index];
}
void erase(metadata* it)
{
it->kind = TOMBSTONE;
++tombstones;
}
} the_hashset;
struct heap_debugger
{
heap_debugger()
{
puts("heap debugger started");
}
~heap_debugger()
{
the_hashset.print();
puts("heap debugger shutting down");
}
} the_heap_debugger;
void* allocate(size_t size, kind_of_memory kind) throw (std::bad_alloc)
{
byte* raw = static_cast<byte*>(malloc(size + 2 * sizeof CANARY));
if (raw == 0) throw std::bad_alloc();
memcpy(raw, CANARY, sizeof CANARY);
byte* payload = raw + sizeof CANARY;
memcpy(payload + size, CANARY, sizeof CANARY);
metadata md = {payload, size, kind};
the_hashset.insert(md);
printf("allocated ");
md.print();
return payload;
}
void release(void* payload, kind_of_memory kind) throw ()
{
if (payload == 0) return;
metadata* p = the_hashset.find(payload);
if (!p->in_use())
{
printf("ERROR: no dynamic memory at %p\n", payload);
}
else if (p->kind != kind)
{
printf("ERROR:wrong form of delete at %p\n", payload);
}
else if (p->canaries_alive())
{
printf("releasing ");
p->print();
free(static_cast<byte*>(payload) - sizeof CANARY);
the_hashset.erase(p);
}
}
}
void* operator new(size_t size) throw (std::bad_alloc)
{
return allocate(size, NON_ARRAY_MEMORY);
}
void* operator new[](size_t size) throw (std::bad_alloc)
{
return allocate(size, ARRAY_MEMORY);
}
void operator delete(void* payload) throw ()
{
release(payload, NON_ARRAY_MEMORY);
}
void operator delete[](void* payload) throw ()
{
release(payload, ARRAY_MEMORY);
}
int main()
{
int* p = new int[1];
delete p; // wrong form of delete
delete[] p; // ok
delete p; // no dynamic memory (double delete)
p = new int[1];
p[-1] = 0xcafebabe;
p[+1] = 0x12345678;
delete[] p; // underflow and overflow prevent release
// p is not released, hence leak
}
Very nice, indeed. Your canaries could actually reveal some real cases of overflow/underflow (though not all of them as Matthieu pointed out).
What more. You might run into some problems with a multi-threaded application. Perhaps protect the hashtable from concurrent access?
Now that you log every allocation and deallocation, you can (if you like) provide more information about the program being tested. It might be interesting to know the total and average number of allocations at any given time? The total, max, min and average bytes allocated, and the average lifespan of allocations.
If you want to compare different threads, at least with Pthreads you can identify them with pthread_self(). This heap debugger could become a quite useful analysis tool.
Are you using a very weak malloc that doesn't already have this sort of stuff built into it? Because if it's there, you are doubling the overhead for little gain. Also, this kind of system really hurts when doing small object allocation or is ineffective with them as people do 1 alloc and manage the memory themselves.
As far as the code is concerned, it looks like it will do what you say it will do and it looks well designed and is easy to read. But, if you are going to go through the trouble of doing this though, why not catch your buffer over/under flows at the source by using managed containers/pointers/operator[] thingies. That way, you can debug on the spot of the failure instead of finding out at free that something evil has occured.
There are efficiencies to be had that I'm sure others will find, but these are just some thoughts off the top of my head after looking over your code for a few minutes.
I wonder about the detection of underflows / overflows.
I mean, if I have a 10 elements arrays, then it seems you'll detect if I write at -1 and 10, but what if I write at 20 ? Underflow or Overflow are not necessarily done as part of a buffer overrun (contiguous).
Furthermore, what's the point of preventing release of the block ? This block is (relatively) fine, it's the neighbors you've (unfortunately) corrupted.
Anyway, it seems pretty fine to me, though I would probably have more than one return per function because there's no point in Single Exit. You seem more of a C programmer than a C++ one :)