I would like to have an example showing the difference between passing values to a function by value and by reference with filling of the memory.
The question here is: How can I monitor inside a C++ program how much stack/ heap memory is being used:
I have a recursive function which I hope is a good example:
#include <iostream>
#include <vector>
typedef std::vector<double> myVec;
void recursiveFunc(myVec n)
{
if (n[0] == 0)
{
//std::cout << "I am the last one" << std::endl;
return;
}
else
{
n[0] -= 1;
//std::cout << "I am here" << std::endl;
recursiveFunc(n);
}
}
int main()
{
myVec v2 = {1000000,2,3,4,5};
recursiveFunc(v2);
return 0;
}
How can I monitor inside a C++ program how much stack/ heap memory is being used
There is no standard way to measure how much stack or heap memory is being used in C++. System specific API exist. First step in using such API to find out what system you are programming for.
External tools exist to profile peak memory use (both stack and heap), such as valgrind.
Related
I have currently a memory issue using the Botan library (version 2.15) for cryptography functions within a C++ project. My development environment is Solus Linux 4.1 (kernel-current), but I could observe this issue on Debian Buster too.
I observed that some memory allocated internally by Botan for calculations is not deallocated when going out of scope. When I called Botan::HashFunction, Botan::StreamCipher and Botan::scrypt multiple times, always going out of scope in between, the memory footprint increases steadily.
For example, consider this code:
#include <iostream>
#include <vector>
#include "botan/scrypt.h"
void pause() {
char ch;
std::cout << "Insert any key to proceed... ";
std::cin >> ch;
}
std::vector<uint8_t> get_scrypt_passhash(std::string const& password, std::string const& name) {
std::vector<uint8_t> key (32);
Botan::scrypt(key.data(), key.size(), password.c_str(), password.length(), salt.c_str(), salt.length(), 65536, 32, 1);
std::cout << "From function: before closing.\n";
pause();
return key;
}
int main(int argc, char *argv[]) {
std::cout << "Beginning test.\n";
pause();
auto pwhashed = get_scrypt_passhash(argv[1], argv[2]);
std::cout << "Test ended.\n";
pause();
}
I used the pause() function to observe the memory consumption (I called top/pmap and observed KSysGuard during the pause), when it is called from within get_scrypt_passhash before terminating, the used memory (both by top/pmap and KSysGuard) is about 2 MB more than at beginning, and after terminating the same.
I tried to dive into the Botan source code, but I cannot find memory leaks or the like. Valgrind also outputted that all allocated bytes have been freed, so no memory leaks were possible.
Just for information, I tried the same functionality with Crypto++ without observing this behavior.
Has anyone experienced the same issue? Is there a way to fix it?
We are under a PCI PA-DSS certification and one of its requirements is to avoid writing clean PAN (card number) to disk. The application is not writing such information to disk, but if the operating system (Windows, in this case) needs to swap, the memory contents is written to page file. Therefore the application must clean up the memory to prevent from RAM capturer services to read sensitive data.
There are three situations to handle:
heap allocation (malloc): before freeing the memory, the area can be cleaned up with memset
static or global data: after being used, the area can be cleaned up using memset
local data (function member): the data is put on stack and is not accessible after the function is finished
For example:
void test()
{
char card_number[17];
strcpy(card_number, "4000000000000000");
}
After test executes, the memory still contains the card_number information.
One instruction could zero the variable card_number at the end of test, but this should be for all functions in the program.
memset(card_number, 0, sizeof(card_number));
Is there a way to clean up the stack at some point, like right before the program finishes?
Cleaning the stack right when the program finishes might be too late, it could have already been swapped out during any point at its runtime. You should keep your sentitive data only in memory locked with VirtualLock so it does not get swapped out. This has to happen before said sensitive data is read.
There is a small limit on how much memory you can lock like this so you can propably not lock the whole stack and should avoid storing sensitive data on the stack at all.
I assume you want to get rid of this situation below:
#include <iostream>
using namespace std;
void test()
{
char card_number[17];
strcpy(card_number, "1234567890123456");
cout << "test() -> " << card_number << endl;
}
void test_trash()
{
// don't initialize, so get the trash from previous call to test()
char card_number[17];
cout << "trash from previous function -> " << card_number << endl;
}
int main(int argc, const char * argv[])
{
test();
test_trash();
return 0;
}
Output:
test() -> 1234567890123456
trash from previous function -> 1234567890123456
You CAN do something like this:
#include <iostream>
using namespace std;
class CardNumber
{
char card_number[17];
public:
CardNumber(const char * value)
{
strncpy(card_number, value, sizeof(card_number));
}
virtual ~CardNumber()
{
// as suggested by #piedar, memset_s(), so the compiler
// doesn't optimize it away.
memset_s(card_number, sizeof(card_number), 0, sizeof(card_number));
}
const char * operator()()
{
return card_number;
}
};
void test()
{
CardNumber cardNumber("1234567890123456");
cout << "test() -> " << cardNumber() << endl;
}
void test_trash()
{
// don't initialize, so get the trash from previous call to test()
char card_number[17];
cout << "trash from previous function -> " << card_number << endl;
}
int main(int argc, const char * argv[])
{
test();
test_trash();
return 0;
}
Output:
test() -> 1234567890123456
trash from previous function ->
You can do something similar to clean up memory on the heap or static variables.
Obviously, we assume the card number will come from a dynamic source instead of the hard-coded thing...
AND YES: to explicit answer the title of your question: The stack will not be cleaned automatically... you have to clean it by yourself.
I believe it is necessary, but this is only half of the problem.
There are two issues here:
In principle, nothing prevents the OS from swapping your data while you are still using it. As pointed out in the other answer, you want VirtualLock on windows and mlock on linux.
You need to prevent the optimizer from optimizing out the memset. This also applies to global and dynamically allocated memory. I strongly suggest to take a look at cryptopp SecureWipeBuffer.
In general, you should avoid to do it manually, as it is an error-prone procedure. Instead, consider using a custom allocator or a custom class template for secure data that can be freed in the destructor.
The stack is cleaned up by moving the stack pointer, not by actually popping values from it. The only mechanics are to pop the return into the appropriate registers. You must do it all manually. Also -- volatile can help you avoid optimizations on a per variable basis. You can manually pop the stack clean, but -- you need assembler to do that -- and it is not so simple to start manipulating the stack -- it is not actually your resource -- the compiler owns it as far as you are concerned.
I am trying to write a object oriented C++ code that is parallelized with OpenACC.
I was able to find some stackoverflow questions and GTC talks on OpenACC, but I could not find some real world examples of object oriented code.
In this question an example for a OpenACCArray was shown that does some memory management in the background (code available at http://www.pgroup.com/lit/samples/gtc15_S5233.tar).
However, I am wondering if it is possible create a class that manages the arrays on a higher level. E.g.
struct Data
{
// OpenACCArray<float> a;
OpenACCArray<Vector3<float>> a3;
Data(size_t len) {
#pragma acc enter data copyin(this)
// a.resize(len);
a3.resize(len);
}
~Data() {
#pragma acc exit data delete(this)
}
void update_device() {
// a.update_device();
a3.update_device();
}
void update_host() {
// a.update_host();
a3.update_host();
}
};
int main(int argc, char *argv[])
{
const size_t len = 32*128;
Data d(len);
d.update_device();
#pragma acc kernels loop independent present(d)
for (int i=0; i < len; ++i) {
float val = (float)i/(float)len;
d.a3[i].x = val;
d.a3[i].y = i;
d.a3[i].z = d.a3[i].x / d.a3[i].y;
}
d.update_host();
for (int i=0; i < len/128; ++i) {
cout << i << ": " << d.a3[i].x << "," << d.a3[i].y << "," << d.a3[i].z << endl;
}
cout << endl;
return 0;
}
Interestingly this program works, but as soon as I uncomment OpenACCArray<float> a;, i.e. add another member to that Data struct, I get memory errors.
FATAL ERROR: variable in data clause is partially present on the device.
Since the OpenACCArray struct is a flat structure that handles the pointer indirections on its own it should work to copy it as member?
Or does need to be a pointer to the struct and the pointers have to be hardwired with directives?
Then I fear the problem that I have to use alias pointers as suggested by jeff larkin at the above mentioned question.
I don't mind doing the work to get this running, but I cannot find any reference how to do that.
Using compiler directives keepgpu,keepptx helps a bit to understand what the compiler is doing, but I would prefer an alternative to reverse engineering generated ptx code.
Any pointers to helpful reference project or documents are highly appreciated.
In the OpenACCArray1.h header, remove the two "#pragma acc enter data create(this)" pragmas. What's happening is that the "Data" constructor is creating the "a" and "a3" objects on the device. Hence, when the second enter data region is encountered in the OpenACCArray constructor, the device this pointer is already there.
It works when there is only one data member since "a3" and "Data" share the same address for the this pointer. Hence when the second enter data pragma is encountered, the present check sees that it's already on the device so doesn't created it again. When "a" is added, the size of "Data" is twice that of "a", hence the present check sees that the this pointer is already there but has a different size than before. That's what the "partially present" error means. The data is there but has a different than expected size.
Only the parent class/struct should create the this pointer on the device.
Hope this helps,
Mat
I am learning C++ and I am having quite a lot of trouble with my current assignment. I have completed a good amount of it so far. However I have been making very slow progress of late due to what I think is my poor understanding of what is going on behind the scenes.
What I am trying to do in the following code is:
Get two separate values (Bullet damage). Done.
Create a dynamic array. Done.
Fill a part (that is the size of a modulus of a random number between 1 and 10) of said dynamic array with one value and the rest with the other in a random order. Here I am having trouble.
Clean up the memory used by said dynamic array. Done.
The error I get is as follows:
Unhandled exception at 0x00a323e3 in Class 3.exe: 0xC0000005: Access
violation reading location 0xcdcdcdcd.
I'm pretty sure that the error occurs when I try to set ammoArray[i] to a value. But I don't know why it's giving it to me, my code compiles fine. I played around with it a bit and in one case I got it to store the memory addresses of bDamage and sDamage and then print out the memory addresses of each element of the array. What I want it to do is store the values held by bDamage and sDamage.
Now for my question:
Why won't ammoArray store the values of bDamage and sDamage instead of the memory addresses of the array's elements? And How do I get it to store them?
Here is my Main.cpp:
#include <cstdlib>
#include "Ammunition.h"
#include "AmmunitionManager.h"
#include "Bullet.h"
#include "Game.h"
#include "Pistol.h"
#include "Player.h"
#include "Point.h"
#include "Shell.h"
#include "Shotgun.h"
#include "WeaponManager.h"
#include "Weapons.h"
using namespace std;
void main()
{
Ammunition amVar;
AmmunitionManager *var = new AmmunitionManager();
amVar.setBDamage(6);
amVar.setSDamage(2);
var->FillAmmoArray(amVar.getSDamage(),amVar.getBDamage());
system("PAUSE");
}
Here is the .h file of the class in question:
#ifndef AMMUNITIONMANAGER_H
#define AMMUNITIONMANAGER_H
#include "Point.h"
#include "Ammunition.h"
class AmmunitionManager
{
public:
AmmunitionManager();
AmmunitionManager(int,int);
~AmmunitionManager();
void FillAmmoArray(int,int);
private:
Ammunition Ammo;
int **ammoArray;
};
#endif
Here is the .cpp file of the class in question:
#include <iostream>
#include <cstdlib>
#include <ctime>
#include "AmmunitionManager.h"
#include "Point.h"
#include "Ammunition.h"
using namespace std;
AmmunitionManager::AmmunitionManager()
{
}
AmmunitionManager::AmmunitionManager(int sDamage,int bDamage)
:Ammo(sDamage,bDamage)
{
cout << "Filling ammo reservoir." << endl;
ammoArray = new int* [10];
}
void AmmunitionManager::FillAmmoArray(int sDamage,int bDamage)
{
srand(time(NULL));
int *holdS = &sDamage;
int *holdB = &bDamage;
if(ammoArray)
{
for(int i = 0;i < 9;i++)
{
int randC = rand() % 2 + 1;
if(randC == 1)
{
cout << "Was: " << ammoArray[i] << endl;//I am getting the error here.
ammoArray[i] = holdS;
cout << "Is: " << ammoArray[i] << endl;
}
if(randC == 2)
{
cout << "Was: " << ammoArray[i] << endl;//I am getting the error here.
ammoArray[i] = holdB;
cout << "Is: " << ammoArray[i] << endl;
}
}
}
}
AmmunitionManager::~AmmunitionManager()
{
*ammoArray = 0;
if(ammoArray)
{
delete [] ammoArray;
}
}
Why won't ammoArray store the values of bDamage and sDamage instead of the memory addresses of the array's elements?
Because you said it should store addresses.
Here is a pointer to a pointer:
int **ammoArray;
and here is an array of pointers:
ammoArray = new int* [10];
And How do I get it to store them?
By doing this instead:
int *ammoArray;
and this:
ammoArray = new int [10];
and adjusting FillAmmoArray accordingly.
The default constructor should look like this:
AmmunitionManager::AmmunitionManager()
: ammoArray(nullptr)
{
}
The destructor should look like this:
AmmunitionManager::~AmmunitionManager()
{
delete [] ammoArray;
}
And you should only call srand once.
It's usually done at the beginning of main.
I'm not getting any errors (VS2013). But the values stored are the addresses of sDamage and bDamage.
Did you properly use AmmunitionManager(int sDamage,int bDamage) as a constructor for creating the AmmunitionManager object? From what I'm seeing, you're not.
Apart from that, may I ask why you're using exotic constructs such as **ammoArray instead of e.g. a simple vector<int>? I'm guessing it's part of your assignment, but I'm asking just to make sure I'm not missing anything.
I called the object like this:
int _tmain(int argc, _TCHAR* argv[])
{
AmmunitionManager* tc = new AmmunitionManager(5,10);
tc->FillAmmoArray(10,10);
return 0;
}
The problem is that you initialize AmmunitionManager with the default constructor:
AmmunitionManager *var = new AmmunitionManager();
In you default constructor you do nothing so ammoArray may contain any value.
It is better to initialize all the data to their default values:
AmmunitionManager::AmmunitionManager() : Ammo(), ammoArray(NULL/* or nullptr for C++11 */)
{
}
Now if you call for
var->FillAmmoArray(amVar.getSDamage(),amVar.getBDamage());
It will exit immediately since ammoArray is NULL.
Or probably you want to initialize ammoArray anyway, so the default constructor should have its initialization as well:
AmmunitionManager::AmmunitionManager() : Ammo()
{
ammoArray = new int* [10];
}
Also srand should be called only once, so better to place this code
srand(time(NULL));
in the main() or any other module which is guaranteed to be executed only once.
In the destructor, there is no need to zero *ammoArray=0, it actually puts 0 at the first element of that array (and that's it), you anyway delete it. And imagine that ammoArray is NULL, accessing *ammoArray would cause another segmentation fault.
Also there is no need to check for ammoArray beibg NULL before deleting it. The standard allows to 'delete' NULL pointers. delete will just return without doing nothing.
General note
It is better to use (safer and easier to maintain) std::vector instead of (dynamic) arrays and smart pointers instead of flat ones.
It's a bit tricky answering without building and debugging, but the first thing that strikes me are: Why are you using pointers (*) to int throughout?
Why don't you just have the array as a pointer:
int *ammoArray;
and make the other int-instances (remove the pointers - * and the address-of's (&))?
Regards
Keeping track of how many times a function is called is easy when passing the counter as an argument into the function. It's also easy when returning a one from the called function. But, I do not want to go that route. The reason behind this is because it seems like bad programming (letting the function know too much information). Is there a better way to keep track of how many times this function has been called?
I'm just looking for concepts that I could study. Providing code examples is not neccessary, but might be helpful.
Edit: I'm not actually looking for profiling tools. Let me add some code to get my point across. Because scope for funcCounter ends in main, I have no way of getting back a variable from myFunction that will increment funcCounter. I could possibly return 1 from myFunction and then increment funcCounter that way, but this doesn't seem like very good programming. Is there another way to do it?
int main()
{
int funcCounter = 0;
char *mystring = "This is a silly function.";
myFunction(mystring);
cout << "Times function is called: " << funcCounter << endl;
return 0;
}
void myFunction(char *mystring)
{
cout << mystring << endl;
}
Have a static variable in your function and keep incrementing it each time the function in called.
void my_Function(void) {
static unsigned int call_count = 0;
call_count++;
}
If you want to do it for debugging reasons, then there are tools like gcov which do this for you. (I'm pretty sure Microsoft doesn't have an alternative bundled with Microsoft Visual C++)
I would do this through the use of a profiling tool like gcov (which is for linux). These programs do the work of inserting code into your program during compilation and give you a report of how many times a function is called, where its called from, and how long the program spent executing that function.
It sounds like what you are looking for is a profiler. Depending on the platform you are using there are a slew of tools available that can help you hunt down the (ab)uses of a routine.
Please revise your question with the platform for which you need profiling tools.
If the function is part of a class, you can add a static counter to the class, plus an accessor and/or reset functions:
class X
{
private:
/* diagnostics */
static int counter = 0;
int read_counter() const { return counter; }
void reset_counter() { counter = 0; }
public:
/* real code */
fcn() {
++counter;
/* ... */
}
};
The problem with adding a static counter to a standalone function is that there's no way to get at the value.
You could add a global, of course, but instead of a raw global I'd suggest an instance of a singleton containing all your diagnostic code and data.
Use a class like this one, and simply instantiate it at the top of a function (or any other block) like is done in f() below.
Note: There is some overhead for gettimeofday() so you may want to use a different timing method, but that is a completely different topic worthy of it's own question (and has been addressed before on SO).
#include <iostream>
#include <string>
#include <map>
#include <sstream>
#include <ctime>
#include <cstdlib>
#include <sys/time.h>
class PerfStats
{
private:
std::string which_;
timeval begin_;
public:
PerfStats(std::string const &file, int line)
{
std::stringstream ss;
ss << file << ':' << line;
which_ = ss.str();
gettimeofday(&begin_, NULL);
}
~PerfStats()
{
timeval end;
gettimeofday(&end, NULL);
Times[which_] = (end.tv_sec - begin_.tv_sec) + (end.tv_usec - begin_.tv_usec)/1000000.0;
++Counts[which_];
}
static std::map<std::string, double> Times;
static std::map<std::string, unsigned int> Counts;
static void Print()
{
for(std::map<std::string, double>::iterator it = Times.begin(); it != Times.end(); ++it)
std::cout << it->first << " :\t" << it->second << "s" << std::endl;
for(std::map<std::string, unsigned int>::iterator it = Counts.begin(); it != Counts.end(); ++it)
std::cout << it->first << " :\t" << it->second << " times" << std::endl;
}
};
std::map<std::string, double> PerfStats::Times;
std::map<std::string, unsigned int> PerfStats::Counts;
void f()
{
PerfStats(__FILE__, __LINE__);
usleep(1);
}
main()
{
srand(time(NULL));
for(int i = 0; i < rand(); ++i)
f();
PerfStats::Print();
}
Sample output:
test.cpp:54 : 2e-06s
test.cpp:54 : 21639 times
Bad coding style, but maybe adding global variables and if necessary mutex locks may do the trick.