Is it possible, for a c++ program, to track how much memory the program is using at one time?
For example, a function with a prototype:
int getEstimatedTotalMemoryUsage();
I suppose if it's not possible, then one will have to get out of the program, do a system call and check the results from there. If so, what tools are available for such purposes? Assuming such a thing is possible, that is.
edit: I'm using linux, any tools that can do this for you?
Yes - use POSIX getrusage. From the Linux man page:
Synopsis
#include <sys/time.h>
#include <sys/resource.h>
int getrusage(int who, struct rusage *usage);
Description
getrusage() returns current resource usages, for a who of either RUSAGE_SELF or RUSAGE_CHILDREN. The former asks for resources used by the current process, the latter for resources used by those of its children that have terminated and have been waited for.
struct rusage {
struct timeval ru_utime; /* user time used */
struct timeval ru_stime; /* system time used */
long ru_maxrss; /* maximum resident set size */
long ru_ixrss; /* integral shared memory size */
long ru_idrss; /* integral unshared data size */
long ru_isrss; /* integral unshared stack size */
long ru_minflt; /* page reclaims */
long ru_majflt; /* page faults */
long ru_nswap; /* swaps */
long ru_inblock; /* block input operations */
long ru_oublock; /* block output operations */
long ru_msgsnd; /* messages sent */
long ru_msgrcv; /* messages received */
long ru_nsignals; /* signals received */
long ru_nvcsw; /* voluntary context switches */
long ru_nivcsw; /* involuntary context switches */
};
Here is an example of measuring memory used by process on Windows.
#include <windows.h>
#include <Psapi.h>
// [...]
PROCESS_MEMORY_COUNTERS memCounter;
BOOL result = K32GetProcessMemoryInfo(GetCurrentProcess(), &memCounter, sizeof(memCounter));
std::cout << "WorkingSetSize " << memCounter.WorkingSetSize << std::endl;
And explanations of returned values https://learn.microsoft.com/en-gb/windows/win32/api/psapi/ns-psapi-process_memory_counters
I wanted this today, myself, so sharing the tested results here. I believe a call to getmem() will do what the OP asked, on any unix box. Written in very generic C, it will work in C or C++.
// Calling function must free the returned result.
char* exec(const char* command) {
FILE* fp;
char* line = NULL;
// Following initialization is equivalent to char* result = ""; and just
// initializes result to an empty string, only it works with
// -Werror=write-strings and is so much less clear.
char* result = (char*) calloc(1, 1);
size_t len = 0;
fflush(NULL);
fp = popen(command, "r");
if (fp == NULL) {
printf("Cannot execute command:\n%s\n", command);
return NULL;
}
while(getline(&line, &len, fp) != -1) {
// +1 below to allow room for null terminator.
result = (char*) realloc(result, strlen(result) + strlen(line) + 1);
// +1 below so we copy the final null terminator.
strncpy(result + strlen(result), line, strlen(line) + 1);
free(line);
line = NULL;
}
fflush(fp);
if (pclose(fp) != 0) {
perror("Cannot close stream.\n");
}
return result;
}
int getmem() {
pid_t pid = getpid();
char cmd[64];
snprintf(cmd, 64, "/bin/ps -p %d -o size", pid);
char* result = exec(cmd);
if (!result) {
return 0;
}
// Find first newline.
int pos = 0;
while (result[pos] != '\n') {
pos++;
}
// Remove the final newline.
result[strlen(result) - 1] = '\0';
// Convert to integer.
int size = atoi(result + pos + 1);
free(result);
return size;
}
Technically, I suppose the printf(...) line should be fprintf(stderr, ...), but I tend to have stderr redirected for certain environment-specific logging reasons, and this is how I compiled and tested the code, so I'm copying verbatim to avoid breakage.
Get your PID: pid_t getpid(void); // unistd.h
Parse /proc/<id>/smaps
If you don't care about shared libraries in mem total it may be simpler
make a system call to ps -p <id> -o %mem
Related
I understand that the disk space in linux could be programmatically retrieved using:
// header for statvfs
#include <sys/statvfs.h>
long GetAvailableSpace(const char* path)
{
struct statvfs stat;
if (statvfs(path, &stat) != 0) {
// error happens, just quits here
return -1;
}
// the available size is f_bsize * f_bavail
return stat.f_bsize * stat.f_bavail;
}
int main(int argc, const char *argv[])
{
// assuming input is the directory which one is interested
printf(" The remaining size is %ld \n", GetAvailableSpace(argv[1]));
}
However on checking the file structure for struct statvfs, they are
struct statvfs {
unsigned long f_bsize; /* filesystem block size */
unsigned long f_frsize; /* fragment size */
fsblkcnt_t f_blocks; /* size of fs in f_frsize units */
fsblkcnt_t f_bfree; /* # free blocks */
fsblkcnt_t f_bavail; /* # free blocks for unprivileged users */
fsfilcnt_t f_files; /* # inodes */
fsfilcnt_t f_ffree; /* # free inodes */
fsfilcnt_t f_favail; /* # free inodes for unprivileged users */
unsigned long f_fsid; /* filesystem ID */
unsigned long f_flag; /* mount flags */
unsigned long f_namemax; /* maximum filename length */
};
Theoretically the size of stat.f_bavail, could be also an unsigned long (I suspect). Wouldn't the multiplication of two unsigned long be much more than a long return type would hold?
I'm currently trying to write to a LTO tape device via C++. My problem is, that subsequent write requests will result in "Invalid argument" errors, while the first call seemingly works just fine.
I have set up the tape drive to use a fixed block size of 64k (Which means, that it can only be written to in chunks of 64k or 64k multiples at a time).
When now trying to write via fwrite() to it, the first call will be successful (or at least return that the requested amount of entries where written), but subsequent calls (with the same parameters) will result in no data being written (fwrite returns 0) and an error of "Invalid argument".
Following is a small sample application thrown together to replicate the issue.
#include <iostream>
#include <unistd.h>
#define BLOCK_SIZE (64 * 1024)
#define DATA_BLOCKS 32
#define byte unsigned char
int main(int argc, char *argv[]) {
if (argc <= 1) {
std::cout << "Missing first parameter: Target file\n";
return 1;
}
// Create file handle to access tape device in RW mode
FILE* handle;
const char* targetFile = argv[1];
if (access(targetFile, R_OK | W_OK) == 0) {
handle = fopen(targetFile, "r+");
printf("Create handle OK: %s\n", targetFile);
} else {
printf("Could not access %s: Missing rad or write permission\n", targetFile);
return 1;
}
// Create an byte array with data for DATA_BLOCKS blocks
byte *data = new byte[BLOCK_SIZE * DATA_BLOCKS];
// Initialize with some data
for (int i = 0; i < BLOCK_SIZE * DATA_BLOCKS; i++) {
data[i] = '5';
}
// Write data in BLOCK_SIZE chunks, blocksToWrite times per fwrite() call, in numberOfWriteCalls fwrite() calls
size_t blocksToWrite = 4;
int numberOfWriteCalls = 5;
size_t written;
for (int i = 0; i < numberOfWriteCalls; i++) {
written = fwrite(data, BLOCK_SIZE, blocksToWrite, handle);
printf("Round %d: Wrote %d entries of expected %d entries\n", i+1, (int) written, (int) blocksToWrite);
// Check if there was an error and if so, print error info to stderr
if (ferror(handle)) {
printf("Error while writing:\n");
perror("");
clearerr(handle);
}
/* Start modification: Added flushing, as pointed out by user7860670 */
fflush(handle);
// Check if there was an error and if so, print error info to stderr
if (ferror(handle)) {
printf("Error while flushing:\n");
perror("");
clearerr(handle);
}
/* End modification: Added flushing, as pointed out by user7860670 */
}
delete[] data;
// Close file handle
fclose(handle);
return 0;
}
This code calls the fwrite function 5 times, each time trying to write 4 blocks of 64k data.
This works fine on normal files, but returns the following output on all of my tape devices:
Create handle OK: /dev/nst0
Round 1: Wrote 4 entries of expected 4 entries
Round 2: Wrote 0 entries of expected 4 entries
Invalid argument
Round 3: Wrote 0 entries of expected 4 entries
Invalid argument
Round 4: Wrote 0 entries of expected 4 entries
Invalid argument
Round 5: Wrote 0 entries of expected 4 entries
Invalid argument
As can be seen, the first call reacts as expected: fwrite return that 4 blocks of data have been written. But all subsequent calls, even while using the same parameters, will return 0 written blocks and an error of "Invalid argument".
Is there any "special sauce" when trying to write to tape files that I'm not aware of, or am I maybe using the fwrite functionality in a wrong way?
Can anyone help me understand what these two lines are doing
buf = (char *)(malloc(2 * pagesize) & pagemask);
buf = (char *)(((long)buf + pagesize) & ∼pagemask);
I understand malloc but not sure what the & operation is trying to achieve in both expressions
Pagesize and pagemask are defined as follows earlier
pagesize = sysconf(_SC_PAGESIZE);
pagemask = pagesize - 1;
Thanks!
Edit1
This code is from a book "Unix FileSystems" by Steve D. Pate
Edit2
This is the full code
#include <sys/unistd.h>
#include <sys/types.h>
#include <fcntl.h>
#include “sys/fs/vx_ioctl.h”
#define MB (1024 * 1024)
main(int argc, char argv[])
{
char *buf;
int i, fd, advisory;
long pagesize, pagemask;
if (argc != 2) {
exit(1);
}
if (strcmp(argv[1], “VX_SEQ”) == 0) {
advisory = VX_SEQ;
} else if (strcmp(argv[1], “VX_RANDOM”) == 0) {
advisory = VX_RANDOM;
} else if (strcmp(argv[1], “VX_DIRECT”) == 0) {
advisory = VX_DIRECT;
}
pagesize = sysconf(_SC_PAGESIZE);
pagemask = pagesize - 1;
buf = (char *)(malloc(2 * pagesize) & pagemask);
buf = (char *)(((long)buf + pagesize) & ∼pagemask);
fd = open(“myfile”, O_RDWR);
ioctl(fd, VX_SETCACHE, advisory);
for (i=0 ; i<MB ; i++) {
read(fd, buf, 4096);
}
}
Preliminary Notes
Memory "pages" always (well, effectively always) have a size which is a power of 2. That means that, for a given page size, the upper bits of an address indicate a page, and the lower bits - an offset into the page.
Don't write this kind of code. Make explicit conversions; use multiple, simpler instructions; try using more meaningful variable names (e.g. offset_into_page = ((const uintptr_t) address) & page_mask;) and so on.
First line
The first line does the following:
Allocate 2 pages' worth of memory.
Treating the address as a number, keep only the bits of the offset within the page in which the allocated area begins.
It's not clear to me why this is useful.
Second line
The second line does the following:
Treat the address of buf as a number (although in a crooked, error-prone and non-portable way).
Move one page ahead from the address of buf.
Keep the page bits of the address - obtaining the first page-boundary-aligned address within buf.
This sacrifices some of the space allocated for buf, making it aligned (and keeping it in actually allocated space.
I'm trying to write a image via UART on Beaglebone Black. But when I use the write() function in library .
int write(int handle, void *buffer, int nbyte);
Regardless of the agurment nbyte has int type, I can not transfer 70kB at once. I displayed the number of byte which is transfered, and the result is the number of byte = 4111.
length = write(fd,body.c_str(),strlen(body.c_str())); //
cout<<length<<endl; // result length = 4111;
cout<<strlen(body.c_str())<<endl; // result strlen(body.c_str()) = 72255;
I hope to hear from you!
The write call does not assure that you can write the amount of data supplied, that's why it as an integer as its return, not a Boolean. The behavior you see is actually common among different operating systems, it may be due to the underline device might does not have sufficient buffer or storage for you to write 70kb. What you need is to write in a loop, each write will write the amount that is left unwritten:
int total = body.length(); // or strlen(body.c_str())
char *buffer = body.c_str();
int written = 0;
int ret;
while (written < total) {
ret = write(fd, buffer + written, total - written);
if (ret < 0) {
// error
break;
}
written += ret;
}
I made a custom source block that is reading switch values on a zedboard. It is accessing them via a proc driver that I wrote. The /var/log/kern.log is reporting proper output. The debug printf in the source block is reporting proper output.
However pushing the data to a filesink as well as a GUI number sink is only reading zeros. Did I not set up the block properly?
#ifdef HAVE_CONFIG_H
#include "config.h"
#endif
#include <gnuradio/io_signature.h>
#include "switches_impl.h"
#include <stdio.h>
#include <stdlib.h>
#include <uinstd.h>
namespace gr {
namespace zedboard {
switches::sptr
switches::make()
{
return gnuradio::get_initial_sptr
(new switches_impl());
}
/*
* The private constructor
*/
switches_impl::switches_impl()
: gr::block("switches",
gr::io_signature::make(0,0,0),
gr::io_signature::make(1, 1, sizeof(unsigned int *)))
{}
/*
* Our virtual destructor.
*/
switches_impl::~switches_impl()
{
}
void
switches_impl::forecast (int noutput_items, gr_vector_int &ninput_items_required)
{
/* <+forecast+> e.g. ninput_items_required[0] = noutput_items */
}
int
switches_impl::general_work (int noutput_items,
gr_vector_int &ninput_items,
gr_vector_const_void_star &input_items,
gr_vector_void_star &output_items)
{
//const <+ITYPE+> *in = (const <+ITYPE+> *) input_items[0];
unsigned int *out = (unsigned int *) output_items[0];
// Do <+signal processing+>
// Tell runtime system how many input items we consumed on
// each input stream.
char buffer[5];
size_t size = 1;
size_t nitems = 5;
FILE* fp;
fp = fopen("/proc/zedSwitches","r");
if (fp == NULL)
{
printf("Cannot open for read\n");
return -1;
}
/*
Expect return format:
0x00
*/
fread(buffer, size, nitems, fp);
fclose(fp);
out=(unsigned int *)strtoul(buffer,NULL,0);
printf("read: 0x%02x",out);
consume_each (noutput_items);
// Tell runtime system how many output items we produced.
return noutput_items;
}
} /* namespace zedboard */
} /* namespace gr */
A pointer is a pointer to data, usually:
unsigned int *out = (unsigned int *) output_items[0];
out refers to the buffer for your output.
But you overwrite that pointer with another pointer:
out=(unsigned int *)strtoul(buffer,NULL,0);
which just bends around your copy of that pointer, and doesn't affect the content of that buffer at all. Basic C!
You probably meant to say something like:
out[0]= strtoul(buffer,NULL,0);
That will put your value into the first element of the buffer.
However, you tell GNU Radio that you not only produced a single item (the line above), but noutput_items:
return noutput_items;
That must read
return 1;
when you're only producing a single item, or you must actually produce as many items as you return.
Your consume_each call is nonsensical – GNU Radio Sources are typically instances of gr::sync_block, which means that you'd write a work() instead of a general_work() method as you did.
From the fact alone that this is a general_work and not a work I'd say you haven't used gr_modtool (with block type set to source!) to generate the stub for this block – you really should. Again, I'd like to point you to the Guided Tutorials which should really quickly explain usage of gr_modtool as well as the underlying GNU Radio concepts.