Remaining disk space in linux and the limits of long - c++

I understand that the disk space in linux could be programmatically retrieved using:
// header for statvfs
#include <sys/statvfs.h>
long GetAvailableSpace(const char* path)
{
struct statvfs stat;
if (statvfs(path, &stat) != 0) {
// error happens, just quits here
return -1;
}
// the available size is f_bsize * f_bavail
return stat.f_bsize * stat.f_bavail;
}
int main(int argc, const char *argv[])
{
// assuming input is the directory which one is interested
printf(" The remaining size is %ld \n", GetAvailableSpace(argv[1]));
}
However on checking the file structure for struct statvfs, they are
struct statvfs {
unsigned long f_bsize; /* filesystem block size */
unsigned long f_frsize; /* fragment size */
fsblkcnt_t f_blocks; /* size of fs in f_frsize units */
fsblkcnt_t f_bfree; /* # free blocks */
fsblkcnt_t f_bavail; /* # free blocks for unprivileged users */
fsfilcnt_t f_files; /* # inodes */
fsfilcnt_t f_ffree; /* # free inodes */
fsfilcnt_t f_favail; /* # free inodes for unprivileged users */
unsigned long f_fsid; /* filesystem ID */
unsigned long f_flag; /* mount flags */
unsigned long f_namemax; /* maximum filename length */
};
Theoretically the size of stat.f_bavail, could be also an unsigned long (I suspect). Wouldn't the multiplication of two unsigned long be much more than a long return type would hold?

Related

How to use Viterbi decoder from GNU Radio library

I need to make Viterbi decoding of some convolutional-encoded signal. My application shall work with large files, therefore I cannot insert all the signal into a heap, so I need to process a data by a sequence of separate buffers. I have found a good library for Viterbi decoding - Encoder and a Viterbi decoder in C++ on the dr. Dobbs. I have applied the decoder from the libarary, it works correct, but doesn't provide a function for continuous use (call a function many times for each signal buffer with considering of previous calculations). Then I have found the GNU Radio C++ library which provide the necessary functions. But I don't understand how to use its functions, because it doesn't provide a documentation. It contains the example of Viterbi decoding with is present below:
extern "C" {
#include <gnuradio/fec/viterbi.h>
}
#include <cstdio>
#include <cmath>
#define MAXCHUNKSIZE 4096
#define MAXENCSIZE MAXCHUNKSIZE*16
int main()
{
unsigned char data[MAXCHUNKSIZE];
signed char syms[MAXENCSIZE];
int count = 0;
// Initialize metric table
int mettab[2][256];
int amp = 100; // What is it? ***
float RATE=0.5;
float ebn0 = 12.0;
float esn0 = RATE*pow(10.0, ebn0/10);
gen_met(mettab, amp, esn0, 0.0, 4);
// Initialize decoder state
struct viterbi_state state0[64];
struct viterbi_state state1[64];
unsigned char viterbi_in[16];
viterbi_chunks_init(state0);
while (!feof(stdin)) {
unsigned int n = fread(syms, 1, MAXENCSIZE, stdin);
unsigned char *out = data;
for (unsigned int i = 0; i < n; i++) {
// FIXME: This implements hard decoding by slicing the input stream
unsigned char sym = syms[i] > 0 ? -amp : amp; // What is it? ***
// Write the symbol to the decoder input
viterbi_in[count % 4] = sym;
// Every four symbols, perform the butterfly2 operation
if ((count % 4) == 3) {
viterbi_butterfly2(viterbi_in, mettab, state0, state1);
// Every sixteen symbols, perform the readback operation
if ((count > 64) && (count % 16) == 11) {
viterbi_get_output(state0, out);
fwrite(out++, 1, 1, stdout);
}
}
count++;
}
}
return 0;
}
File viterbi.c from it also contains the next function viterbi(), without a declaration:
/* Viterbi decoder */
int viterbi(unsigned long *metric, /* Final path metric (returned value) */
unsigned char *data, /* Decoded output data */
unsigned char *symbols, /* Raw deinterleaved input symbols */
unsigned int nbits, /* Number of output bits */
int mettab[2][256] /* Metric table, [sent sym][rx symbol] */
) { ...
Also I found one more implementation for Viterbi decoding - The Spiral project. But it also doesn't contain a normal description and doesn't want to compile. And two more implementation on the ExpertCore and Forward Error Correction DSP library.
My question: Can anyone understand how to use the above GNU Radio's implementation of the Viterbi algorithm for continuous use for binary interleaved digital signal (Encoder parameters of my signal: K=7 rate=1/2, every bit in my file is a demodulated sample of a signal)?

gnuradio source only outputting zeros

I made a custom source block that is reading switch values on a zedboard. It is accessing them via a proc driver that I wrote. The /var/log/kern.log is reporting proper output. The debug printf in the source block is reporting proper output.
However pushing the data to a filesink as well as a GUI number sink is only reading zeros. Did I not set up the block properly?
#ifdef HAVE_CONFIG_H
#include "config.h"
#endif
#include <gnuradio/io_signature.h>
#include "switches_impl.h"
#include <stdio.h>
#include <stdlib.h>
#include <uinstd.h>
namespace gr {
namespace zedboard {
switches::sptr
switches::make()
{
return gnuradio::get_initial_sptr
(new switches_impl());
}
/*
* The private constructor
*/
switches_impl::switches_impl()
: gr::block("switches",
gr::io_signature::make(0,0,0),
gr::io_signature::make(1, 1, sizeof(unsigned int *)))
{}
/*
* Our virtual destructor.
*/
switches_impl::~switches_impl()
{
}
void
switches_impl::forecast (int noutput_items, gr_vector_int &ninput_items_required)
{
/* <+forecast+> e.g. ninput_items_required[0] = noutput_items */
}
int
switches_impl::general_work (int noutput_items,
gr_vector_int &ninput_items,
gr_vector_const_void_star &input_items,
gr_vector_void_star &output_items)
{
//const <+ITYPE+> *in = (const <+ITYPE+> *) input_items[0];
unsigned int *out = (unsigned int *) output_items[0];
// Do <+signal processing+>
// Tell runtime system how many input items we consumed on
// each input stream.
char buffer[5];
size_t size = 1;
size_t nitems = 5;
FILE* fp;
fp = fopen("/proc/zedSwitches","r");
if (fp == NULL)
{
printf("Cannot open for read\n");
return -1;
}
/*
Expect return format:
0x00
*/
fread(buffer, size, nitems, fp);
fclose(fp);
out=(unsigned int *)strtoul(buffer,NULL,0);
printf("read: 0x%02x",out);
consume_each (noutput_items);
// Tell runtime system how many output items we produced.
return noutput_items;
}
} /* namespace zedboard */
} /* namespace gr */
A pointer is a pointer to data, usually:
unsigned int *out = (unsigned int *) output_items[0];
out refers to the buffer for your output.
But you overwrite that pointer with another pointer:
out=(unsigned int *)strtoul(buffer,NULL,0);
which just bends around your copy of that pointer, and doesn't affect the content of that buffer at all. Basic C!
You probably meant to say something like:
out[0]= strtoul(buffer,NULL,0);
That will put your value into the first element of the buffer.
However, you tell GNU Radio that you not only produced a single item (the line above), but noutput_items:
return noutput_items;
That must read
return 1;
when you're only producing a single item, or you must actually produce as many items as you return.
Your consume_each call is nonsensical – GNU Radio Sources are typically instances of gr::sync_block, which means that you'd write a work() instead of a general_work() method as you did.
From the fact alone that this is a general_work and not a work I'd say you haven't used gr_modtool (with block type set to source!) to generate the stub for this block – you really should. Again, I'd like to point you to the Guided Tutorials which should really quickly explain usage of gr_modtool as well as the underlying GNU Radio concepts.

Segmentation fault when saving a JPEG file (from an array of RGB data)

I find the follow code on internet for read and write a JPEG file using the library libjpeg.
I changed the function void write_JPEG_file (char * filename, int quality) to the following:
void write_JPEG_vetor (JSAMPLE * image_data, int height, int width, int quality)
{
printf("%s\n","write_JPEG_vetor");
/* This struct contains the JPEG compression parameters and pointers to
* working space (which is allocated as needed by the JPEG library).
* It is possible to have several such structures, representing multiple
* compression/decompression processes, in existence at once. We refer
* to any one struct (and its associated working data) as a "JPEG object".
*/
struct jpeg_compress_struct cinfo;
/* This struct represents a JPEG error handler. It is declared separately
* because applications often want to supply a specialized error handler
* (see the second half of this file for an example). But here we just
* take the easy way out and use the standard error handler, which will
* print a message on stderr and call exit() if compression fails.
* Note that this struct must live as long as the main JPEG parameter
* struct, to avoid dangling-pointer problems.
*/
struct jpeg_error_mgr jerr;
/* More stuff */
FILE * outfile; /* target file */
JSAMPROW row_pointer[1]; /* pointer to JSAMPLE row[s] */
int row_stride; /* physical row width in image buffer */
printf ("%s\n","Step 1: allocate and initialize JPEG compression object */");
/* We have to set up the error handler first, in case the initialization
* step fails. (Unlikely, but it could happen if you are out of memory.)
* This routine fills in the contents of struct jerr, and returns jerr's
* address which we place into the link field in cinfo.
*/
cinfo.err = jpeg_std_error(&jerr);
/* Now we can initialize the JPEG compression object. */
jpeg_create_compress(&cinfo);
printf ("%s\n","/* Step 2: specify data destination (eg, a file) */");
/* Note: steps 2 and 3 can be done in either order. */
/* Here we use the library-supplied code to send compressed data to a
* stdio stream. You can also write your own code to do something else.
* VERY IMPORTANT: use "b" option to fopen() if you are on a machine that
* requires it in order to write binary files.
*/
char * filename = {"novo_arquivo.jpeg"};
if ((outfile = fopen(filename, "wb")) == NULL) {
fprintf(stderr, "can't open %s\n", filename);
exit(1);
}
jpeg_stdio_dest(&cinfo, outfile);
printf ("%s\n","/* Step 3: set parameters for compression */");
/* First we supply a description of the input image.
* Four fields of the cinfo struct must be filled in:
*/
cinfo.image_width = width; /* image width and height, in pixels */
cinfo.image_height = height;
cinfo.input_components = 3; /* # of color components per pixel */
cinfo.in_color_space = JCS_RGB; /* colorspace of input image */
/* Now use the library's routine to set default compression parameters.
* (You must set at least cinfo.in_color_space before calling this,
* since the defaults depend on the source color space.)
*/
jpeg_set_defaults(&cinfo);
/* Now you can set any non-default parameters you wish to.
* Here we just illustrate the use of quality (quantization table) scaling:
*/
jpeg_set_quality(&cinfo, quality, TRUE /* limit to baseline-JPEG values */);
printf ("%s\n","/* Step 4: Start compressor */");
/* TRUE ensures that we will write a complete interchange-JPEG file.
* Pass TRUE unless you are very sure of what you're doing.
*/
jpeg_start_compress(&cinfo, TRUE);
printf ("%s\n","/* Step 5: while (scan lines remain to be written) */");
/* jpeg_write_scanlines(...); */
/* Here we use the library's state variable cinfo.next_scanline as the
* loop counter, so that we don't have to keep track ourselves.
* To keep things simple, we pass one scanline per call; you can pass
* more if you wish, though.
*/
row_stride = width * 3; /* JSAMPLEs per row in image_buffer */
while (cinfo.next_scanline < cinfo.image_height) {
printf ("%s\n","Loop WHILE");
/* jpeg_write_scanlines expects an array of pointers to scanlines.
* Here the array is only one element long, but you could pass
* more than one scanline at a time if that's more convenient.
*/
row_pointer[0] = &image_data[cinfo.next_scanline * row_stride];
(void) jpeg_write_scanlines(&cinfo, row_pointer, row_stride);
}
printf ("%s\n","/* Step 6: Finish compression */");
jpeg_finish_compress(&cinfo);
/* After finish_compress, we can close the output file. */
fclose(outfile);
printf ("%s\n","/* Step 7: release JPEG compression object */");
/* This is an important step since it will release a good deal of memory. */
jpeg_destroy_compress(&cinfo);
/* And we're done! */
}
Now, when I run the program (in a Linux enviroment), I am receiving an error Segmentation Fault. Someone can tell why this is happening? My main suspect is the code:
while (cinfo.next_scanline < cinfo.image_height) {
printf ("%s\n","Loop WHILE");
/* jpeg_write_scanlines expects an array of pointers to scanlines.
* Here the array is only one element long, but you could pass
* more than one scanline at a time if that's more convenient.
*/
row_pointer[0] = &image_data[cinfo.next_scanline * row_stride];
(void) jpeg_write_scanlines(&cinfo, row_pointer, row_stride);
}
but i'm not sure about that, and can't find a solution to solve this, despite spend a good time trying.
=== UPDATE ===
I included the follow debugging code in this part of the code:
while (cinfo.next_scanline < cinfo.image_height) {
printf ("%s\n","Loop WHILE");
/* jpeg_write_scanlines expects an array of pointers to scanlines.
* Here the array is only one element long, but you could pass
* more than one scanline at a time if that's more convenient.
*/
printf ("%s\n","parte 1.1");
row_pointer[0] = &image_data[cinfo.next_scanline * row_stride];
printf ("%s\n","parte 1.2");
printf ("%s\n","parte 2.1");
(void) jpeg_write_scanlines(&cinfo, row_pointer, 1);
printf ("%s\n","parte 2.2");
}
And this way the output when running the programa is:
Loop WHILE
parte 1.1
parte 1.2
parte 2.1
=== UPDATE 2 ===
For the record, in my program, this function is receiving the return value of this function:
JSAMPLE * inverte_imagem()
{
int tamanho = image_height*image_width*image_colors;
int i;
JSAMPLE * vetor = malloc(sizeof(JSAMPLE)*(image_height*image_width*image_colors));
for( i=0; i<tamanho; i++)
vetor [i] = image_buffer [tamanho - (i+1)];
}
This looks wrong:
(void) jpeg_write_scanlines(&cinfo, row_pointer, row_stride);
That last parameter is the number of lines to write, not the row length. You probably want:
(void) jpeg_write_scanlines(&cinfo, row_pointer, 1);
OK, I solve the problem putting the call for write_JPEG_vector inside the function inverte_imagem(). I don't know why, but when I make the call to this function from my main function, a memory problem (error segmentation fault on linux) occurs.

libJpeg garbled read

I'm using the example for the most part and it does create an image it's just garbled:
unsigned char* readJpeg(JNIEnv* env, libraw_processed_image_t *raw)
{
// http://sourceforge.net/p/libjpeg-turbo/code/HEAD/tree/trunk/example.c#l109
// http://stackoverflow.com/questions/5616216/need-help-in-reading-jpeg-file-using-libjpeg
struct jpeg_decompress_struct cinfo;
struct jpeg_error_mgr jerr;
int row_stride; /* physical row width in output buffer */
cinfo.err = jpeg_std_error(&jerr);
/* Now we can initialize the JPEG decompression object. */
jpeg_create_decompress(&cinfo);
/* Step 2: specify data source (eg, a file) */
jpeg_mem_src(&cinfo, raw->data, raw->data_size);
/* Step 3: read file parameters with jpeg_read_header() */
(void) jpeg_read_header(&cinfo, TRUE);
/* Step 4: set parameters for decompression */
/* In this example, we don't need to change any of the defaults set by
* jpeg_read_header(), so we do nothing here.
*/
/* Step 5: Start decompressor */
(void) jpeg_start_decompress(&cinfo);
/* We can ignore the return value since suspension is not possible
* with the stdio data source.
*/
/* We may need to do some setup of our own at this point before reading
* the data. After jpeg_start_decompress() we have the correct scaled
* output image dimensions available, as well as the output colormap
* if we asked for color quantization.
* In this example, we need to make an output work buffer of the right size.
*/
/* JSAMPLEs per row in output buffer */
row_stride = cinfo.output_width * cinfo.output_components;
JSAMPROW rowData;
unsigned char* imageData = new unsigned char[cinfo.output_height * row_stride];
/* Step 6: while (scan lines remain to be read) */
/* jpeg_read_scanlines(...); */
/* Here we use the library's state variable cinfo.output_scanline as the
* loop counter, so that we don't have to keep track ourselves.
*/
__android_log_write(ANDROID_LOG_INFO, "JNI", "Made it to read lines");
int row = 0;
while (cinfo.output_scanline < cinfo.output_height)
{
rowData = imageData + (row * row_stride);
jpeg_read_scanlines(&cinfo, &rowData, 1);
++row;
}
/* Step 7: Finish decompression */
(void) jpeg_finish_decompress(&cinfo);
/* We can ignore the return value since suspension is not possible
* with the stdio data source.
*/
/* Step 8: Release JPEG decompression object */
/* This is an important step since it will release a good deal of memory. */
jpeg_destroy_decompress(&cinfo);
/* At this point you may want to check to see whether any corrupt-data
* warnings occurred (test whether jerr.pub.num_warnings is nonzero).
*/
/* And we're done! */
return imageData;
}
The image I'm reading loads fine in existing readers. I'm guessing I'm missing some sort of decompress setting, though I though it would be getting those from the header.
I faced similar problem but then later figured out that you have to align the RGB array to 8 byte boundary. check http://atlc.sourceforge.net/bmp.html#_toc381201083

How to measure memory usage from inside a C++ program?

Is it possible, for a c++ program, to track how much memory the program is using at one time?
For example, a function with a prototype:
int getEstimatedTotalMemoryUsage();
I suppose if it's not possible, then one will have to get out of the program, do a system call and check the results from there. If so, what tools are available for such purposes? Assuming such a thing is possible, that is.
edit: I'm using linux, any tools that can do this for you?
Yes - use POSIX getrusage. From the Linux man page:
Synopsis
#include <sys/time.h>
#include <sys/resource.h>
int getrusage(int who, struct rusage *usage);
Description
getrusage() returns current resource usages, for a who of either RUSAGE_SELF or RUSAGE_CHILDREN. The former asks for resources used by the current process, the latter for resources used by those of its children that have terminated and have been waited for.
struct rusage {
struct timeval ru_utime; /* user time used */
struct timeval ru_stime; /* system time used */
long ru_maxrss; /* maximum resident set size */
long ru_ixrss; /* integral shared memory size */
long ru_idrss; /* integral unshared data size */
long ru_isrss; /* integral unshared stack size */
long ru_minflt; /* page reclaims */
long ru_majflt; /* page faults */
long ru_nswap; /* swaps */
long ru_inblock; /* block input operations */
long ru_oublock; /* block output operations */
long ru_msgsnd; /* messages sent */
long ru_msgrcv; /* messages received */
long ru_nsignals; /* signals received */
long ru_nvcsw; /* voluntary context switches */
long ru_nivcsw; /* involuntary context switches */
};
Here is an example of measuring memory used by process on Windows.
#include <windows.h>
#include <Psapi.h>
// [...]
PROCESS_MEMORY_COUNTERS memCounter;
BOOL result = K32GetProcessMemoryInfo(GetCurrentProcess(), &memCounter, sizeof(memCounter));
std::cout << "WorkingSetSize " << memCounter.WorkingSetSize << std::endl;
And explanations of returned values https://learn.microsoft.com/en-gb/windows/win32/api/psapi/ns-psapi-process_memory_counters
I wanted this today, myself, so sharing the tested results here. I believe a call to getmem() will do what the OP asked, on any unix box. Written in very generic C, it will work in C or C++.
// Calling function must free the returned result.
char* exec(const char* command) {
FILE* fp;
char* line = NULL;
// Following initialization is equivalent to char* result = ""; and just
// initializes result to an empty string, only it works with
// -Werror=write-strings and is so much less clear.
char* result = (char*) calloc(1, 1);
size_t len = 0;
fflush(NULL);
fp = popen(command, "r");
if (fp == NULL) {
printf("Cannot execute command:\n%s\n", command);
return NULL;
}
while(getline(&line, &len, fp) != -1) {
// +1 below to allow room for null terminator.
result = (char*) realloc(result, strlen(result) + strlen(line) + 1);
// +1 below so we copy the final null terminator.
strncpy(result + strlen(result), line, strlen(line) + 1);
free(line);
line = NULL;
}
fflush(fp);
if (pclose(fp) != 0) {
perror("Cannot close stream.\n");
}
return result;
}
int getmem() {
pid_t pid = getpid();
char cmd[64];
snprintf(cmd, 64, "/bin/ps -p %d -o size", pid);
char* result = exec(cmd);
if (!result) {
return 0;
}
// Find first newline.
int pos = 0;
while (result[pos] != '\n') {
pos++;
}
// Remove the final newline.
result[strlen(result) - 1] = '\0';
// Convert to integer.
int size = atoi(result + pos + 1);
free(result);
return size;
}
Technically, I suppose the printf(...) line should be fprintf(stderr, ...), but I tend to have stderr redirected for certain environment-specific logging reasons, and this is how I compiled and tested the code, so I'm copying verbatim to avoid breakage.
Get your PID: pid_t getpid(void); // unistd.h
Parse /proc/<id>/smaps
If you don't care about shared libraries in mem total it may be simpler
make a system call to ps -p <id> -o %mem