The following code shows me getting the filesize of a certain file to then later on make a large enough buffer to ensure I can store all the files content in this buffer. So what I did was allocating it on the heap, because I couldn't know if the file is huge or not etc.
#include <iostream>
#include <string>
#include <cstdio>
#include <cstdlib>
size_t filesize(FILE* f) {
size_t size;
fseek(f, 0L, SEEK_END);
size = ftell(f);
fseek(f, 0L, SEEK_SET);
return size;
}
char* read_file(std::string name) {
FILE* f;
fopen_s(&f, name.c_str(), "rb");
size_t size = filesize(f);
char* buffer = new char[size+1];
memset(buffer, 0, size+1);
fread(buffer, sizeof(char), size+1, f);
fclose(f);
return buffer; //this is the buffer with the content to send
}
int main() {
char* buffer = read_file("main.cpp");
printf("%s", buffer);
delete[] buffer;
buffer = nullptr;
getchar();
return 0;
}
My question is, have I successfully deleted
char* buffer = new char[size+1];
from the heap by doing this:
char* buffer = read_file("main.cpp");
delete[] buffer;
buffer = nullptr;
Or does it still remain somewhere?
And if it does, how would I pin-point it and delete it?
Any other tips on how to handle raw pointers are appreciated as well.
Yes your code is correctly deleting the buffer.
C++ has various ways to handle this for you though so you don't need to worry about it and will be less likely to make mistakes and forget to free the buffer in some or all code paths, e.g. its easy to make mistakes like this:
int main()
{
char* buffer = read_file("main.cpp");
if ( buffer[0] != 'A' )
{
std::cout << "data is invalid\n";
return 1; // oops forgot to free buffer
}
delete[] buffer;
// data is valid
return 0;
}
One option is to use std::unique_ptr which will free the buffer for you when it goes out of scope:
#include <memory>
#include <string>
#include <iostream>
std::unique_ptr<char[]> read_file(std::string name) {
....
std::unique_ptr<char[]> buffer(new char[size+1]);
....
return buffer;
}
int main()
{
std::unique_ptr<char[]> buffer = read_file("main.cpp");
if ( buffer[0] != 'A' )
{
std::cout << "data is invalid\n";
return 1; // buffer is freed automatically
}
buffer.reset(); // can manually free if we are finished with buffer before it goes out of scope
// data is valid
return 0;
}
Related
This is the function which I read data from micro, but why when I allocate buffer by call new, the application crash, if I use malloc, it ok
void AlsaMicrophoneWrapper::readThreadFunction()
{
int bufSize = m_bitsPerFrame * m_frames;
// char *buf = new char(bufSize);//crash
char *buf = (char *)malloc(bufSize);
if (NULL == buf)
{
printf("Snd_ReadThread allocate mem error\n");
return;
}
snd_pcm_sframes_t retFrame;
ssize_t returnCode;
while (true)
{
retFrame = snd_pcm_readi(m_pcmHandle, buf, m_frames);
if (-EPIPE == retFrame)
{
snd_pcm_prepare(m_pcmHandle);
}
else if (retFrame > 0)
{
returnCode = m_writer->write(buf, retFrame);
if (returnCode <= 0)
{
printf("Failed to write to stream.\n");
}
}
}
free (buf);
return;
}
new char(bufSize) allocates a single char and initializes it to bufSize. You want new char[bufSize]. And when you new[] something, you must delete[] it later, not free it.
char *buf = new char[bufSize];
...
delete[] buf;
To avoid having to manage memory manually, you could use std::unique_ptr or std::vector.
auto buf = std::make_unique<char[]>(bufSize);
// use buf.get() to access the allocated memory
Or
std::vector<char> buf(bufSize);
// use buf.data() to access the allocated memory
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
I have a loop that brings me a buffer each time, I want to concatenate the buffers in a char array named data and then I want write this data into a file, but the problem is that I can't concatenate the char without seg-fault/core-dumped.
Here's the code (I also tried new char[sizeof] instead of malloc):
int size = getFrames() * 4;
char * buffer = (char *) malloc(size);
char * data;
char * tmp;
for (loops = 100; loops > 0; loops--) {
buffer = getBuffer();
tmp = (char *) malloc(sizeof(data));
tmp = data;
data = (char *) malloc(sizeof(tmp) + sizeof(buffer));
data = strcat(tmp, buffer);
}
free(tmp);
free(data);
free(buffer);
You are not managing your buffers correctly, not even close. In fact, there are a LOT of mistakes in your code. Every one of your sizeof() calls is wrong. You are leaking buffer, tmp, and data on every loop iteration. You are using strcat() incorrectly. And worse, you are processing binary audio data using string functions? No wonder why your code is failing.
Try something more like this instead:
char* getBuffer(int *bufsize)
{
...
*bufsize = ...; // <-- number of bytes being returned
return ... ; // <-- malloc()'ed pointer to actual bytes
}
...
char *data = NULL;
int datasize = 0;
int allocsize = 0;
char *buffer;
int bufsize;
for (loops = 100; loops > 0; loops--) {
buffer = getBuffer(&bufsize);
if ((datasize + bufsize) > allocsize)
{
// round new size up to next 1K boundary
int tmpsize = (datasize + bufsize + 1023) & ~1023;
char *tmp = (char*) realloc(data, tmpsize);
if (!tmp) {
free(buffer);
break;
}
data = tmp;
allocsize = tmpsize;
}
memcpy(data + datasize, buffer, bufsize);
datasize += bufsize;
free(buffer);
}
FILE *f = fopen(..., "wb");
fwrite(data, datasize, 1, f);
fclose(f);
free(data);
Or simpler:
char *buffer;
int bufsize;
FILE *f = fopen(..., "wb");
for (loops = 100; loops > 0; loops--) {
buffer = getBuffer(&bufsize);
if (fwrite(buffer, bufsize, 1, f) != 1) {
free(buffer);
break;
}
free(buffer);
}
fclose(f);
However, you tagged your question as C++, even though you are not actually using C++ code. The C++ way to handle this would look more like this instead:
#include <vector>
#include <string>
#include <fstream>
void getBuffer(std::vector<char> &buffer)
{
...
buffer.resize(...); // <-- number of bytes
// fill buffer with bytes as needed...
}
...
std::string data;
for (loops = 100; loops > 0; loops--) {
std::vector<char> buffer;
getBuffer(buffer);
data.append(buffer.data(), buffer.size());
}
std::ofstream f(..., std::ios_base::binary);
f.write(data.c_str(), data.size());
f.close();
Or simpler:
#include <string>
#include <fstream>
void appendBuffer(std::string &buffer)
{
...
buffer.append(...); // <-- append bytes
}
...
std::string data;
for (loops = 100; loops > 0; loops--) {
appendBuffer(data);
}
std::ofstream f(..., std::ios_base::binary);
f.write(data.c_str(), data.size());
f.close();
Or simpler:
#include <fstream>
bool outputBuffer(std::ostream &out)
{
...
return out.write(...); // <-- write bytes
}
...
std::ofstream f(..., std::ios_base::binary);
for (loops = 100; loops > 0; loops--) {
if (!outputBuffer(f)) break;
}
f.close();
I'm having trouble printing the contents of a file to console.
file.bin contents are "abc".
data holds value, but it just doesn't print it...
#include <Windows.h>
#include <iostream>
int main()
{
wchar_t *data;
FILE* file;
int err = _wfopen_s(&file, L"file.bin", L"rb");
if (err != 0)
{
std::cout << "Error";
return 0;
}
fseek(file, 0, SEEK_END);
long lSize;
lSize = ftell(file);
rewind(file);
data = (wchar_t *)malloc(lSize + 1);
fread(data, 1, lSize, file);
//dereference pointer
wchar_t data2 = *data;
std::wcout << data2; // prints nothing...
system("PAUSE");
return 0;
}
EDIT
I know about fstream but I would really prefer C style opening/reading files.
#include <fstream>
#include <string>
#include <iostream>
int main()
{
std::ifstream ifs("file.bin");
std::string content( (std::istreambuf_iterator<char>(ifs) ),
(std::istreambuf_iterator<char>() ) );
std::cout<<content;
return 0;
}
Use std::ifstream if you're using c++. You're making this much more complicated then you need to. See this former answer.
There is a usage example at the zlib website: http://www.zlib.net/zlib_how.html
However in the example they are compressing a file. I would like to compress a binary data stored in a buffer in memory. I don't want to save the compressed buffer to disk either.
Basically here is my buffer:
fIplImageHeader->imageData = (char*)imageIn->getFrame();
How can I compress it with zlib?
I would appreciate some code example of how to do that.
zlib.h has all the functions you need: compress (or compress2) and uncompress. See the source code of zlib for an answer.
ZEXTERN int ZEXPORT compress OF((Bytef *dest, uLongf *destLen, const Bytef *source, uLong sourceLen));
/*
Compresses the source buffer into the destination buffer. sourceLen is
the byte length of the source buffer. Upon entry, destLen is the total size
of the destination buffer, which must be at least the value returned by
compressBound(sourceLen). Upon exit, destLen is the actual size of the
compressed buffer.
compress returns Z_OK if success, Z_MEM_ERROR if there was not
enough memory, Z_BUF_ERROR if there was not enough room in the output
buffer.
*/
ZEXTERN int ZEXPORT uncompress OF((Bytef *dest, uLongf *destLen, const Bytef *source, uLong sourceLen));
/*
Decompresses the source buffer into the destination buffer. sourceLen is
the byte length of the source buffer. Upon entry, destLen is the total size
of the destination buffer, which must be large enough to hold the entire
uncompressed data. (The size of the uncompressed data must have been saved
previously by the compressor and transmitted to the decompressor by some
mechanism outside the scope of this compression library.) Upon exit, destLen
is the actual size of the uncompressed buffer.
uncompress returns Z_OK if success, Z_MEM_ERROR if there was not
enough memory, Z_BUF_ERROR if there was not enough room in the output
buffer, or Z_DATA_ERROR if the input data was corrupted or incomplete. In
the case where there is not enough room, uncompress() will fill the output
buffer with the uncompressed data up to that point.
*/
This is an example to pack a buffer with zlib and save the compressed contents in a vector.
void compress_memory(void *in_data, size_t in_data_size, std::vector<uint8_t> &out_data)
{
std::vector<uint8_t> buffer;
const size_t BUFSIZE = 128 * 1024;
uint8_t temp_buffer[BUFSIZE];
z_stream strm;
strm.zalloc = 0;
strm.zfree = 0;
strm.next_in = reinterpret_cast<uint8_t *>(in_data);
strm.avail_in = in_data_size;
strm.next_out = temp_buffer;
strm.avail_out = BUFSIZE;
deflateInit(&strm, Z_BEST_COMPRESSION);
while (strm.avail_in != 0)
{
int res = deflate(&strm, Z_NO_FLUSH);
assert(res == Z_OK);
if (strm.avail_out == 0)
{
buffer.insert(buffer.end(), temp_buffer, temp_buffer + BUFSIZE);
strm.next_out = temp_buffer;
strm.avail_out = BUFSIZE;
}
}
int deflate_res = Z_OK;
while (deflate_res == Z_OK)
{
if (strm.avail_out == 0)
{
buffer.insert(buffer.end(), temp_buffer, temp_buffer + BUFSIZE);
strm.next_out = temp_buffer;
strm.avail_out = BUFSIZE;
}
deflate_res = deflate(&strm, Z_FINISH);
}
assert(deflate_res == Z_STREAM_END);
buffer.insert(buffer.end(), temp_buffer, temp_buffer + BUFSIZE - strm.avail_out);
deflateEnd(&strm);
out_data.swap(buffer);
}
You can easily adapt the example by replacing fread() and fwrite() calls with direct pointers to your data. For zlib compression (referred to as deflate as you "take out all the air of your data") you allocate z_stream structure, call deflateInit() and then:
fill next_in with the next chunk of data you want to compress
set avail_in to the number of bytes available in next_in
set next_out to where the compressed data should be written which should usually be a pointer inside your buffer that advances as you go along
set avail_out to the number of bytes available in next_out
call deflate
repeat steps 3-5 until avail_out is non-zero (i.e. there's more room in the output buffer than zlib needs - no more data to write)
repeat steps 1-6 while you have data to compress
Eventually you call deflateEnd() and you're done.
You're basically feeding it chunks of input and output until you're out of input and it is out of output.
The classic way more convenient with C++ features
Here's a full example which demonstrates compression and decompression using C++ std::vector objects:
#include <cstdio>
#include <iosfwd>
#include <iostream>
#include <vector>
#include <zconf.h>
#include <zlib.h>
#include <iomanip>
#include <cassert>
void add_buffer_to_vector(std::vector<char> &vector, const char *buffer, uLongf length) {
for (int character_index = 0; character_index < length; character_index++) {
char current_character = buffer[character_index];
vector.push_back(current_character);
}
}
int compress_vector(std::vector<char> source, std::vector<char> &destination) {
unsigned long source_length = source.size();
uLongf destination_length = compressBound(source_length);
char *destination_data = (char *) malloc(destination_length);
if (destination_data == nullptr) {
return Z_MEM_ERROR;
}
Bytef *source_data = (Bytef *) source.data();
int return_value = compress2((Bytef *) destination_data, &destination_length, source_data, source_length,
Z_BEST_COMPRESSION);
add_buffer_to_vector(destination, destination_data, destination_length);
free(destination_data);
return return_value;
}
int decompress_vector(std::vector<char> source, std::vector<char> &destination) {
unsigned long source_length = source.size();
uLongf destination_length = compressBound(source_length);
char *destination_data = (char *) malloc(destination_length);
if (destination_data == nullptr) {
return Z_MEM_ERROR;
}
Bytef *source_data = (Bytef *) source.data();
int return_value = uncompress((Bytef *) destination_data, &destination_length, source_data, source.size());
add_buffer_to_vector(destination, destination_data, destination_length);
free(destination_data);
return return_value;
}
void add_string_to_vector(std::vector<char> &uncompressed_data,
const char *my_string) {
int character_index = 0;
while (true) {
char current_character = my_string[character_index];
uncompressed_data.push_back(current_character);
if (current_character == '\00') {
break;
}
character_index++;
}
}
// https://stackoverflow.com/a/27173017/3764804
void print_bytes(std::ostream &stream, const unsigned char *data, size_t data_length, bool format = true) {
stream << std::setfill('0');
for (size_t data_index = 0; data_index < data_length; ++data_index) {
stream << std::hex << std::setw(2) << (int) data[data_index];
if (format) {
stream << (((data_index + 1) % 16 == 0) ? "\n" : " ");
}
}
stream << std::endl;
}
void test_compression() {
std::vector<char> uncompressed(0);
auto *my_string = (char *) "Hello, world!";
add_string_to_vector(uncompressed, my_string);
std::vector<char> compressed(0);
int compression_result = compress_vector(uncompressed, compressed);
assert(compression_result == F_OK);
std::vector<char> decompressed(0);
int decompression_result = decompress_vector(compressed, decompressed);
assert(decompression_result == F_OK);
printf("Uncompressed: %s\n", uncompressed.data());
printf("Compressed: ");
std::ostream &standard_output = std::cout;
print_bytes(standard_output, (const unsigned char *) compressed.data(), compressed.size(), false);
printf("Decompressed: %s\n", decompressed.data());
}
In your main.cpp simply call:
int main(int argc, char *argv[]) {
test_compression();
return EXIT_SUCCESS;
}
The output produced:
Uncompressed: Hello, world!
Compressed: 78daf348cdc9c9d75128cf2fca495164000024e8048a
Decompressed: Hello, world!
The Boost way
#include <iostream>
#include <boost/iostreams/filtering_streambuf.hpp>
#include <boost/iostreams/copy.hpp>
#include <boost/iostreams/filter/zlib.hpp>
std::string compress(const std::string &data) {
boost::iostreams::filtering_streambuf<boost::iostreams::output> output_stream;
output_stream.push(boost::iostreams::zlib_compressor());
std::stringstream string_stream;
output_stream.push(string_stream);
boost::iostreams::copy(boost::iostreams::basic_array_source<char>(data.c_str(),
data.size()), output_stream);
return string_stream.str();
}
std::string decompress(const std::string &cipher_text) {
std::stringstream string_stream;
string_stream << cipher_text;
boost::iostreams::filtering_streambuf<boost::iostreams::input> input_stream;
input_stream.push(boost::iostreams::zlib_decompressor());
input_stream.push(string_stream);
std::stringstream unpacked_text;
boost::iostreams::copy(input_stream, unpacked_text);
return unpacked_text.str();
}
TEST_CASE("zlib") {
std::string plain_text = "Hello, world!";
const auto cipher_text = compress(plain_text);
const auto decompressed_plain_text = decompress(cipher_text);
REQUIRE(plain_text == decompressed_plain_text);
}
This is not a direct answer on your question about the zlib API, but you may be interested in boost::iostreams library paired with zlib.
This allows to use zlib-driven packing algorithms using the basic "stream" operations notation and then your data could be easily compressed by opening some memory stream and doing the << data operation on it.
In case of boost::iostreams this would automatically invoke the corresponding packing filter for every data that passes through the stream.
Is there any way to create a memory buffer as a FILE*. In TiXml it can print the xml to a FILE* but i cant seem to make it print to a memory buffer.
There is a POSIX way to use memory as a FILE descriptor: fmemopen or open_memstream, depending on the semantics you want: Difference between fmemopen and open_memstream
I guess the proper answer is that by Kevin. But here is a hack to do it with FILE *. Note that if the buffer size (here 100000) is too small then you lose data, as it is written out when the buffer is flushed. Also, if the program calls fflush() you lose the data.
#include <stdio.h>
#include <stdlib.h>
int main(int argc, char **argv)
{
FILE *f = fopen("/dev/null", "w");
int i;
int written = 0;
char *buf = malloc(100000);
setbuffer(f, buf, 100000);
for (i = 0; i < 1000; i++)
{
written += fprintf(f, "Number %d\n", i);
}
for (i = 0; i < written; i++) {
printf("%c", buf[i]);
}
}
fmemopen can create FILE from buffer, does it make any sense to you?
I wrote a simple example how i would create an in-memory FILE:
#include <unistd.h>
#include <stdio.h>
int main(){
int p[2]; pipe(p); FILE *f = fdopen( p[1], "w" );
if( !fork() ){
fprintf( f, "working" );
return 0;
}
fclose(f); close(p[1]);
char buff[100]; int len;
while( (len=read(p[0], buff, 100))>0 )
printf(" from child: '%*s'", len, buff );
puts("");
}
C++ basic_streambuf inheritance
In C++, you should avoid FILE* if you can.
Using only the C++ stdlib, it is possible to make a single interface that transparently uses file or memory IO.
This uses techniques mentioned at: Setting the internal buffer used by a standard stream (pubsetbuf)
#include <cassert>
#include <cstring>
#include <fstream>
#include <iostream>
#include <ostream>
#include <sstream>
/* This can write either to files or memory. */
void write(std::ostream& os) {
os << "abc";
}
template <typename char_type>
struct ostreambuf : public std::basic_streambuf<char_type, std::char_traits<char_type> > {
ostreambuf(char_type* buffer, std::streamsize bufferLength) {
this->setp(buffer, buffer + bufferLength);
}
};
int main() {
/* To memory, in our own externally supplied buffer. */
{
char c[3];
ostreambuf<char> buf(c, sizeof(c));
std::ostream s(&buf);
write(s);
assert(memcmp(c, "abc", sizeof(c)) == 0);
}
/* To memory, but in a hidden buffer. */
{
std::stringstream s;
write(s);
assert(s.str() == "abc");
}
/* To file. */
{
std::ofstream s("a.tmp");
write(s);
s.close();
}
/* I think this is implementation defined.
* pusetbuf calls basic_filebuf::setbuf(). */
{
char c[3];
std::ofstream s;
s.rdbuf()->pubsetbuf(c, sizeof c);
write(s);
s.close();
//assert(memcmp(c, "abc", sizeof(c)) == 0);
}
}
Unfortunately, it does not seem possible to interchange FILE* and fstream: Getting a FILE* from a std::fstream
You could use the CStr method of TiXMLPrinter which the documentation states:
The TiXmlPrinter is useful when you
need to:
Print to memory (especially in non-STL mode)
Control formatting (line endings, etc.)
https://github.com/Snaipe/fmem is a wrapper for different platform/version specific implementations of memory streams
It tries in sequence the following implementations:
open_memstream.
fopencookie, with growing dynamic buffer.
funopen, with growing dynamic buffer.
WinAPI temporary memory-backed file.
When no other mean is available, fmem falls back to tmpfile()