I have a problem where I try to compress a file's data. Everything works up to the compression call, but it isn't the compression call itself, as the segfault is thrown before it. Showing my code will make it much clearer:
std::cout << "FILENAME: ";
std::cin >> filename;
if(!fileExists(filename))
{
std::cout << "ERR: FILE NOT FOUND." << std::endl;
continue;
}
std::cout << "Compressing file data...";
writeFile(filename, zlib_compress(readFile(filename)));
std::cout << " Done." << std::endl;
At the function zlib_compress...
std::string zlib_compress(const std::string& str)
{
std::cout << "DEBUG" << std::endl;
z_stream zs; // z_stream is zlib's control structure
memset(&zs, 0, sizeof(zs));
if (deflateInit(&zs, 9) != Z_OK)
std::cout << "deflateInit failed while compressing." << std::endl;
zs.next_in = (Bytef*)str.data();
zs.avail_in = str.size(); // set the z_stream's input
int ret;
char outbuffer[1073741824];
std::string outstring;
// retrieve the compressed bytes blockwise
do
{
zs.next_out = reinterpret_cast<Bytef*>(outbuffer);
zs.avail_out = sizeof(outbuffer);
ret = deflate(&zs, Z_FINISH);
if (outstring.size() < zs.total_out)
{
// append the block to the output string
outstring.append(outbuffer, zs.total_out - outstring.size());
}
} while(ret == Z_OK);
deflateEnd(&zs);
if(ret != Z_STREAM_END) // an error occurred that was not EOF
{
std::ostringstream oss;
oss << "Exception during zlib compression: (" << ret << ") " << zs.msg;
std::cout << oss.str();
}
return outstring;
}
I know, I know, that function needs work, I just C&P'd from somewhere to try it out.
But the thing is this:
std::cout << "DEBUG" << std::endl; is never called. The compiler says that the seg fault is coming from here:
std::string zlib_compress(const std::string& str)
> {
But why...? It was working earlier. I just don't know what went wrong!
Edit: Debugger output.
#0 00000000 0x00402cbb in __chkstk_ms() (??:??)
#1 004013BE zlib_compress(str=...) (C:\Users\***\Documents\Work\Programming\Compressor\z.cpp:5)
#2 00401DDA _fu15___ZSt4cout() (C:\Users\***\Documents\Work\Programming\Compressor\main.cpp:80)
char outbuffer[1073741824];
That's too large to put on the stack
You are taking a constant reference to a string as a parameter in your zlib_compress - you need to make sure that memory is available (whatever is returned from your readfile) in your zlib_compress. It would be good if you can share the prototype of your readFile function too.
Related
I'm trying to use libzip in a program that needs to archive several data chunks in different files. At the moment I have a code similar to the following snippet, edited from in-memory.c example in libzip examples.
The zip file is correctly saved with the files inside, but each file contains garbage.
Any help is appreciated.
bool push_files(zip_t* za) {
for (int i = 0; i < 10; i++) {
// Generate data
std::stringstream ss;
ss << "Test file #" << i;
std::string a = ss.str();
zip_source_t* source = zip_source_buffer(za, a.c_str(), a.size(), 0);
if (source == NULL) {
std::cerr << "error creating source: " << zip_strerror(za) << std::endl;
return false;
}
// Add buffer with filename
std::stringstream fname;
fname << "TEST-" << i;
a = fname.str();
if (zip_file_add(za, a.c_str(), source, ZIP_FL_ENC_UTF_8) < 0) {
std::cerr << "error adding source: " << zip_strerror(za) << std::endl;
return false;
}
}
return true;
}
int main() {
zip_source_t* src;
zip_error_t error;
zip_t* za;
zip_error_init(&error);
if ((src = zip_source_buffer_create(NULL, 0, 1, &error)) == NULL) {
std::cerr << "can't create source: " << zip_error_strerror(&error) << std::endl;
zip_error_fini(&error);
return 1;
}
if ((za = zip_open_from_source(src, ZIP_TRUNCATE, &error)) == NULL) {
std::cerr << "can't open zip from source: " << zip_error_strerror(&error) << std::endl;
zip_source_free(src);
zip_error_fini(&error);
return 1;
}
zip_error_fini(&error);
zip_source_keep(src);
if (!push_files(za))
return -1;
if (zip_close(za) < 0) {
std::cerr << "can't close zip archive" << zip_strerror(za) << std::endl;
return 1;
}
// ... omissis, save archive to file as in in-memory.c
}
zip_source_buffer does not copy the data out of the buffer - it just creates a zip_source_t which points to the same buffer. So you must keep the buffer alive until you're done adding the file.
Your code does not keep the buffer alive. The buffer you use is a.c_str() which is the data buffer of the string a. Fair enough, so far. But then before adding the file, you reassign the variable a = fname.str(); which (probably) frees that buffer and allocates a new one.
Solution: use a separate variable for the filename. Don't overwrite a until the file has been added.
I have two processes. One writes to a file, one has to read from it (At the same time..). So there's two fstreams open at a given time for the file (Although they may be in different processes).
I wrote a simple test function to crudely implement the sort of functionality I need:
void test_file_access()
{
try {
std::string file_name = "/Users/xxxx/temp_test_folder/test_file.dat";
std::ofstream out(file_name,
std::ios_base::out | std::ios_base::app | std::ios_base::binary);
out.write("Hello\n", 7);
std::this_thread::sleep_for(std::chrono::seconds(1));
std::array<char, 4096> read_buf;
std::ifstream in(file_name,
std::ios_base::in | std::ios_base::binary);
if (in.fail()) {
std::cout << "Error reading file" << std::endl;
return;
}
in.exceptions(std::ifstream::failbit | std::ifstream::badbit);
//Exception at the below line.
in.read(read_buf.data(), read_buf.size());
auto last_read_size = in.gcount();
auto offset = in.tellg();
std::cout << "Read [" << read_buf.data() << "] from file. read_size = " << last_read_size
<< ", offset = " << offset << std::endl;
out.write("World\n", 7);
std::this_thread::sleep_for(std::chrono::seconds(1));
//Do this so I can continue from the position I was before?
//in.clear();
in.read(read_buf.data(), read_buf.size());
last_read_size = in.gcount();
offset = in.tellg();
std::cout << "Read [" << read_buf.data() << "] from file. read_size = " << last_read_size
<< ", offset = " << offset << std::endl;
//Remove if you don't have boost.
boost::filesystem::remove(file_name);
}
catch(std::ios_base::failure const & ex)
{
std::cout << "Error : " << ex.what() << std::endl;
std::cout << "System error : " << strerror(errno) << std::endl;
}
}
int main()
{
test_file_access();
}
Run, and the output is like this:
Error : ios_base::clear: unspecified iostream_category error
System error : Operation timed out
So two questions,
What is going wrong here? Why do I get an Operation timed out error?
Is this an incorrect attempt to do what I need to get done? If so, what are the problems here?
You write into this file 7 bytes, but then try to read 4096 bytes. So in stream will read only 7 bytes and throw an exception as requested. Note that if you catch this exception the rest of the code will be executed correctly, e.g. last_read_size will be 7 and you can access those 7 bytes in buffer.
Hello I'm writing a short program to implement a shell and I'm running into an unusual problem. For some reason I can't clear to the std::cout buffer. The program wont print out messages. I understand a simple solution is to switch to std::cerr, but is there a way to get messages to print with cout?
Things I've tryed:
std::cout.flush()
Inserting std::endl after anything is written to standard out.
Inserting an std::flush into the output stream
std::cout.setf(std::ios::unitbuf); which was something I found that should unbuffer output.
Any help is much appreciated here is my code:
int main()
{
//Tryed this to unbuffer cout, no luck.
std::cout.setf(std::ios::unitbuf);
std::string input;
//Print out shell prompt and read in input from keyboard.
std::cout << "myshell> ";
std::getline(std::cin, input);
//**********************************************************************
//Step 1) Read in string and parse into tokens.
//**********************************************************************
char * buf = new char[input.length() + 1];
strcpy(buf, input.c_str());
int index = 0;
char * command[256];
command[index] = std::strtok(buf, " "); //Get first token.
std::cout << command[index] << std::endl;
while (command[index] != NULL)
{
++index;
command[index] = std::strtok(NULL," "); //Get remaining tokens.
std::cout << command[index] << std::endl;
}
std::cout.flush(); //No luck here either
//HERE IS WHERE MY PROBLEM IS.
std::cout << index << " items were added to the command array" << std::endl;
delete[] buf;
return 0;
}
The problem is that you're sending NULL to cout on the last iteration of the while loop, which leads to UB, and in your case is jamming cout. Check for NULL before you send anything to cout and you're fine:
if (command[index] != NULL) {
std::cout << command[index] << std::endl;
}
If you ever need to know what happened to your streams, remember that they can carry status information (the iostate, which I recommend you read about). The following code could have helped track your error:
try {
std::cout.exceptions(std::cout.failbit);
} catch(const std::ios_base::failure& e) {
std::cerr << "stream error: " << e.what() << std::endl;
std::cout.clear();
}
// continue working with cout, because std::cout.clear() removed
// failbit
Or, even simpler:
if(not std::cout) {
// address your error (if it is recoverable)
}
This is how your code would have looked like:
#include <cstring>
#include <string>
#include <iostream>
int main()
{
//Tryed this to unbuffer cout, no luck.
std::cout.setf(std::ios::unitbuf);
std::string input;
//Print out shell prompt and read in input from keyboard.
std::cout << "myshell> ";
std::getline(std::cin, input);
//**********************************************************************
//Step 1) Read in string and parse into tokens.
//**********************************************************************
char * buf = new char[input.length() + 1];
strcpy(buf, input.c_str());
int index = 0;
char * command[256];
command[index] = std::strtok(buf, " "); //Get first token.
std::cout << command[index] << std::endl;
while (command[index] != NULL)
{
++index;
command[index] = std::strtok(NULL," "); //Get remaining tokens.
std::cout << command[index] << std::endl;
}
// I added from here...
if(not std::cout) {
std::cerr << "cout is messed up... fixing it..." << std::endl;
std::cout.clear();
}
// ... to here.
std::cout.flush(); //No luck here either
//HERE IS WHERE MY PROBLEM IS.
std::cout << index << " items were added to the command array" << std::endl;
delete[] buf;
return 0;
}
Result:
$ ./a.out
myshell> 1 2 3
1
2
3
cout is messed up... fixing it...
3 items were added to the command array
Using GCC 4.7.3 on Cygwin 1.7.24. Compiler options include: -std=gnu++11 -Wall -Wextra
I am working on a command line application and I needed to be able to load and save a set of strings so I wrote a quick wrapper class around std::set to add load and save methods.
// KeySet.h
#ifndef KEYSET_H
#define KEYSET_H
#include <cstdlib>
#include <sys/stat.h>
#include <cerrno>
#include <cstring>
#include <string>
#include <set>
#include <iostream>
#include <fstream>
inline bool file_exists (const std::string& filename)
{
/*
Utility routine to check existance of a file. Returns true or false,
prints an error and exits with status 2 on an error.
*/
struct stat buffer;
int error = stat(filename.c_str(), &buffer);
if (error == 0) return true;
if (errno == ENOENT) return false;
std::cerr << "Error while checking for '" << filename << "': " << strerror(errno) << std::endl;
exit (2);
}
class KeySet
{
private:
std::string filename;
std::set<std::string> keys;
public:
KeySet() {}
KeySet(const std::string Pfilename) : filename(Pfilename) {}
void set_filename (const std::string Pfilename) {filename = Pfilename;}
std::string get_filename () {return filename;}
auto size () -> decltype(keys.size()) {return keys.size();}
auto cbegin() -> decltype(keys.cbegin()) {return keys.cbegin();}
auto cend() -> decltype(keys.cend()) {return keys.cend();}
auto insert(const std::string key) -> decltype(keys.insert(key)) {return keys.insert(key);}
void load ();
void save ();
};
void KeySet::load ()
{
if (file_exists(filename)) {
errno = 0;
std::ifstream in (filename, std::ios_base::in);
if (in.fail()) {
std::cerr << "Error opening '" << filename << "' for reading: " << strerror(errno) << std::endl;
exit (2);
}
std::string token;
if (token.capacity() < 32) token.reserve(32);
while (in >> token) keys.insert(token);
if (!in.eof()) {
std::cerr << "Error reading '" << filename << "': " << strerror(errno) << std::endl;
exit (2);
}
in.clear(); // need to clear flags before calling close
in.close();
if (in.fail()) {
std::cerr << "Error closing '" << filename << "': " << strerror(errno) << std::endl;
exit (2);
}
}
}
void KeySet::save ()
{
errno = 0;
std::ofstream out (filename, std::ios_base::out);
if (out.fail()) {
std::cerr << "Error opening '" << filename << "' for writing: " << strerror(errno) << std::endl;
exit (2);
}
for (auto key = keys.cbegin(), end = keys.cend(); key != end; ++key) {
out << *key << std::endl;
}
out.close();
if (out.fail()) {
std::cerr << "Error writing '" << filename << "': " << strerror(errno) << std::endl;
exit (2);
}
}
#endif
//
Here's a quick program to test the load method.
// ks_test.cpp
#include "KeySet.h"
int main()
{
KeySet test;
std::string filename = "foo.keys.txt";
test.set_filename(filename);
test.load();
for (auto key = test.cbegin(), end = test.cend(); key != end; ++key) {
std::cout << *key << std::endl;
}
}
The data file just has "one two three" in it.
When I go to run the test program, I get the following error from my test program:
$ ./ks_test
Error closing 'foo.keys.txt': No error
Both cppreference.com and cplusplus.com say that the close method should set the fail bit on error. The save method works fine, and the load method works correctly if I comment out the error check after the close. Should this really work or have I misunderstood how close is supposed to work? Thanks in advance.
Edited to clarify, fix typo's and adjust code per Joachim Pileborg's and Konrad Rudolph's comments.
Edited to add solution to the code.
You have two errors here: The first is about how you do your reading, more specifically the loop for reading. The eof flag will not be set until after you tried to read and the read failed. Instead you should do like this:
while (in >> token) { ... }
Otherwise you will loop one time to many and try to read beyond the end of the file.
The second problem is the one you notice, and it depends on the the first problem. Since you try to read beyond the end of the file, the stream will set failbit causing in.fail() to return true even though there is no real error.
As it turns out, the close method for ifstream (and I assume all other IO objects) DOES NOT clear the error flags before closing the file. This means you need to add an explicit clear() call before you close the stream after end of file if you are checking for errors during the close. In my case, I added in.clear(); just before the in.close(); call and it is working as I expect.
I'm writing program using Boost::Asio, I want to implement simple chat. Currently I'm struggling with problem that when I put some code inline of class function it's working, but when same code is provided by another class object is not. I think it could be connected to Boost::Asio, but I'm not sure:
void Connection::session(socket_ptr sock)
{
try{
for(;;) {
char mesg[1024];
boost::system::error_code error;
size_t length = sock->read_some(boost::asio::buffer(mesg), error);
if (error == boost::asio::error::eof){
disconnected(sock);
break;
}
else if (error)
throw boost::system::system_error(error);
message msg(mesg,length);
char *data;
data = msg.getData();
std::cout << "In session " << strlen(data) << " " << data;
/*This is working
string s_msg,s_data;
s_msg = mesg;
s_data = s_msg.substr(2,length);
std::vector<char> writable(s_data.size() + 1);
std::copy(s_data.begin(), s_data.end(), writable.begin());
std::cout << "In session " << strlen(&writable[0]) << " " << &writable[0];
send_all(sock,&writable[0],strlen(&writable[0]));
*/
send_all(sock,data,strlen(data));
}
}
catch (std::exception& e){
std::cerr << "Exception in thread: " << e.what() << "\n";
}
}
Class message that is only parsing data
message::message(char *c_msg, size_t length)
{
msg = c_msg;
id = msg.at(0);
data = msg.substr(2,length);
}
char* message::getData()
{
std::vector<char> writable(data.size() + 1);
std::copy(data.begin(), data.end(), writable.begin());
std::cout << data;
std::cout << &writable[0];
return &writable[0];
}
So when using class message this line:
std::cout << "In session " << strlen(data) << " " << data;
I get:
st String //this is cout from message getData
st String //this is cout from message getData
In session 0
With inline version:
In session 11 st String
So, in session function string is empty although message cout shows something opposite.
I don't know if it's important, but this function is invoked as new thread from main.
Regards,
Piotr
You're returning the address of a temporary:
char* message::getData()
{
std::vector<char> writable(data.size() + 1);
//...
return &writable[0];
}
This is undefined behaviour.
I'm assuming that data is just a std::string. You could do this instead:
const char* message::getData() const
{
return data.c_str();
}