Segmentation fault while declaring an iterator in c++ example of leveldb - c++

I was trying to apply an iteration to my leveldb file, but unfortunately I couldn't get the result. The problem that I'm facing is a segmentation fault in using the pointer of the iterator. I used gdb And I got that the problem is in the line
leveldb::Iterator* it = db->NewIterator(leveldb::ReadOptions());
(I added before some values to this file /tem/userDb , and the add is working good .)
#include <assert.h>
#include <leveldb/db.h>
#include <iostream>
#include <sstream>
using namespace std;
void iteration(leveldb::DB* db)
{
leveldb::Iterator* it = db->NewIterator(leveldb::ReadOptions());
for (it->SeekToFirst(); it->Valid(); it->Next())
{
cout << "key :" << it->key().ToString() << " : "
<< "value:" << it - > value().ToString() << endl;
}
if (false == it->status().ok())
{
cerr << "An error was found during the scan" << endl;
cerr << it->status().ToString() << endl;
}
delete it;
}
int main(int argc, char *argv[])
{
leveldb::DB* db;
leveldb::Options options;
options.create_if_missing = true;
// erase error if the database exists
options.error_if_exists = true;
leveldb::Status s = leveldb::DB::Open(options, "/tmp/userDb", &db);
if (!s.ok())
cerr << s.ToString() << endl;
iteration(db);
delete db;
}

Not familiar with the leveldb API, but you're using db regardless of whether s.ok() is true or false. I'd assume that if s.ok() is false, db is either NULL, or in a state such that iteration or other operations won't work.
You should change the code to:
if (!s.ok()) {
cerr << s.ToString() << endl;
return -1;
}

Related

std::system_error occurs at Protobuf ParseFromZeroCopyStream()

I'm testing protobuf with zlib compression.
I wrote some c++ sample code using protobuf 3.8.0, but the following error occurred at calling ParseFromZeroCopyStream() at Ubuntu.
terminate called after throwing an instance of 'std::system_error'
what(): Unknown error -1
(core dumped)
what can I do?
I tried to replace ParseFromZeroCopyStream() with ParseFromBoundedZeroCopyStream().
That results in no core dump, but ParseFromBoundedZeroCopyStream() returned false.
test.proto
syntax = "proto2";
package test;
message Msg
{
required uint32 data = 1;
}
test.cc
#include <iostream>
#include <google/protobuf/io/zero_copy_stream_impl.h>
#include <google/protobuf/io/zero_copy_stream_impl_lite.h>
#include <google/protobuf/io/gzip_stream.h>
#include "test.pb.h"
using namespace std;
using namespace google::protobuf;
using namespace test;
int main(void)
{
Msg srcMsg;
srcMsg.set_data(1);
long sSize = srcMsg.ByteSizeLong();
cout << "SerializedSize = " << sSize << endl;
char * compressedMsg = new char[sSize];
io::ArrayOutputStream aos(compressedMsg, sSize);
io::GzipOutputStream gos(&aos);
long cSize;
if (srcMsg.SerializeToZeroCopyStream(&gos) == true)
{
gos.Close();
cSize = aos.ByteCount();
cout << "compression success : " << cSize << " bytes" << endl;
}
else
{
cout << "compression error" << endl;
return 1;
}
Msg targetMsg;
io::ArrayInputStream ais(compressedMsg, cSize);
io::GzipInputStream gis(&ais);
if (targetMsg.ParseFromZeroCopyStream(&gis) == false)
{
cout << "decompression error" << endl;
}
else
{
cout << "decompression success : " << targetMsg.ByteSizeLong() << " bytes" << endl;
cout << "data = " << targetMsg.data() << endl;
}
delete[] compressedMsg;
return 0;
}
I expect that decompression succeeds.
You will need to learn to use a debugger to investigate further why exactly this "unknown error: -1" is thrown - if possible.
That being said, unknown library errors is sometimes caused by a failed memory allocation or in rarer cases some other ressource constraint like failing to start a thread/process, etc.

How to read data from AVRO file using C++ interface?

I'm attempting to write a simple program to extract some data from a bunch of AVRO files. The schema for each file may be different so I would like to read the files generically (i.e. without having to pregenerate and then compile in the schema for each) using the C++ interface.
I have been attempting to follow the generic.cc example but it assumes a separate schema where I would like to read the schema from each AVRO file.
Here is my code:
#include <fstream>
#include <iostream>
#include "Compiler.hh"
#include "DataFile.hh"
#include "Decoder.hh"
#include "Generic.hh"
#include "Stream.hh"
const std::string BOLD("\033[1m");
const std::string ENDC("\033[0m");
const std::string RED("\033[31m");
const std::string YELLOW("\033[33m");
int main(int argc, char**argv)
{
std::cout << "AVRO Test\n" << std::endl;
if (argc < 2)
{
std::cerr << BOLD << RED << "ERROR: " << ENDC << "please provide an "
<< "input file\n" << std::endl;
return -1;
}
avro::DataFileReaderBase dataFile(argv[1]);
auto dataSchema = dataFile.dataSchema();
// Write out data schema in JSON for grins
std::ofstream output("data_schema.json");
dataSchema.toJson(output);
output.close();
avro::DecoderPtr decoder = avro::binaryDecoder();
auto inStream = avro::fileInputStream(argv[1]);
decoder->init(*inStream);
avro::GenericDatum datum(dataSchema);
avro::decode(*decoder, datum);
std::cout << "Type: " << datum.type() << std::endl;
return 0;
}
Everytime I run the code, no matter what file I use, I get this:
$ ./avrotest twitter.avro
AVRO Test
terminate called after throwing an instance of 'avro::Exception'
what(): Cannot have negative length: -40 Aborted
In addition to my own data files, I have tried using the data files located here: https://github.com/miguno/avro-cli-examples, with the same result.
I tried using the avrocat utility on all of the same files and it works fine. What am I doing wrong?
(NOTE: outputting the data schema for each file in JSON works correctly as expected)
After a bunch more fooling around, I figured it out. You're supposed to use DataFileReader templated with GenericDatum. With the end result being something like this:
#include <fstream>
#include <iostream>
#include "Compiler.hh"
#include "DataFile.hh"
#include "Decoder.hh"
#include "Generic.hh"
#include "Stream.hh"
const std::string BOLD("\033[1m");
const std::string ENDC("\033[0m");
const std::string RED("\033[31m");
const std::string YELLOW("\033[33m");
int main(int argc, char**argv)
{
std::cout << "AVRO Test\n" << std::endl;
if (argc < 2)
{
std::cerr << BOLD << RED << "ERROR: " << ENDC << "please provide an "
<< "input file\n" << std::endl;
return -1;
}
avro::DataFileReader<avro::GenericDatum> reader(argv[1]);
auto dataSchema = reader.dataSchema();
// Write out data schema in JSON for grins
std::ofstream output("data_schema.json");
dataSchema.toJson(output);
output.close();
avro::GenericDatum datum(dataSchema);
while (reader.read(datum))
{
std::cout << "Type: " << datum.type() << std::endl;
if (datum.type() == avro::AVRO_RECORD)
{
const avro::GenericRecord& r = datum.value<avro::GenericRecord>();
std::cout << "Field-count: " << r.fieldCount() << std::endl;
// TODO: pull out each field
}
}
return 0;
}
Perhaps an example like this should be included with libavro...

failbit getting set with ifstream object, what can be the cause?

I have written a small C++ program to set a property in a text file. The implementation is as following:
#include <cstdio>
#include <fstream>
#include <iostream>
#include <string>
using namespace std;
const string PROPFILE = "./propfile";
const string TEMPFILE = PROPFILE + ".tmp";
int setProp(const string &key, const string &val) {
try {
ifstream original(PROPFILE, ios::in);
ofstream tempfile(TEMPFILE, ios::out | ios::trunc);
for (string line; getline(original, line); ) {
if (line.compare(0, key.size(), key) == 0 && line[key.size()] == '=') {
tempfile << key << '=' << val << endl;
} else {
tempfile << line << endl;
}
}
cout << "original.rdstate()" << original.rdstate() << endl;
original.close();
tempfile.close();
} catch (ifstream::failure e) {
cerr << e.what() << endl;
}
if (rename(TEMPFILE.c_str(), PROPFILE.c_str()) != 0) {
cout << "Could not move " + TEMPFILE << "to " << PROPFILE << endl;
return 1;
}
return 0;
}
int main(int argc, const char *argv[]) {
try {
return setProp(argv[1], argv[2]);
} catch (logic_error) {
cout << "Invalid args" << endl;
return 1;
}
}
However, when I try to compile and execute it from commandline via ./a.out TESTPROP TESTVALUE, the value IS set as expected in propfile but rdstate() returns 6 (which means failbit and eofbit are set), I can't understand why are they getting set, can somebody explain ?
Contents of propfile before running ./a.out TESTPROP TESTVALUE are:
TESTPROP=NOTHING
After running the progam:
TESTPROP=TESTVALUE
I'm just a student, please don't mind if it's a dumb question :)
This is expected behaviour, the failbit is set whenever there is a failure to read the expected value. Even if that failure is because of end of file.
For instance see here
If no characters were extracted for whatever reason (not even the
discarded delimiter), getline sets failbit and returns.

How to get value from char pointer member of a structure modified in another function

I passed a structure pointer to a function. What I need is to get the string "connection refused"(return from ssh_get_error()) in the main function.
#include <libssh/libssh.h>
#include <stdlib.h>
#include <stdio.h>
#include <iostream>
#include <cstring>
using namespace std;
typedef struct rtns {
bool state;
char* desc;
} rtns;
int connect_host(rtns *con_host) {
const char* desc;
char desc2[1000];
ssh_session my_ssh_session;
int rc;
my_ssh_session = ssh_new();
if (my_ssh_session == NULL)
exit(-1);
ssh_options_set(my_ssh_session, SSH_OPTIONS_HOST, "localhost");
rc = ssh_connect(my_ssh_session);
if (rc != SSH_OK) {
(*con_host).state = false;
desc = ssh_get_error(my_ssh_session);
strcpy(desc2, ssh_get_error(my_ssh_session));
(*con_host).desc = strdup(desc2);
cout << "from connect_host function->" <<con_host->desc << "\n";
} else {
(*con_host).state = true;
}
ssh_disconnect(my_ssh_session);
ssh_free(my_ssh_session);
}
int main() {
rtns con_host;
rtns *p = (rtns*) malloc(sizeof(struct rtns));
p = (rtns*) malloc(20);
connect_host(&con_host);
cout << "from main function->" << *(con_host.desc) << "\n\n";
}
Output-
from connect_host function->Connection refused
from main function->C
The problem is that i am only getting one character "C" in the main function. I have some idea that I am doing memory allocation wrong but i have no idea how to do it correctly. I also tried to shift the malloc allocation in con_host function but it didn't work either.
Think about what you are printing. You pass to the stream a char, not a char *:
cout << "from main function->" << *(con_host.desc) << "\n\n";
// *(con_host.desc) = *(char *) = char
// Above is a simplification but you get the point.
So long as con_host.desc is a null terminated c string you can print it like this:
cout << "from main function->" << con_host.desc << "\n\n";
Here is a live example.
It's not memory allocation error. It's caused by the output statement. Change to
cout << "from main function->" << con_host.desc << "\n\n";

Why does boost's managed_mapped_file :: shrink_to_fit behave differently on Windows and Linux?

This is about C ++ library boost.
The managed_mapped_file :: shrink_to_fit function works differently on Linux and Windows.
On Linux, this function succeeds even if the target instance exists.
However, on Windows, this function will fail if the target instance exists.
Is this correct behavior?
It seems correct to do the same behavior, is this a bug?
I put the sample code below.
Compilation environment
boost:version.1.65.1
Windows
VisualStudio2017
WSL(Ubuntu16.04)
Linux
UbuntuServer17.10,
Clang++5.0,
g++7.2.0
Compile with
clang++-5.0 -std=c++1z ./test.cpp -o test -lpthread
#define BOOST_DATE_TIME_NO_LIB
#include <boost/interprocess/managed_mapped_file.hpp>
#include <boost/interprocess/file_mapping.hpp>
#include <boost/interprocess/allocators/allocator.hpp>
#include <vector>
#include <iostream>
namespace bip = boost::interprocess;
using intAlloc = bip::allocator<int, bip::managed_mapped_file::segment_manager>;
using intVec = std::vector<int, intAlloc>;
int main() {
bip::managed_mapped_file *p_file_vec;
intVec *vecObj;
std::string fileName = "tmp.dat";
size_t fileSize = 1024 * 1024 * 1;
bip::file_mapping::remove(fileName.c_str());
p_file_vec = new bip::managed_mapped_file(bip::create_only, fileName.c_str(), fileSize);
vecObj = p_file_vec->construct<intVec>("myVecName")(p_file_vec->get_allocator<int>());
for (size_t i = 0; i < 10; i++)
{
vecObj->push_back(1 + 100);
}
p_file_vec->flush();
try
{ //Fail when execute on Windows(WSL),but Success on Linux(Ubuntu17.10).
std::cout << "try to shrink:pointer has existed yet!" << std::endl;
bip::managed_mapped_file::shrink_to_fit(fileName.c_str());
std::cout << "success to shrink!" << std::endl;
}
catch (const boost::interprocess::interprocess_exception &ex)
{
std::cerr << "fail to shrink!" << std::endl;
std::cerr << ex.what() << std::endl;;
}
std::cout <<"please pless enter key."<< std::endl;
std::cin.get();
try
{ //Success when execute on Windows(WSL) and Linux(Ubuntu17.10).
delete p_file_vec;
std::cout << "try to shrink:pointer has deleted!" << std::endl;
bip::managed_mapped_file::shrink_to_fit(fileName.c_str());
std::cout << "success to shrink!" << std::endl;
}
catch (const std::exception& ex)
{
std::cerr << "fail to shrink!" << std::endl;
std::cerr << ex.what() << std::endl;;
}
std::cout << "please pless enter key." << std::endl;
std::cin.get();
}
Don't use new and delete in C++ (rule of thumb).
Apart from that
delete p_file_vec;
does NOT delete anything physical. It effectively disconnects from the mapped file. This is also why shrink_to_fit works: the documentation explicitly says:
If the application can find a moment where no process is attached it can grow or shrink to fit the managed segment.
And here
So, in short: the behaviour is correct on both platforms. It's just UNDEFINED what happens in your case when you shrink while the mapped file is in use (on Ubuntu).
Fixed Code:
Live On Coliru
#include <boost/interprocess/managed_mapped_file.hpp>
#include <iostream>
#include <vector>
namespace bip = boost::interprocess;
using intAlloc = bip::allocator<int, bip::managed_mapped_file::segment_manager>;
using intVec = std::vector<int, intAlloc>;
int main() {
std::string const fileName = "tmp.dat";
bip::file_mapping::remove(fileName.c_str());
{
bip::managed_mapped_file file_vec(bip::create_only, fileName.c_str(), 1l << 20);
auto *vecObj = file_vec.construct<intVec>("myVecName")(file_vec.get_allocator<int>());
for (size_t i = 0; i < 10; i++) {
vecObj->push_back(1 + 100);
}
}
try { // Success when execute on Windows(WSL) and Linux(Ubuntu17.10).
std::cout << "try to shrink:pointer has deleted!" << std::endl;
bip::managed_mapped_file::shrink_to_fit(fileName.c_str());
std::cout << "success to shrink!" << std::endl;
} catch (const std::exception &ex) {
std::cerr << "fail to shrink!" << std::endl;
std::cerr << ex.what() << std::endl;
;
}
}