encrypting and serializing stl string and other containers - c++

I have data in stl containers (vector). Each node in the vector is a structure which also contains stl strings.
struct record
{
string name;
string location;
int salary;
}
vector< record > employees;
I want to serialize employees but I also want to encrypt it before serializing.
my encryption function looks like this:
Encode(const char * inBfr, const int in_size, char ** outBfr, int& out_size )
By searching it looks like the stl standard doesn't require the memory of my structure to be contiguous so I can't just grab the memory of employees variable. Is there any other smart way that I can use this encoding function with my stl based structures/container? It is good for me that Encode function works in plain char * buffers so I know exactly what goes in and out but stl structures are not and I am tring to find a nice way so I can use stl with this function.
I am also opening to using any other stl containers if that helps.

Although the element in the std::vector<T> are guaranteed to be laid out contiguously, this doesn't really help: the record you have may include padding and, more importantly, will store the std::string's content external to the std::string object (in case the small string optimization is used, the value may be embedded inside the std::string but it will also contain a couple of bytes which are not part of the std::strings value). Thus, you best option is to format your record and encrypt the formatted string.
The formatting is straight forward but personally I would encapsulate the encoding function into a simple std::streambuf so that the encryption can be done by a filtering stream buffer. Given the signature you gave, this could look something like this:
class encryptbuf
: public std::streambuf {
std::streambuf* d_sbuf;
char d_buffer[1024];
public:
encryptbuf(std::streambuf* sbuf)
: d_sbuf(sbuf) {
this->setp(this->d_buffer, this->d_buffer + sizeof(this->d_buffer) - 1);
}
int overflow(int c) {
if (c != std::char_traits<char>::eof()) {
*this->pptr() = std::char_traits<char>::to_char_type(c);
this->pbump(1);
}
return this->pubsync()? std::char_traits<char>::eof(): std::char_traits<char>::not_eof(c);
}
int sync() {
char* out(0);
int size(0);
Encode(this->pbase(), this->pptr() - this->pbase(), &out, size);
this->d_sbuf->sputn(out, size);
delete[] out; // dunno: it seems the output buffer is allocated but how?
this->setp(this->pbase(), this->epptr());
return this->d_sbuf->pubsync();
}
};
int main() {
encryptbuf sbuf(std::cout.rdbuf());
std::ostream eout(&sbuf);
eout << "print something encoded to standard output\n" << std::flush;
}
Now, creating an output operator for your records just printing to an std::ostream can be used to create an encoded

It's probably easiest to serialize your structure into a string, then encrypt the string. For example:
std::ostringstream buffer;
buffer << a_record.name << "\n" << a_record.location << "\n" << a_record.salary;
encode(buffer.str().c_str(), buffer.str().length(), /* ... */);
If it were me, I'd probably write encode (or at least a wrapper for it) to take input (and probably produce output) in a vector, string, or stream though.
If you want to get ambitious, there are other possibilities. First of all, #MooingDuck raises a good point that it's often worthwhile to overload operator<< for the class, instead of working with the individual items all the time. This will typically be a small function similar to what's above:
std::ostream &operator<<(std::ostream &os, record const &r) {
return os << r.name << "\n" << r.location << "\n" << r.salary;
}
Using this, you'd just have:
std::ostringstream os;
os << a_record;
encode(os.str().c_str(), os.str().length(), /* ... */);
Second, if you want to get really ambitious, you can put the encryption into (for one example) a codecvt facet, so you can automatically encrypt all the data as you write it to a stream, and decrypt it as you read it back in. Another possibility is to build the encryption into a filtering streambuf object instead. The codecvt facet is probably the method that should theoretically be preferred, but the streambuf is almost certainly easier to implement, with less unrelated "stuff" involved.

Related

The difference between QDataStream and QByteArray

QTemporaryFile tf;
tf.open ();
QDataStream tfbs (&tf);
tfbs << "hello\r\n" << "world!\r\n";
const int pos = int (tf.pos ());
QByteArray ba;
ba.append ("hello\r\n");
ba.append ("world!\r\n");
const int size = ba.size ();
Basically my question is, what am I doing wrong? Why is pos > size? Should I not be using << ? Should I not be using QDataStream?
Edit: Is there a way to configure QDataStream or QTemporaryFile so that the << operator doesn't prepend strings with 32bit lengths and store the null terminators in the file? Calling QDataStream::writeBytes when I just have a series of quoted strings and QStrings makes for very ugly code.
The answer is in the docs. I'm not going to go over QByteArray, as I believe it's fairly obvious that it is working as expected.
The QDataStream operator<<(char*) overload evaluates to the writeBytes() function.
This function outputs:
Writes the length specifier len and the buffer s to the stream and
returns a reference to the stream. The len is serialized as a quint32,
followed by len bytes from s. Note that the data is not encoded.
So for "hello\r\n", I would expect the output to be:
0,0,0,8,'h','e','l','l','o','\r','\n',0
The 4-byte length, followed by the bytes from the string. The string-ending NULL is probably also being added to the end, which would account for the otherwise mysterious extra two bytes.
So I ended up writing my own helper class to serialize my data:
class QBinaryStream
{
public:
QBinaryStream (QIODevice& iod) : m_iod (iod) {}
QBinaryStream& operator << (const char* data)
{
m_iod.write (data);
return *this;
}
QBinaryStream& operator << (const QString& data)
{
return operator << (data.toUtf8 ());
}
QBinaryStream& operator << (const QByteArray& data)
{
m_iod.write (data);
return *this;
}
private:
QIODevice& m_iod;
};
Should I not be using QDataStream?
In your case maybe QTextStream or even QString would do.
The QTextStream class provides a convenient interface for reading and
writing text.
QTextStream can operate on a QIODevice, a QByteArray or a QString.
Using QTextStream's streaming operators, you can conveniently read and
write words, lines and numbers.
As for QByteArray, Qstring should be preferred to it whenever possible
The QByteArray class provides an array of bytes.
QByteArray can be used to store both raw bytes (including '\0's) and
traditional 8-bit '\0'-terminated strings. Using QByteArray is much
more convenient than using const char *. Behind the scenes, it always
ensures that the data is followed by a '\0' terminator, and uses
implicit sharing (copy-on-write) to reduce memory usage and avoid
needless copying of data.
In addition to QByteArray, Qt also provides the QString class to store
string data. For most purposes, QString is the class you want to use.
It stores 16-bit Unicode characters, making it easy to store
non-ASCII/non-Latin-1 characters in your application. Furthermore,
QString is used throughout in the Qt API. The two main cases where
QByteArray is appropriate are when you need to store raw binary data,
and when memory conservation is critical (e.g., with Qt for Embedded
Linux).

Why doesn't the string class have a << operator (operator<<) predefined so that strings work like ostringstreams?

It seems to me that defining the << operator (operator<<) to work directly with strings is more elegant than having to work with ostringstreams and then converting back to strings. Is there a reason why c++ doesn't do this out of the box?
#include <string>
#include <sstream>
#include <iostream>
using namespace std;
template <class T>
string& operator<<(string& s, T a) {
ostringstream ss;
ss << a;
s.append(ss.str());
return s;
}
int main() {
string s;
// this prints out: "inserting text and a number(1)"
cout << (s << "inserting text and a number (" << 1 << ")\n");
// normal way
ostringstream os;
os << "inserting text and a number(" << 1 << ")\n";
cout << os.str();
}
Streams contain additional state. Imagine if this were possible:
std::string str;
int n = 1234;
str << std::hex;
str << n;
return str; // returns "0x4d2" (or something, I forget)
In order to maintain this additional state, strings would have to have storage for this state. The C++ standards committee (and C++ programmers in general) have generally frowned upon superfluous resource consumption, under the motto "pay only for what you use". So, no extra fields in the string class.
The subjective answer: is that I think the std::string class was quite poorly designed to begin with, especially compared to other parts of C++'s excellent standard library, and adding features to std::string is just going to make things worse. This is a very subjective opinion and feel free to dismiss me as a raving lunatic.
The problem with the idea of strings being output streams is that they would become too heavy.
Strings are intended to "hold string data", not to format some output. Output streams have a heavy "state" which can be manipulated (see <iomanip>) and thus has to be stored. This means that, of course, this has to be stored for every string in every program, but almost none of them are used as an output stream; so it's a huge waste of resources.
C++ follows the "zero overhead" design principle (or at least no more overhead than totally necessary). Not having a string class which doesn't add any unnecessary overhead would be a huge violation of this design principle. If this was the case: what would people do in overhead-critical cases? Use C-strings... ouch!
In C++11, an alternative is to use the operator+= with std::to_string to append to a string, which can also be chained like the operator<< of the output stream. You can wrap both += and to_string in a nice operator<< for string if you like:
template <class Number>
std::string& operator<<(std::string& s, Number a) {
return s += std::to_string(a);
}
std::string& operator<<(std::string& s, const char* a) {
return s += a;
}
std::string& operator<<(std::string& s, const std::string &a) {
return s += a;
}
Your example, updated using this method: http://ideone.com/4zbVtD
Probably lost in the depths of time now but formatted output was always associated with streams in C (since they didn't have "real" strings) and this may have been carried over into C++ (which was, after all, C with classes). In C, the way to format to a string is to use sprintf, a variation on fprintf, the output-to-stream function.
Obviously conjecture on my part but someone probably thought similarly to yourself that these formatting things in the streams would be brilliant to have on strings as well, so they subclassed the stream classes to produce one that used a string as it's "output".
That seems the elegant solution to getting it working as quickly as possible. Otherwise, you would have had formatting code duplicated in streams and strings.

How to convert pid to const char pointer in c++?

I am trying to convert pid types to a const char pointer so that I can pass them in as an argument in a execlp function.
eg. execlp("/bin/ps", "-f", "--ppid", "9340,9345,9346,9342");
I know that you can convert a pid to a string eg. const std::string my_pid(str_pid.str());
And a string to a const char pointer eg. my_pid.c_str();
But how would you concatenate multiple pids into a const char pointer so I can run the execlp command with them?
ostringstream is probably what you want.
For instance,
std::ostringstream ostr;
for (int i=0; i<pids.count(); i++)
{
if (i > 0) ostr << ',';
ostr << pids[i];
}
execlp("/bin/ps", "-f", "--ppid", ostr.str().c_str());
You can concatenate all of your values together into a single std::string first, the pass the final std::string value to execlp(), eg:
std::string pids;
for (however many pids you have)
{
if (!pids.empty())
pids += ",";
pids += std::string(str_pid.str());
}
execlp("/bin/ps", "-f", "--ppid", pids.c_str());
A stringstream would be a good C++ way to do it.
std::stringstream myStream;
myStream << "a c string" << aStringObject << std::endl; // operate on the stream
std::string myNewString = myStream.str(); // create a string object
Treating the data as a stream is a fairly generic way to allow you to serialize and deserialize custom or builtin types. Custom types could include operator<< and operator>> to allow insertion and extraction respectively.
This approach also should have the advantage of speed over using temporary string objects and manipulating them. The stingstream (or any of its base classes) will employ a buffer under the covers. A loop with temporary strings will invoke several more constructors/allocations/destructors each iteration. This does depend on the underlying string representation as well. A copy on write (COW) string implementation would have fewer allocations and may only update a reference, but updating the work string will still require a new string.

How to put stringstream contents into char instead string type?

Every one know stringstream.str() need a string variable type to store the content of stringstream.str() into it .
I want to store the content of stringstream.str() into char variable or char array or pointer.
Is it possible to do that?
Please, write a simple example with your answer.
Why not just
std::string s = stringstream.str();
const char* p = s.c_str();
?
Edit: Note that you cannot freely give the p outside your function: its lifetime is bound to the lifetime of s, so you may want to copy it.
Edit 2: as #David suggests, copy above means copying of the content, not the pointer itself. There are several ways for that. You can either do it manually (legacy way "inherited" from C) -- this is done with the functions like std::strcpy. This way is quite complicated, since it involves manual resources management, which is usually discouraged, since it leads to a more complicated and error-prone code. Or you can use the smart pointers or containers: it can be either std::vector<char> or std::unique_ptr/std::shared_ptr.
I personally would go for the second way. See the discussion to this and #Oli's answer, it can be useful.
If you want to get the data into a char buffer, why not put it there immediately anyway? Here is a stream class which takes an array, determines its size, fills it with null characters (primarily to make sure the resulting string is null terminated), and then sets up an std::ostream to write to this buffer directly.
#include <iostream>
#include <algorithm>
struct membuf: public std::streambuf {
template <size_t Size> membuf(char (&array)[Size]) {
this->setp(array, array + Size - 1);
std::fill_n(array, Size, 0);
}
};
struct omemstream: virtual membuf, std::ostream {
template <size_t Size> omemstream(char (&array)[Size]):
membuf(array),
std::ostream(this)
{
}
};
int main() {
char array[20];
omemstream out(array);
out << "hello, world";
std::cout << "the buffer contains '" << array << "'\n";
}
Obviously, this stream buffer and stream would probably live in a suitable namespace and would be implemented in some header (there isn't much point in putting anything of it into a C++ file because all the function are templates needing to instantiated). You could also use the [deprecated] class std::ostrstream to do something similar but it is so easy to create a custom stream that it may not worth bothering.
You can do this if you want an actual copy of the string (vital if the stringstream object is going to go out of scope at some point):
const char *p = new char[ss.str().size()+1];
strcpy(p, ss.str().c_str());
...
delete [] p;
As discussed in comments below, you should be wary of doing it like this (manual memory management is error-prone, and very non-idiomatic C++). Why do you want a raw char array?
I figured it out. Using namespace std and replacing tstingstreamwith stringstream. Next step is:
stringstream strstream;
strstream.imbue(std::locale("C"));
string str = strstream.str();
const char *sql= str .c_str();
Now you can execute sql statement.
sqlite3_exec(db, sql, callback, (void*)data, &zErrMsg);
Maybe it helps to somebody.

How to write an object to file in C++

I have an object with several text strings as members. I want to write this object to the file all at once, instead of writing each string to file. How can I do that?
You can override operator>> and operator<< to read/write to stream.
Example Entry struct with some values:
struct Entry2
{
string original;
string currency;
Entry2() {}
Entry2(string& in);
Entry2(string& original, string& currency)
: original(original), currency(currency)
{}
};
istream& operator>>(istream& is, Entry2& en);
ostream& operator<<(ostream& os, const Entry2& en);
Implementation:
using namespace std;
istream& operator>>(istream& is, Entry2& en)
{
is >> en.original;
is >> en.currency;
return is;
}
ostream& operator<<(ostream& os, const Entry2& en)
{
os << en.original << " " << en.currency;
return os;
}
Then you open filestream, and for each object you call:
ifstream in(filename.c_str());
Entry2 e;
in >> e;
//if you want to use read:
//in.read(reinterpret_cast<const char*>(&e),sizeof(e));
in.close();
Or output:
Entry2 e;
// set values in e
ofstream out(filename.c_str());
out << e;
out.close();
Or if you want to use stream read and write then you just replace relevant code in operators implementation.
When the variables are private inside your struct/class then you need to declare operators as friend methods.
You implement any format/separators that you like. When your string include spaces use getline() that takes a string and stream instead of >> because operator>> uses spaces as delimiters by default. Depends on your separators.
It's called serialization. There are many serialization threads on SO.
There are also a nice serialization library included in boost.
http://www.boost.org/doc/libs/1_42_0/libs/serialization/doc/index.html
basically you can do
myFile<<myObject
and
myFile>>myObject
with boost serialization.
If you have:
struct A {
char a[30], b[25], c[15];
int x;
}
then you can write it all just with write(fh, ptr, sizeof(struct a)).
Of course, this isn't portable (because we're not saving the endieness or size of "int," but that may not be an issue for you.
If you have:
struct A {
char *a, *b, *c;
int d;
}
then you're not looking to write the object; you're looking to serialize it. Your best bet is to look in the Boost libraries and use their serialization routines, because it's not an easy problem in languages without reflection.
There's not really a simple way, it's C++ after all, not PHP, or JavaScript.
http://www.parashift.com/c++-faq-lite/serialization.html
Boost also has some library for it: http://www.boost.org/doc/libs/release/libs/serialization ... like Tronic already mentioned :)
The better method is to write each field individually along with the string length.
As an alternative, you can create a char array (or std::vector<char>) and write all the members into the buffer, then write the buffer to the output.
The underlying thorn is that a compiler is allowed to insert padding between members in a class or structure. Use memcpy or std::copy will result in padding bytes written to the output.
Just remember that you need to either write the string lengths and the content or the content followed by some terminating character.
Other people will suggest checking out the Boost Serialization library.
Unfortunately that is generally not quite possible. If your struct only contains plain data (no pointers or complex objects), you can store it as a one chunk, but care must be taken if portability is an issue. Padding, data type size and endianess issues make this problematic.
You can use Boost.Serialization to minimize the amount of code required for proper portable and versionable searialization.
Assuming your goal is as stated, to write out the object with a single call to write() or fwrite() or whatever, you'd first need to copy the string and other object data into a single contiguous block of memory. Then you could write() that block of memory out with a single call. Or you might be able to do a vector-write by calling writev(), if that call is available on your platform.
That said, you probably won't gain much by reducing the number of write calls. Especially if you are using fwrite() or similar already, then the C library is already doing buffering for you, so the cost of multiple small calls is minimal anyway. Don't put yourself through a lot of extra pain and code complexity unless it will actually do some good...