I've using OpenDDS in a project. Now, for interoperability, we need to send topics also with a custom framework to other machines. Since this custom framework allows to send strings, I'd like to serialize the topics in a string and then send them.
I was using boost::serialization, but then I've made up the idea that in order to send a topic, OpenDDS should be able to serialize a topic some way, so I should be able to pick the corresponding function and use it for serialize data.
Inspecting the code I was able to find the overload of >>= and <<= operators:
void
operator<<= (
::CORBA::Any &_tao_any,
BasicType::LocalForceDataDataReader_ptr _tao_elem)
{
BasicType::LocalForceDataDataReader_ptr _tao_objptr =
BasicType::LocalForceDataDataReader::_duplicate (_tao_elem);
_tao_any <<= &_tao_objptr;
}
/// Non-copying insertion.
void
operator<<= (
::CORBA::Any &_tao_any,
BasicType::LocalForceDataDataReader_ptr *_tao_elem)
{
TAO::Any_Impl_T<BasicType::LocalForceDataDataReader>::insert (
_tao_any,
BasicType::LocalForceDataDataReader::_tao_any_destructor,
BasicType::_tc_LocalForceDataDataReader,
*_tao_elem);
}
It serializes the topic into Corba::Any. It seems to work, but now I need to send the content of Corba::Any. Is there a way to put the content of Corba::Any to a string, and retrieve its data from a string? Or, in other words, how can I serialize and deserialize Corba::Any?
Or there's a better way to serialize a OpenDDS topic to a string?
It's possible to use TAO's serialization system to do this, but it's probably better to use what OpenDDS is using: https://github.com/objectcomputing/OpenDDS/blob/master/dds/DCPS/Serializer.h (or at least it's easier for me to write an example for since I know it much better)
These are some functions that will serialize types to and from std::strings:
const OpenDDS::DCPS::Encoding encoding(OpenDDS::DCPS::Encoding::KIND_XCDR2);
template <typename IdlType>
std::string serialize_to_string(const IdlType& idl_value)
{
const size_t xcdr_size = OpenDDS::DCPS::serialized_size(encoding, idl_value);
ACE_Message_Block mb(xcdr_size);
OpenDDS::DCPS::Serializer serializer(&mb, encoding);
if (!(serializer << idl_value)) {
throw std::runtime_error("failed to serialize");
}
return std::string(mb.base(), mb.length());
}
template <typename IdlType>
IdlType deserialize_from_string(const std::string& xcdr)
{
ACE_Message_Block mb(xcdr.size());
mb.copy(xcdr.c_str(), xcdr.size());
OpenDDS::DCPS::Serializer serializer(&mb, encoding);
IdlType idl_value;
if (!(serializer >> idl_value)) {
throw std::runtime_error("failed to deserialize");
}
return idl_value;
}
Also be careful when using std::string for any binary data like CDR to make sure it's not interpreted as a null-terminated string.
Related
Documentation states that FILE is object type that identifies a stream. So, is it possible to get the stream object associated with a FILE?
For example, I'd like to get std::cout object from stdout FILE pointer, or std::cerr from stderr etc. More generally I want to write a function that redirects a given stream and sets the custom streambuf to it, something like this:
void redirect(FILE* file, std::ios stream) {
freopen_s((FILE**)file, "CONOUT$", "w", file);
stream.rdbuf(customBuffer);
}
used to redirect streams
redirect(stdout, std::cout);
redirect(stderr, std::cerr);
It seems redundant to have 2 parameters, since both parameters are always associated with each other.
The C++ standard library includes the C standard library. A FILE is a C stream, which is quite a different animal than a C++ iostream. It is possible for an std::stream implementation to rely of an underlying FILE, but this is not required by the standard, and even in that case there is no way to retrieve it.
What is possible is to build a custom std::streambuf that explicitly uses an underlying FILE *, and use it in a std::stream. std::basic_streambuf is one of the few classes from the C++ standard library that is explicitely designed as a base class for custom derivation. Unfortunately I could not find a tutorial for it, but the class contains a number of virtual methods that you just have to override. It is not exactly an easy path, but is possible with some works, heavy testing, and eventually some help from SO if you get stuck somewhere. But a full implementation is far beyond a SO answer.
TL/DR: there is no underlying std::stream associated with a FILE but with some work you can build a custom stream_buffer that will use an underlying FILE *. Though those are rather advanced operations...
While it is not possible to cleanly do this in C++ you could do something like this.
FILE * file = popen("someFile")
const unsigned BUFF = 2048;
string total;
bool done = false;
while (!done) {
vector<char> cBuf[BUFF];
size_t read = fread((void *)&cBuf[0], 1, BUFF, f);
if (read)
{
total.append(cBuf.begin(), cBuf.end());
}
if (read < BUFF)
{
done = true;
}
}
pclose(f);
istringstream filey(total);
Hope this helps.
I would like to serialize/deserialize some structured data in order to send it over the network via a char* buffer.
More precisely, suppose I have a message of type struct Message.
struct Message {
Header header;
Address address;
size_t size; // size of data part
char* data;
} message
In C, I would use something such as:
size = sizeof(Header) + sizeof(Address) + sizeof(size_t) + message.size;
memcpy(buffer, (char *) message, size);
to serialize, and
Message m = (Message) buffer;
to deserialize.
What would be the "right" way to do it in C++. Is it better to define a class rather than a struct. Should I overload some operators? are there alignment issues to consider?
EDIT: thanks for pointing the "char *" problem. The provided C version is incorrect. The data section pointed to by the data field should be copied separately.
Actually there are many flavors:
You can boost let it do for you: http://www.boost.org/doc/libs/1_52_0/libs/serialization/doc/tutorial.html
Overloading the stream operators << for serialization and >> for deserialization works well with file and string streams
You could specify a constructor Message (const char*) for constructing from a char*.
I am a fan of static methods for deserialization like:
Message {
...
static bool desirialize (Message& dest, char* source);
}
since you could catch errors directly when deserializing.
And the version you proposed is ok, when applying the modifications in the comments are respected.
Why not insert a virtual 'NetworkSerializable' Class into your inheritance tree? A 'void NetSend(fd socket)' method would send stuff, (without exposing any private data), and 'int(bufferClass buffer)' could return -1 if no complete, valid message was deserilalized, or, if a valid message has been assembled, the number of unused chars in 'buffer'.
That encapsulates all the assembly/disassembly protocol state vars and other gunge inside the class, where it belongs. It also allows message/s to be assembled from multiple stream input buffers.
I'm not a fan of static methods. Protocol state data associated with deserialization should be per-instance, (thread-safety).
I have a callback function, which provides pointer to data and it's size. I don't know what size will be next time and which call will be the last. And I need to match incoming data with regex and save matches.
Something like that.
class data_filter
{
public:
data_filter(const std::string& re)
: re_(re)
{}
public:
// callback func. It will be called many times with data parts
void process(const char* data, const size_t len)
{
re_.match(data, len, m_); // if found match, add it to matches
}
public:
void print_matches()
{
for(size_t i = 0; i < m_.size(); ++i)
{
std::cout << m_[i] << std::endl;
}
}
private:
some_cool_regex re_;
cool_regex_matches m_;
};
If absolutely neccessary i can provide some fixed buffer for regex backtracking, but i would like to avoid it.
I already had a brief look at boost::regex with partial_match option. As far as i understood from a first glance it can provide such functionality, but user should manually deal with temporary buffer.
So, should i stick with boost or there are some libraries that match my needs closer?
Thanks.
Since, indeed, there could be a need for backtracking, your options for streaming are limited or non-existent.
Boost Spirit "solves" the same issue by using the multi_pass_iterator<> adapter around input iterators. The adapter is able to maintain a buffer of previously read data for backtracking, freeing it as soon as it is no longer required (e.g. due to an expectation point).
If you shared some details about "some cool regex" then I could probably show you how to do this.
UPDATE Just found this library: https://github.com/openresty/sregex
libsregex - A non-backtracking regex engine library for large data streams
Is there an easy way to check if something was serialized in stl::ostream. I am looking for something like:
some preparation
// ... a very complex code that may result in adding or not to the stream,
// that I will prefer not to change
check if the stream has something added
Note that this will need to works recursively. Is using register_callback is a good idea or there is easier way?
First the immediate question: register_callback() is intended to deal with appropriate copying and releasing of resources stored in pword() and will have operations only related to that (i.e., copying, assigning, and releasing plus observing std::locale changes). So, no, that won't help you at all.
What you can do, however, is to create a filtering stream buffer which observes if there was a write to the stream, e.g., something like this:
class changedbuf: std::streambuf {
std::streambuf* d_sbuf;
bool d_changed;
int_type overflow(int_type c) {
if (!traits_type::eq_int_type(c, traits_type::eof())) {
this->d_changed = true;
}
return this->d_sbuf->sputc(c);
}
public:
changedbuf(std::streambuf* sbuf): d_sbuf(d_sbuf), d_changed() {}
bool changed() const { return this->d_changed; }
}
You can use this in place of the std::ostream you already have, e.g.:
void f(std::ostream& out) {
changedbuf changedbuf(out.rdbuf());
std::ostream changedout(&changedbuf);
// use changedout instead of out; if you need to use a global objects, you'd
// replace/restore the used stream buffer using the version of rdbuf() taking
// an argument
if (changedbuf.change()) {
std::cout << "there was a change\n";
}
}
A real implementation would actually provide a buffer and deal with proper flushing (i.e., override sync()) and sequence output (i.e., override xsputn()). However, the above version is sufficient as a proof-of-concept.
Others are likely to suggest the use of std::ostringstream. Depending on the amount of data written, this can easily become a performance hog, especially compared to an advanced version of changedbuf which appropriately deals with buffering.
Are you passing the stream into the complex code, or is it globally visible? Can it be any kind of ostream or can you constrain the type to ofstream or ostringstream?
You may be able to use tellp to determine whether the file position has changed since your preparation code, if your ostream type supports it (such as with most fstreams). Or, if you're passing the stream in, you could pass an empty ostringstream in and check that it's not empty when the string is extracted to be printed out.
It's not entirely obvious which solution, if any, would be appropriate for you without knowing more about the context of your code and the specifics of your problem. The best answer may be to return (or set as a by-reference out-parameter) a flag indicating whether the stream was inserted into.
I'm working in a project where I'm writing a plugin for a particular package.
This package implements a "new" method in one of its headers, and as such, I am unable to include <strstream> as it also implements "new".
The package sdk also includes a thinned out and very old version of boost, which means that I can't use the boost serialization classes. It is built on Qt for VS2008, and we are (required for this project) to be in VS2005, so I can't include Qt either.
I need to be able to get data from an externally running application, sending the data over TCPIP. What is the best way for me to serialize out the data from the source and read it back in with these limitations?
I'm currently tempted to make a struct which could contain all possible data that might be sent over, and then just copying the memory of that struct into a block of bytes which gets sent over, but this sounds like a bad approach to me.
Thanks,
Liron
Google Protobuf
boost.serialization is the way to go here. It is the most comprehensible serialization library for C++ that I know of, and it comes with support for standard containers.
You can extract the bytes from your data and pass it around, see a basic example.
QByteArray vectorToBin(const QVector<qint32> & vec)
{
size_t size = sizeof(qint16);
QByteArray result;
foreach(qint16 e, vec) {
for(int n = 0; n<=(size-1)*8; n+=8) {
char c = static_cast<char>((e >> n));
result.append(c);
}
}
return result;
}
QVector<qint32> binToVector(const QByteArray & bytes)
{
QVector<qint32> result;
size_t size = sizeof(qint16);
for(int i=0; i<bytes.size(); i+=size) {
qint16 e = ((bytes[i+1] & 0xff)<<8) | (bytes[i] & 0xff);
result << e;
}
return result;
}