I have the below code.
main()
{
test::RouteMessage *Rtmesg = new test::RouteMessage;
test::RouteV4Prefix *prefix = new test::RouteV4Prefix;
test::RouteMessage testRtmesg;
prefix->set_family(test::RouteV4Prefix::RT_AFI_V4);
prefix->set_prefix_len(24);
prefix->set_prefix(1000);
Rtmesg->set_routetype(test::RouteMessage::RT_TYPE_BGP);
Rtmesg->set_allocated_v4prefix(prefix);
Rtmesg->set_flags(test::RouteMessage::RT_FLGS_NONE);
Rtmesg->set_routeevnt(test::RouteMessage::BGP_EVNT_V4_RT_ADD);
Rtmesg->set_nexthop(100);
Rtmesg->set_ifindex(200); Rtmesg->set_metric(99);
Rtmesg->set_pref(1);
int size = Rtmesg->ByteSize();
char const *rt_msg = (char *)malloc(size);
google::protobuf::io::ArrayOutputStream oarr(rt_msg, size);
google::protobuf::io::CodedOutputStream output (&oarr)
Rtmesg->SerializeToCodedStream(&output);
// Below code is just to see if everything is fine.
google::protobuf::io::ArrayInputtStream iarr(rt_msg, size);
google::protobuf::io::CodedInputStream Input (&iarr)
testRtmesg.ParseFromCodedStream(&Input);
Vpe::RouteV4Prefix test_v4Prefix = testRtmesg.v4prefix();
cout << std::endl;
std::cout << "Family " << test_v4Prefix.family() << std::endl;
std::cout << "Prefix " << test_v4Prefix.prefix()<< std::endl;
std::cout << "PrefixLen " << test_v4Prefix.prefix_len() << std::endl;
// All the above outputs are fine.
cout << std::endl;
cout << rt_msg; <<------------ This prints absolutely junk.
cout << std::endl;
amqp_bytes_t str2;
str2 = amqp_cstring_bytes(rt_msg); <<----- This just crashes.
printf("\n str2=%s %d", str2.bytes, str2.len);
}
Any operation on the above rt_msg just crashes. I want to use the above buffer to send to socket and another rabbitmq publish APIs.
Anybody out there who had similar issue...or worked out similar code ?
Protocol Buffers is a binary serialization format, not text. This means:
Yes, if you write the binary data to cout, it will look like junk (or crash).
The data is not NUL-terminated like C strings. Therefore, you cannot pass it into a function like amqp_cstring_bytes which expects a NUL-terminated char* -- it may cut the data short at the first 0 byte, or it may search for a 0 byte past the end of the buffer and crash. In general, any function that takes a char* but does not also take a length won't work.
I'm not familiar with amqp, but it looks like the function you are trying to call, amqp_cstring_bytes, just builds a amqp_bytes_t, which is defined as follows:
typedef struct amqp_bytes_t_ {
size_t len;
void *bytes;
} amqp_bytes_t;
So, all you have to do is something like:
amqp_bytes_t str2;
str2.bytes = rt_msg;
str2.len = size;
Related
I want to store some Capnproto struct in a LevelDB, so I have to serialize it to string and deserialize it back from a std::string later. Currently, I play around with the following (adapted from here: https://groups.google.com/forum/#!msg/capnproto/viZXnQ5iN50/B-hSgZ1yLWUJ):
capnp::MallocMessageBuilder message;
WortData::Builder twort = message.initRoot<WortData>();
twort.setWid(1234);
twort.setW("Blabliblub");
kj::Array<capnp::word> dataArr = capnp::messageToFlatArray(message);
kj::ArrayPtr<kj::byte> bytes = dataArr.asBytes();
std::string data(bytes.begin(), bytes.end());
std::cout << data << std::endl;
const kj::ArrayPtr<const capnp::word> view(
reinterpret_cast<const capnp::word*>(&(*std::begin(data))),
reinterpret_cast<const capnp::word*>(&(*std::end(data))));
capnp::FlatArrayMessageReader message2(view);
WortData::Reader wortRestore = message2.getRoot<WortData>();
std::cout << wortRestore.getWid() << " " << std::string(wortRestore.getW()) << std::endl;
And it basically works, but the people in the link above were unsure if this approach will cause errors later and since the discussion is pretty old, I wanted to ask if there's a better way.
Someone in the end said something like "use memcpy!", but I'm not sure if that's useful and how to do this with the array types needed for FlatArrayMessageReader.
Thanks in advance!
dvs23
Update:
I tried to implement the suggestion related to the word-aligning:
capnp::MallocMessageBuilder message;
WortData::Builder twort = message.initRoot<WortData>();
twort.setWid(1234);
twort.setW("Blabliblub");
kj::Array<capnp::word> dataArr = capnp::messageToFlatArray(message);
kj::ArrayPtr<kj::byte> bytes = dataArr.asBytes();
std::string data(bytes.begin(), bytes.end());
std::cout << data << std::endl;
if(reinterpret_cast<uintptr_t>(data.data()) % sizeof(void*) == 0) {
const kj::ArrayPtr<const capnp::word> view(
reinterpret_cast<const capnp::word*>(&(*std::begin(data))),
reinterpret_cast<const capnp::word*>(&(*std::end(data))));
capnp::FlatArrayMessageReader message2(view);
WortData::Reader wortRestore = message2.getRoot<WortData>();
std::cout << wortRestore.getWid() << " " << std::string(wortRestore.getW()) << std::endl;
}
else {
size_t numWords = data.size() / sizeof(capnp::word);
if(data.size() % sizeof(capnp::word) != 0) {
numWords++;
std::cout << "Something wrong here..." << std::endl;
}
std::cout << sizeof(capnp::word) << " " << numWords << " " << data.size() << std::endl;
capnp::word dataWords[numWords];
std::memcpy(dataWords, data.data(), data.size());
kj::ArrayPtr<capnp::word> dataWordsPtr(dataWords, dataWords + numWords);
capnp::FlatArrayMessageReader message2(dataWordsPtr);
WortData::Reader wortRestore = message2.getRoot<WortData>();
std::cout << wortRestore.getWid() << " " << std::string(wortRestore.getW()) << std::endl;
}
The linked conversation is still accurate to the best of my knowledge. (Most of the messages on that thread are me, and I'm the author of Cap'n Proto...)
It's very likely that the buffer backing any std::string will be word-aligned in practice -- but it is not guaranteed. When reading from a std::string, you should probably check that the pointer is aligned (e.g. by reinterpret_cast<uintptr_t>(str.data()) % sizeof(void*) == 0). If aligned, you can reinterpret_cast the pointer to capnp::word*. If not aligned, you'll need to make a copy. In practice the code will probably never make a copy because std::string's backing buffer is probably always aligned.
On the writing end, avoiding copies is trickier. Your code as you've written it actually makes two copies.
One here:
kj::Array<capnp::word> dataArr = capnp::messageToFlatArray(message);
And one here:
std::string data(bytes.begin(), bytes.end());
It looks like LevelDB supports a type called Slice, which you can use instead of std::string when writing, to avoid the second copy:
leveldb::Slice data(bytes.begin(), bytes.size());
This will reference the underlying bytes rather than make a copy, and should be usable in all the LevelDB write functions.
Unfortunately, one copy is unavoidable here, because LevelDB wants the value to be one contiguous byte array, whereas a Cap'n Proto message may be broken into multiple segments. The only way to avoid this would be for LevelDB to add support for "gather writes".
I'm trying to write a program that reads in an OpenGL shader from a .txt file. I've actually already done this a few days ago, this was the code I used:
char vShaderData[2000];
char fShaderData[2000];
void readShaders() {
std::ifstream vShaderF;
std::ifstream fShaderF;
vShaderF.open("shaders//vertexShader.txt");
fShaderF.open("shaders//fragShader.txt");
if (vShaderF.is_open() && fShaderF.is_open()) std::cout << m << "Shader read success" << std::endl;
else std::cout << "Shader read fail" << std::endl;
std::cout << m << "vertex shader: " << std::endl;
vShaderF.read(vShaderData, 2000);
for (int i = 0; i < 2000; i++) {
std::cout << vShaderData[i];
}
std::cout << std::endl << std::endl;
std::cout << m << "frag shader: " << std::endl;
fShaderF.read(fShaderData, 2000);
for (int i = 0; i < 2000; i++) {
std::cout << fShaderData[i];
}
std::cout << std::endl;
vShaderF.close();
fShaderF.close();
}
This worked great. my shader file was not actually not 2000 in length, but the read() call seemed to store the extra characters as whitespace into the char array which is what I wanted.
Now having restructured my code a little bit in a newer program, my reader now looks like this:
std::ifstream shaderFile;
shaderFile.open(path);
if (shaderFile.is_open()) cout << "Shader at: " << path << ", initalized" << endl;
char data[2000];
shaderFile.read(data, 2000);
for (int i = 0; i < 2000; i++) std::cout << data[i];
The actual text portion still reads correct. However, now the extra space in the char array is stored with this instead of whitespace:
In case the image won't show, it is basically just a reapeating pattern of these two characters [|[|[|....
Why is this happening and how can I fix it?
NOTE: I'm using the same shader file, same computer, same IDE, same everything. The old one still works.
When using std::istream:read() it will not set the parts of the buffer to spaces which were not read. The memory will be left untouched. If you want to get spaces into an unread area of the buffer, you'll need to put the spaces there yourself. If the program indeed had spaces in the buffer it was because the buffer somehow already contained spaces by chance.
You can use std::istream::gcount() to determine how many characters were read.
If you want the arrays to contain predefined data, you'll have to initialize it with such predefined data. If the stream reads fewer data than the array size, you will have the padding you want.
I am transferring a struct over socket using c++. I read some earlier questions on sending structs and one approach suggested was to transfer using a char* after cast. Since both server and client are on same machine so no issues of endianness here.
Couple of questions here. I get size of struct as 48. As per my calculation shouldn't it be 43? 8x4 + 10 +1
Secondly on server side when i print the received buffer I only get the text elements. The long integers are not received.
struct testStruct{
char type;
char field1[10];
char field2[8];
char field3[8];
long num1, num2;
};
testStruct ls;
ls.type = 'U';
strcpy(ls.field1, "NAVEENSHAR");
strcpy(ls.field2, "abcd1234");
strcpy(ls.field3, "al345678");
ls.num1 = 40;
ls.num2 = 200;
char* bytes = static_cast<char*>(static_cast<void*>(&ls));
bytes_sent = send(socketfd, bytes, sizeof(ls), 0);
cout << "bytes sent: " << bytes_sent<< "\n";
//On server sidechar
incomming_data_buffer[1000];
bytes_recieved = recv(new_sd, incomming_data_buffer,1000, 0);
cout << "|" << incomming_data_buffer << "|\n";
It shows 48 bytes received and no trailing integers which i added.
Any idea on why this could be happening. I have read about sending structs using boost serialization but at the same time that overhead is huge for simple structs.
You are almost certainly receiving all the data. The problem is with this line:
cout << "|" << incomming_data_buffer << "|\n";
which prints incomming_data_buffer as a C style string, so stops at the first zero-byte. Since your long values are encoded in binary for, there will be zeros at least there (there may also be zeros in the padding between fields).
You could try doing something like:
cout << "|";
for (int i = 0; i < bytes_received; i++)
{
cout << hex << (((int)incomming_data_buffer[i]) & 0xff) << " ";
}
cout << "|\n";
to show all bytes of the package you received.
Consider
#include <string>
#include <iostream>
int main()
{
/*
hello
5
hel
3
*/
char a[] = "hello";
std::cout << a << std::endl;
std::cout << strlen(a) << std::endl;
a[3] = 0;
std::cout << a << std::endl;
std::cout << strlen(a) << std::endl;
/*
hello
5
hel o
5
*/
std::string b = "hello";
std::cout << b << std::endl;
std::cout << b.length() << std::endl;
b[3] = 0;
std::cout << b << std::endl;
std::cout << b.length() << std::endl;
getchar();
}
I expect std::string will behave identical to char array a. That's it, insert null character in the middle of the string, will "terminate" the string. However, it is not the case. Is my expectation wrong?
A std::string is not like a usual C string, and can contain embedded NUL characters without problems. However, if you do this you will notice the string is prematurely terminated if you use the .c_str() function to return a const char *.
No - std::strings are not NUL-terminated like C "strings"; the std::string records its length independently.
#Lou is right: don't do that. Instead, do this:
b.erase (3, b.length());
Yes, your expectation is wrong. std::string is meant to be different from C strings (e.g. not necessarily stored in consecutive memory / an array).
To duplicate the first section's behavior, try std::cout << b.c_str() instead of std::cout << b.
I expect std::string will behave identical to char array a.
Why? Nothing in the documentation, anywhere, having to do with std::string says it does this.
My suggestion, stop treating like C++ as C plus some stuff.
in a function, that gets unsigned char && unsigned char length,
void pcap_callback(u_char *args, const struct pcap_pkthdr* pkthdr, const u_char* packet)
{
std::vector<unsigned char> vec(packet, packet+pkthdr->len); // optimized from foo.
std::stringstream scp;
for (int i=0;i<pkthdr->len;i++) {
scp<<vec[i];
}
std::string mystr = std::string(scp.rdbuf()->str());
std::cout << "WAS: " << packet << std::endl;
std::cout << "GOOD: " << scp.str() << std::endl;
std::cout << "BAD: " << scp.str().c_str() << std::endl;
std::cout << "TEST: " << mystr.size() << std::endl;
assert(mystr.size() == pkthdr->len);
}
Results:
WAS: prints nothing (guess there is a pointer to const.. case)
GOOD: prints data
BAD: prints nothing
TEST, assert: prints that mystr.size() is equal to passed unsigned char size.
I tried:
string.assign(scp.rdbuf());
memcpy(char, scp.str(), 10);
different methods of creating/allocating temporary chars, strings
No help.. it is wanted to get a std::cout'able std::string that contains data, (which was picked from foo, which was unsigned char, which was packet data).
Guessing either the original foo may not be null-terminated, or the problem is something like this - simple, but can't get in.. what are the things to look for here?
(this code is another attempt to use libpcap, just to print packets in C++ way, without using known C++ magic wrappers like libpcapp).
For a quick test, throw in a check for scp.str().size() == strlen(scp.str().c_str()) to see if there are embedded '\0' characters in the string, which is what I suspect is happening.
I think you're going about this the wrong way. It looks like you're dealing with binary data here, in which case you can't expect to meaningfully output it to the screen as text. What you really need is a hex dump.
const unsigned char* ucopy = packet;
std::ios_base::fmtflags old_flags = std::cout.flags();
std::cout.setf(std::ios::hex, std::ios::basefield);
for (const unsigned char* p = ucopy, *e = p + pkthdr->len; p != e; ++p) {
std::cout << std::setw(2) << std::setfill('0') << static_cast<unsigned>(*p) << " ";
}
std::cout.flags(old_flags);
This will output the data byte-by-byte, and let you examine the individual hex values of the binary data. A null byte will simply be output as 00.
Check std::cout.good() after the failed output attempt. My guess is that there's some failure on output (i.e. trying to write a nonprintable character to the console), which is setting failbit on cout.
Also check to ensure the string does not start with a NULL, which would cause empty output to be the expected behavior :)
(Side note, please use reinterpret_cast for unsigned char *ucopy = (unsigned char*)packet; if you're in C++ ;) )