I'm writing in C++ and want to send binary data (Serialized Google Protobufs) using the SimpleAmqpClient library. The only message type I see is BasicMessage. The only way to populate a BasicMessage seems to be with an std::string. Is publishing a BasicMessage with an std::string holding binary data (GProtobufs serialize themselves this way) going to work?
So I got around to trying this. Initializing a BasicMessage from a string with binary data including nulls does work as I had hoped. I would feel better if I knew this was an intentionally supported feature and part of the SimpleAmqpClient test suite.
I previously posted that it worked; then posted that it didn't. Well it actually does.
Related
sorry for absolutly murdering the tilte. But I am not sure how to frame this question, please edit this if there is a better way of explaining my problem.
I am reading a bitstream from a program which I convert into json data, write it to a socket, where another program reads this data and appends it to a log.json file. I am doing all of this in C++
Now I want to display this data in a better way. So why not try to display this in an html document, with some css applied on it.
My first thought was to simply fetch this with javascript. But now-a-days this throws an error.
So my second thought was to create a simple node.js server which accepts GET requests and then use this to serve the file. But this feels like its a bit overkill.
My third thought is now to perhaps use my original server (who continuously reads from the socket). And use that one to also accept http requests. But then I would have to multithread it, which again seems kinda overkill.
So im kinda falling back to needing 2 different "servers". One that reads from the socket and appends to the log file and another to serve this file to the website.
Am I'm thinking wrong here? What would be a good way to solve this?
I am quite new to parsing text files. While googling a bit, I found out that a parser builds a tree structure usually out of a text file. Most of the examples consists of parsing files, which in my view is quite static. You load the file to parser and get the output.
My problem is something different from parsing files. I have a stream of JSON data coming from a server socket at TCP port 6000. I need to parse the incoming data.I have some questions in mind:
1) Do I need to save the incoming JSON data at the client side with some sought of buffer? Answer: I think yes I need to save it, but are there any parsers which can do it directly like passing the JSON object as an argument to the parse function.
2) How would the structure of the real time parser look like`? Answer: Since on google only static parsing tree structure is available. In my view each object is parsed and have some sought of parsed tree and then it is deleted from the memory. Otherwise it will cause memory overflow because the data is continuous.
There are some parser libraries available like JSON-C and JSON lib. One more thing which comes into my mind is that can we save a JSON object in any C/C++ array. Just thought of that but could realize how to do that.
Can somebody please give me some sample code to do this? I've been reading and looking at the tutorials but they all seem to cover how to create proto files and generate your message classes and how to populated the message. I cannot figure out how to actually send the message once I have it.
Here I am populating my message:
TestMessage message;
message.set_personname("Lucy");
message.set_image(data, elemSize);
string out;
message.SerializeToString(&out);
Did that serialize the message or do I still need to write it out to a buffer? I need to check this code but I believe the object serialized to a string might do what I need for protobufs.
Once I have a serialized class to send, what does the code look like to transmit over http to a url? I mean are there Google provided methods to do this? If so, where are they? If not, then what should I use?
Please note that I am a complete noob at C++. So please forgive me if it seems as if I didn't do enough research. It's just that the more I look the more confused I get. There seems to be so many options. Just one will do.
Say we want to create our message not using any preexisting .proto files and compiled out from them cpp/cxx/h files. We want to use protobuf strictly as a library. For example we got (in some only known to us format) message description: a message called MyMessage has to have MyIntFiels and optional MyStringFiels. How to create such message? for example fill it with simple data save to .bin and read from that binary its contents back?
I looked all over dynamic_message.h help description and DescriptorPool and so on but do not see how to add/remove fields to the message as well as no way to add described on fly message to DescriptorPool.
Can any one please explain?
Short answer: it can't be used that way.
The overview page of Protobuf says:
XML is also – to some extent – self-describing. A protocol buffer is only meaningful if you have the message definition (the .proto file).
Meaning the whole point of Protobuf is to throw-out self-descriptability in favor of parsing speed ==> it's just not it's purpose to create self describing messages.
Consider using XML or JSON or any other serialization format. If the protection is needed, you can use symmetric encryption and/or lzip compression.
forgive me if this is a bit of a newbie question, I started to learn Django yesterday, and I'm trying not to get into bad habits, i.e. I am trying to do things "the django way" from the start.
I have a view that recieves binary data as a http post field. Now Django of course autoconverts my binary data to a unicode string.
My question is, how do I just get the raw binary data?
A couple of things occurred to me. Let request be the request I'm processing.
Using request.raw_post_data would involve parsing the data again - when appearantly request.POST actually stores raw data and I am actually just trying to get around the on-the-fly conversion (and besides, that is new in the development version).
Using base64 or so to transfer the data would work, but seems like too much overhead when the data transfer itself is not the problem.
doing request.encoding="foo" before getting that field (and reassigning afterwards) doesn't work either because I still get a unicode string, besides feeling like a bit of a dirty hack. Using "base64" here (not as bad as for the transfer encoding) gives me an
AssertionError.
Thanks in advance for your ideas!
EDIT:
To clarify - I am not talking about a classic file upload here, but as binary data stored in a POST field. I'd like to do it that way because the only way I want to interface with that view is via an upload script. Using a normal POST field makes both the client and the server much simpler in that case.
Some might say that storing binary data in a standard form field is a bad habit in some way :)
You could use standard library methods of Python to convert your string back to a binary representation.
Take a look at binascii — Convert between binary and ASCI
Posting before edit:
What about this piece of code (receiving data from a POST)
def handleFile(self, request):
file = request.FILES["file"]
destination = open('filename.ext', 'wb')
for chunk in file.chunks():
destination.write(chunk)
destination.close()
Works for me.