I am using a tcp connection to send data to a c++ server. I have read about ei library which when given a binary has a bunch of functions to get the decoded value out of the binary. Egs, ei_decode_string, ei_decode_long and others.
I am trying to do these simple things:
1. create a socket and connect to it.
{ok, Socket} = gen_tcp:connect({127,0,0,1}, 8986, []).
2. Use gen_tcp:send/2 to send data.
gen_tcp:send(Socket, term_to_binary("Stackoverflow")).
Therefore, I am sending a binary format of a string to the server.
My Server, C++ code, gets the data and I am trying to get whatever the client sends me over the socket using ei_decode_string like:
Ideally, when decoded I should get back the string, "Stackoverflow" since I told it to decode_as_string from the binary. Made sure I had enough space in the resulting buffer.
char *p = (char*)malloc(sizeof(char) * 100);
int index = 0;
int decoded = ei_decode_string(buff, &index, p);
cout<<"The decoded value is "<<p<<endl;
I am not able to decode the string which I sent.! Am I missing something? How can I send data and decode it on the server side, if this is not the right approach.
Thank you for the help!
1> term_to_binary("Stackoverflow").
<<131,107,0,13,83,116,97,99,107,111,118,101,114,102,108,
111,119>>
Here the first item in the binary is not the string as small integers, but a version number (131) as described in External Term Format. After that comes the terms each prefixed by a tag and then corresponding data. In this case tag 107 STRING_EXT, two bytes for length (13) and then 13 8bit integers, first one 83 corresponds to letter 'S' (83 in ascii).
To handle this binary correctly correctly, call to ei_decode_version has to be before ei_decode_string
int ei_decode_version(const char *buf, int *index, int *version)
This function decodes the version magic number for the erlang binary
term format. It must be the first token in a binary term.
Related
My question almost exactly the same as this one which is unanswered. I am trying to read the binary data of a .jpg to send as an HTTP response on a simple web server using C++. The code for reading the data is below.
FILE *f = fopen(file.c_str(),"rb");
if(f){
fseek(f,0,SEEK_END);
int length = ftell(f);
fseek(f,0,SEEK_SET);
char* buffer = (char*)malloc(length+1);
if(buffer){
int b = fread(buffer,1,length,f);
std::cout << "bytes read: " << b << std::endl;
}
fclose(f);
buffer[length] = '\0';
return buffer;
}
return NULL;
When the request for the image is made and this code runs, fread() returns 25253 bytes being read, which seems correct. However, when I perform strlen(buffer) I get only 4. Of course, this gives an error on a browser when the image tries to display. I have also tried manually setting the HTTP content length to 25253 but I then a receive a curl error 18, indicating the transfer ended early (as only 4 bytes exist).
As the other poster mentioned in their question, the 5th byte of the image (and I assume most .jpg images) is 0x00, but I am unsure if this has an effect on saving to the buffer.
I have verified the .jpg images I am loading are in the directory, valid, and display properly when opened normally. I have also tried 2 different methods of loading the binary data, and both also give only 4 bytes, so I am really at a loss. Any help is much appreciated.
When the request for the image is made and this code runs, fread()
returns 25253 bytes being read, which seems correct. However, when I
perform strlen(buffer) I get only 4.
Well there is your problem: You read binary data, not text, meaning that special characters like newline or the null character is not a something that indicates the structure of a text, its simple numbers.
strlen is a function to give you the count of characters other than '\0' or simply 0. However in a binary file like jpeg there a dozen of zeros usually in there, and because of a binary header structure, there seems to be always a zero at position 5 so, so strlen will stop at the first it found and return 4.
Also you seem confused by the fact that you try to send this "text interpreted" jpeg to a HTTP server. Of course it will complain, because you can not simply send binary data as text in HTTP, you either have to encode it, base64 is very popular, or set the content length header. Of course you also have to tell the HTTP client/server the type by setting the proper MIME header.
RESOLVED: Problem was primarily with the simulink blockset that was reading in the UDP packet rather than data transmission.
I am trying to send a 20 byte numerical array out of a c++ program by using winsock. The issue I am running into is data packaging, in the end I want each number in the array to go out as its own byte so that my simulink model that is receiving these values does not need an additional processing script.
My array contains 14 boolean values (0|1) and then 6 values that range from -100 to 100. These are reporting the status of a controller input. for example the array would look like
int array msgint[20] = [1,0,1,0,0,1,1,0,0,0,0,0,0,0,50,80,-90,40,90,-20];
I have tried using typecasting and sending multiple strings but all just appear to rearrange the gibberish I am getting or cause a socket error. Currently my sendto function looks like
sendto(sd,message,80,0,(struct sockadd *) &server,server_length)
I know this line works as the packet makes it through it just does not appear as I would like it to. In the send to, message is the formatted string I am trying to create to properly send all of contents of the array. Currently is is arbitrary and has little significance I have it in for debugging purposes essentially.
you are starting at the wrong point. Network communications should start with the design of the wire protocol.
How will you represent something on the wire. Binary or text. Most 'modern' protocols use text (json or xml). A few years ago binary was hot (asn1/ber/der). I suggest json
Then how will you wrap up the payload. Do you need to say 'here is a set of xxxs. now here is a set of yyyys'. I dont know what you are doing so its hard to say what you need
If you want to send 20 bytes, and you know that every integer value in your array will be in the [-100, +100] range, you should not use the int type, which usually contains either 32-bit or 64-bit values on modern platforms.
You might instead want to use the char type, which usually represents a 8-bit value.
For even more certainty, if you can use C++11 features, you should use the <cstdint> header, which defines the int8_t type, guaranteed to be a signed 8-bit type. See http://en.cppreference.com/w/cpp/types/integer.
Your array should look like:
#include <cstdint>
std::int8_t msgint[20] = {1,0,1,0,0,1,1,0,0,0,0,0,0,0,50,80,-90,40,90,-20};
or
char msgint[20] = {1,0,1,0,0,1,1,0,0,0,0,0,0,0,50,80,-90,40,90,-20};
and your sendto will be:
sendto(sd,msgint,20,0,(struct sockadd *) &server,server_length);
"in the end I want each number in the array to go out as its own byte"
Then change your array to
char msgint[20] = ...
I am writing some code that must read utf 8 encoded text files, and send them to OpenGL.
Also using a library which i downloaded from this site: http://utfcpp.sourceforge.net/
When i write down this i can show the right images on OpenGL window:
std::string somestring = "abcçdefgğh";
// Convert string to utf32 encoding..
// I also set local on program startup.
But when i read the utf8 encoded string from file:
The library warns me about that the string has not a valid utf encoding
I can't send the 'read from file' string to OpenGL. It crashes.
But i can still use std::cout for the string that i read from file (it looks right).
I use this code to read from file:
void something(){
std::ifstream ifs("words.xml");
std::string readd;
if(ifs.good()){
while(!ifs.eof()){
std::getline(ifs, readd);
// do something..
}
}
}
Now the question is:
If the string which is read from file is not correct, how does it look as expected when i check it with std::cout?
How can i get this issue solved?
Thanks in advance:)
The shell to which you write output is probably rather robust against characters it doesn't understand. It seems, not all of the used software is. It should, however, be relatively straight forward to verify if you byte sequence is a valid UTF-8 sequence: the UTF-8 encoding is relatively straight forward:
each code point starts with a byte representing the number of bytes to be read and the first couple of bytes:
if the high bit is 0, the code point consists of one byte represented by the 7 lower bits
otherwise the number of leading 1 bits represent the total number of bytes followed by a zero bit (obiously) and the remaining bits become the high bits of the code point
since 1 byte is already represented, bytes with the high bit set and the next bit not set are continuation bytes: the lower 6 bits are part of the representation of the code point
Based on these rules, there are two things which can go wrong and make the UTF-8 invalid:
a continuation byte is encountered at a point where a start byte is expected
there was a start byte indicating more continuation bytes then followed
I don't have code around which could indicate where things are going wrong but it should be fairly straight forward to write such code.
I am using Curlpp to send requests to various webservices to send and receive data.
So far this has worked fine since i have only used it for sending/receiving JSON data.
Now i have a situation where a webservice returns a zip file in binary form. This is where i encountered a problem where the data received is not complete.
I first had Curl set to write any data to a ostringstream by using the option WriteStream, but this proved not to be the correct approach since the data contained null characters, and thus the data stopped at the first null char.
After that, instead of using WriteStream i used WriteFunction with a callback function.
The problem in this case is that this function is always called 2 or 3 times, regardless of the amount of data.
This results in always having a few chunks of data that don't seem to be the first part of the file, although the data always contains PK as the first 2 characters, indicating a zip file.
I used several tools to verify that the data is entirely being sent to my application so this is not a problem of the webservice.
Here the code. Do note that the options like hostname, port, headers and postfields are set elsewhere.
string requestData;
size_t WriteStringCallback(char* ptr, size_t size, size_t nmemb)
{
requestData += ptr;
int totalSize= size*nmemb;
return totalSize;
}
const string CurlRequest::Perform()
{
curlpp::options::WriteFunction wf(WriteStringCallback);
this->request.setOpt( wf );
this->request.perform();
return requestData;
}
I hope anyone can help me out with this issue because i've run dry of any leads on how to fix this, also because curlpp is poorly documented(and even worse since the curlpp website disappeared).
The problem with the code is that the data is put into a std::string, despite having the data in binary (ZIP) format. I'd recommend to put the data into a stream (or a binary array).
You can also register a callback to retrieve the response headers and act in the WriteCallback according to the "Content-type".
curlpp::options::HeaderFunction to register a callback to retrieve response-headers.
std::string is not a problem, but the concatenation is:
requestData += ptr;
C string (ptr) is terminated with zero, if the input contains any zero bytes, the input will be truncated. You should wrap it into a string which knows the length of its data:
requestData += std::string(ptr, size*nmemb);
I am developing a program in C++, using the string container , as in std::string to store network data from the socket (this is peachy), I receive the data in a maximum possible 1452 byte frame at a time, the protocol uses a header that contains information about the data area portion of the packets length, and header is a fixed 20 byte length. My problem is that a string is giving me an unknown debug assertion, as in , it asserts , but I get NO message about the string. Now considering I can receive more than a single packet in a frame at a any time, I place all received data into the string , reinterpret_cast to my data struct, calculate the total length of the packet, then copy the data portion of the packet into a string for regex processing, At this point i do a string.erase, as in mybuff.Erase(totalPackLen); <~ THIS is whats calling the assert, but totalpacklen is less than the strings size.
Is there some convention I am missing here? Or is it that the std::string really is an inappropriate choice here? Ty.
Fixed it on my own. Rolled my own VERY simple buffer with a few C calls :)
int ret = recv(socket,m_buff,0);
if(ret > 0)
{
BigBuff.append(m_buff,ret);
while(BigBuff.size() > 16){
Header *hdr = reinterpret_cast<Header*>(&BigBuff[0]);
if(ntohs(hdr->PackLen) <= BigBuff.size() - 20){
hdr->PackLen = ntohs(hdr->PackLen);
string lData;
lData.append(BigBuff.begin() + 20,BigBuff.begin() + 20 + hdr->PackLen);
Parse(lData); //regex parsing helper function
BigBuff.erase(hdr->PackLen + 20); //assert here when len is packlen is 235 and string len is 1458;
}
}
}
From the code snippet you provided it appears that your packet comprises a fixed-length binary header followed by a variable length ASCII string as a payload. Your first mistake is here:
BigBuff.append(m_buff,ret);
There are at least two problems here:
1. Why the append? You presumably have dispatched with any previous messages. You should be starting with a clean slate.
2. Mixing binary and string data can work, but more often than not it doesn't. It is usually better to keep the binary and ASCII data separate. Don't use std::string for non-string data.
Append adds data to the end of the string. The very next statement after the append is a test for a length of 16, which says to me that you should have started fresh. In the same vein you do that reinterpret cast from BigBuff[0]:
Header *hdr = reinterpret_cast<Header*>(&BigBuff[0]);
Because of your use of append, you are perpetually dealing with the header from the first packet received rather than the current packet. Finally, there's that erase:
BigBuff.erase(hdr->PackLen + 20);
Many problems here:
- If the packet length and the return value from recv are consistent the very first call will do nothing (the erase is at but not past the end of the string).
- There is something very wrong if the packet length and the return value from recv are not consistent. It might mean, for example, that multiple physical frames are needed to form a single logical frame, and that in turn means you need to go back to square one.
- Suppose the physical and logical frames are one and the same, you're still going about this all wrong. As noted, the first time around you are erasing exactly nothing. That append at the start of the loop is exactly what you don't want to do.
Serialization oftentimes is a low-level concept and is best treated as such.
Your comment doesn't make sense:
BigBuff.erase(hdr->PackLen + 20); //assert here when len is packlen is 235 and string len is 1458;
BigBuff.erase(hdr->PackLen + 20) will erase from hdr->PackLen + 20 onwards till the end of the string. From the description of the code - seems to me that you're erasing beyond the end of the content data. Here's the reference for std::string::erase() for you.
Needless to say that std::string is entirely inappropriate here, it should be std::vector.