C++ Read From Socket into std::string - c++

I am writing a program in c++ that uses c sockets. I need a function to receive data that I would like to return a string. I know this will not work:
std::string Communication::recv(int bytes) {
std::string output;
if (read(this->sock, output, bytes)<0) {
std::cerr << "Failed to read data from socket.\n";
}
return output;
}
Because the read()* function takes a char array pointer for an argument. What is the best way to return a string here? I know I could theoretically read the data into a char array then convert that to a string but that seems wasteful to me. Is there a better way?
*I don't actually mind using something other that read() if there is a more fitting alternative
Here is all of the code on pastebin which should expire in a week. If I don't have an answer by then I will re-post it: http://pastebin.com/HkTDzmSt
[UPDATE]
I also tried using &output[0] but got the output contained the following:
jello!
[insert a billion bell characters here]
"jello!" was the data sent back to the socket.

Here are some functions that should help you accomplish what you want. It assumes you'll only receive ascii character from the other end of the socket.
std::string Communication::recv(int bytes) {
std::string output(bytes, 0);
if (read(this->sock, &output[0], bytes-1)<0) {
std::cerr << "Failed to read data from socket.\n";
}
return output;
}
or
std::string Communication::recv(int bytes) {
std::string output;
output.resize(bytes);
int bytes_received = read(this->sock, &output[0], bytes-1);
if (bytes_received<0) {
std::cerr << "Failed to read data from socket.\n";
return "";
}
output[bytes_received] = 0;
return output;
}
When printing the string, be sure to use cout << output.c_str() since string overwrite operator<< and skip unprintable character until it reaches size. Ultimately, you could also resize at the end of the function to the size received and be able to use normal cout.
As pointed out in comments, sending the size first would also be a great idea to avoid possible unnecessary memory allocation by the string class.

Related

How to send image data over linux socket

I have a relatively simple web server I have written in C++. It works fine for serving text/html pages, but the way it is written it seems unable to send binary data and I really need to be able to send images.
I have been searching and searching but can't find an answer specific to this question which is written in real C++ (fstream as opposed to using file pointers etc.) and whilst this kind of thing is necessarily low level and may well require handling bytes in a C style array I would like the the code to be as C++ as possible.
I have tried a few methods, this is what I currently have:
int sendFile(const Server* serv, const ssocks::Response& response, int fd)
{
// some other stuff to do with headers etc. ........ then:
// open file
std::ifstream fileHandle;
fileHandle.open(serv->mBase + WWW_D + resource.c_str(), std::ios::binary);
if(!fileHandle.is_open())
{
// error handling code
return -1;
}
// send file
ssize_t buffer_size = 2048;
char buffer[buffer_size];
while(!fileHandle.eof())
{
fileHandle.read(buffer, buffer_size);
status = serv->mSock.doSend(buffer, fd);
if (status == -1)
{
std::cerr << "Error: socket error, sending file\n";
return -1;
}
}
return 0
}
And then elsewhere:
int TcpSocket::doSend(const char* message, int fd) const
{
if (fd == 0)
{
fd = mFiledes;
}
ssize_t bytesSent = send(fd, message, strlen(message), 0);
if (bytesSent < 1)
{
return -1;
}
return 0;
}
As I say, the problem is that when the client requests an image it won't work. I get in std::cerr "Error: socket error sending file"
EDIT : I got it working using the advice in the answer I accepted. For completeness and to help those finding this post I am also posting the final working code.
For sending I decided to use a std::vector rather than a char array. Primarily because I feel it is a more C++ approach and it makes it clear that the data is not a string. This is probably not necessary but a matter of taste. I then counted the bytes read for the stream and passed that over to the send function like this:
// send file
std::vector<char> buffer(SEND_BUFFER);
while(!fileHandle.eof())
{
fileHandle.read(&buffer[0], SEND_BUFFER);
status = serv->mSock.doSend(&buffer[0], fd, fileHandle.gcount());
if (status == -1)
{
std::cerr << "Error: socket error, sending file\n";
return -1;
}
}
Then the actual send function was adapted like this:
int TcpSocket::doSend(const char* message, int fd, size_t size) const
{
if (fd == 0)
{
fd = mFiledes;
}
ssize_t bytesSent = send(fd, message, size, 0);
if (bytesSent < 1)
{
return -1;
}
return 0;
}
The first thing you should change is the while (!fileHandle.eof()) loop, because that will not work as you expect it to, in fact it will iterate once too many because the eof flag isn't set until after you try to read from beyond the end of the file. Instead do e.g. while (fileHandle.read(...)).
The second thing you should do is to check how many bytes was actually read from the file, and only send that amount of bytes.
Lastly, you read binary data, not text, so you can't use strlen on the data you read from the file.
A little explanations of the binary file problem: As you should hopefully know, C-style strings (the ones you use strlen to get the length of) are terminated by a zero character '\0' (in short, a zero byte). Random binary data can contain lots of zero bytes anywhere inside it, and it's a valid byte and doesn't have any special meaning.
When you use strlen to get the length of binary data there are two possible problems:
There's a zero byte in the middle of the data. This will cause strlen to terminate early and return the wrong length.
There's no zero byte in the data. That will cause strlen to go beyond the end of the buffer to look for the zero byte, leading to undefined behavior.

Apache module - get request body

I am creating simple apache module to capture all HTTP traffic for real time processing by security software. My goal is to get headers and body from both request and response. So far I managed to get all i need except request body. What's the best way to get request body in output filter, or in any other hook/handler to get request-response "tuple" with all releated information ?
static apr_status_t ef_output_filter(ap_filter_t *f, apr_bucket_brigade *bb)
{
apr_status_t rv;
request_rec *r = f->r;
apr_bucket *e = APR_BRIGADE_FIRST(bb);
const char *data;
apr_size_t length;
std::ofstream outfile;
outfile.open("/var/log/apache2/test.txt", std::ios_base::app);
outfile << r->method << r->unparsed_uri << std::endl;
apr_table_do(loop_table, &outfile, r->headers_in, NULL);
//!!! READ REQUST BODY HERE !!!!
outfile << r->status << std::endl;
apr_table_do(loop_table, &outfile, r->headers_out, NULL);
outfile << std::endl;
while (e != APR_BRIGADE_SENTINEL(bb)) {
apr_bucket_read(e, &data, &length, APR_BLOCK_READ);
e = APR_BUCKET_NEXT(e);
outfile << data;
}
outfile.flush();
outfile.close();
return ap_pass_brigade(f->next, bb);
}
Any help appriciated
You can read the body from the request_rec pointer you're deriving from the ap_filter_t pointer variable.
As a first step, you should tell apache you want to read data from the client, by calling ap_setup_client_block, passing the request_rec pointer and a "read policy" as argument.
Second, you call ap_should_client_block (passing the request_rec pointer as argument) to check everything is OK, especially on the client side (expecting true as result).
Then you call (as many times as needed) ap_get_client_block, with the request_rec as argument, a buffer where the data will go, and the size of your buffer. You should get as a response the number of bytes read, and the data should be in your buffer. If you tried to read X bytes maximum and got X bytes returned, you should call again to get the remaining bytes. Note that the header "Content-length" should be use to avoid trying to read too many data, which might cause crashes...
So you'd go for something along the lines of:
char buffer[SOME_BUFER_SIZE];
int ret_code = ap_setup_client_block(r, REQUEST_CHUNKED_ERROR);
if (ret_code == OK) {
if (ap_should_client_block(r)) {
int dataBytesRead = ap_get_client_block(r, buffer, SOME_BUFFER_SIZE);
...
}
}
As of writing, you can find more info here: https://docstore.mik.ua/orelly/apache_mod/139.htm or here: http://byteandbits.blogspot.com/2013/09/example-apache-module-for-reading.html
Hope it helps...

Sending a StringStream of Binary Data from Cereal with ENet

I have been working on wrapping ENet into a set of easy to use functions for a few weeks now and seem to have a bit of an issue.
I have a std::stringstream and am attempting to send the contents to a remote machine using ENet then reconstruct the std::stringstream on the remote machine.
The reason I need to use a std::stringstream is due to the fact that I'm serializing my data with the Cereal Serialization Library which requires a stream.
With Azoth's help he has identified that I need to be using std::istringstream and std::ostringstream. Previously I was only using std::stringstream which was causing an exception.
However now an exception is being thrown within Cereal at portable_binary.hpp line 156:
throw Exception("Failed to read " + std::to_string(size) + " bytes from input stream! Read " + std::to_string(readSize));
Here's what I'm doing:
void Send(ENetHost* Host)
{
std::ostringstream SData;
{
cereal::PortableBinaryOutputArchive Archive(SData);
Archive(PacketData);
}
std::string Out = SData.str();
ENetPacket* Packet = enet_packet_create(Out.c_str(), Out.size(), ENET_PACKET_FLAG_RELIABLE);
enet_host_broadcast(Host, 0, Packet);
}
A Cereal Portable Binary Data Archive is constructed to hold a single vector.
The std::ostringstream is sent off to the host using ENet.
This part seems to work okay, I can print the information out before and after and it appears to be the same, albeit some weird symbols, but they print out the same on both ends.
Now a std::istringstream is created on the host with the data we received.
NetPacket(enet_uint8 const* Data)
{
std::istringstream SData(reinterpret_cast<char const*>(Data));
{
cereal::PortableBinaryInputArchive Archive(SData);
Archive(PacketData);
}
}
At this point I receive the exception at line:
Archive(PacketData)
I have a feeling the data is being changed somehow when it's sent through ENet and/or I'm not pulling the data out of the std::ostringstream correctly and/or not putting the data back into the std::istringstream correctly.
Thank you very much for your time I greatly appreciate it.
Disclaimer: I'm not familiar with enet.
You are getting this error because you aren't constructing the std::stringstream properly upon receiving the packet. A send/receive pair should look something like:
my_send_function()
{
std::ostringstream os;
{
cereal::PortableBinaryOutputArchive ar(os);
ar( whatever_needs_to_be_serialized );
} // the binary archives will flush their output
// immediately, but it's better to
// use cereal archives in an RAII matter all the time
std::string data = os.str();
create_packet(data.c_str(), data.size());
// send out
}
And then on the receiving end, something like this:
my_receive_function( uint8_t const * data ) // data came from some packet
{
MyDataType d;
std::istringstream is(reinterpet_cast<char const *>(data));
// this is safe to do since we generated the data using c_str(), which added
// a null terminator to the data
{
cereal::PortableBinaryInputArchive ar(is);
ar( d );
}
}
The basic idea here: use cereal and some ostringstream to generate a string (which is really just an array of bytes), send those raw bytes over the network, pull them into an istringstream, and then have cereal parse that.

Use stringstream to read from TCP socket

I am using a socket library (I'd rather not not use it) whose recv operations works with std::string, but is just a wrapper for one call of the recv socket function, so it is probably that I only got some part of the message I wanted. My first instinct was to go in a loop and append the received string to another string until I get everything, but this seems inefficient. Another possibility was to do the same with a char array, but this seems messy. (I'd have to check the strings size before adding into the array and if it overflowed I need to store the string somewhere until the array is empty again.. )
So I was thinking about using a stringstream. I use a TLV protocol, so I need to first extract two bytes into an unsigned short, then get a certain amount of bytes from the stringstream and then loop again until I reach a delimiter field.
Is there any better way to do this? Am I completely on the wrong track? Are there any best practices? So far I've always only seen direct use of the socket library with char arrays so I'm curious why using `std::string`` with stringstreams could be a bad idea..
Edit: Replying to the comment below: The library is one we use internally, its not public (its nothing special though, mostly just a wrapper around the socket library to add exceptions, etc.).
I should mention that I have a working prototype using the socket library directly.
This works something like:
int lengthFieldSize = sizeof(unsigned short);
int endOfBuffer= 0;//Pointer to last valid position in buffer.
while(true) {
char buffer[RCVBUFSIZE];
while(true) {
int offset= endOfBuffer;
int rs= 0;
rs= recv(sock, buffer+offset, sizeof(buffer)-offset, 0);
endOfBuffer+= rs;
if(rs < 1) {
// Received nothing or error.
break;
} else if(endOfBuffer == RCVBUFSIZE) {
// Buffer full.
break;
} else if(rs > 0 && endOfBuffer > 1) {
unsigned short msglength= 0;
memcpy((char *) &msglength, buffer+endOfBuffer-lengthFieldSize, lengthFieldSize);
if(msglength == 0) {
break; // Received a full transmission.
}
}
}
unsigned int startOfData = 0;
unsigned short protosize= 0;
while(true) {
// Copy first two bytes into protosize (length field)
memcpy((char *) &protosize, buffer+startOfData, lengthFieldSize);
// Is the last length field the delimiter?
// Then reply and return. (We're done.)
// Otherwise: Is the next message not completely in the buffer?
// Then break. (Outer while will take us back to receiving)
if(protosize == 0) {
// Done receiving. Now send:
SendReplyMsg(sock, lengthFieldSize);
// Clean up.
close(sock);
return;
} else if((endOfBuffer-lengthFieldSize-startOfData) < protosize) {
memmove(buffer, buffer+startOfData, RCVBUFSIZE-startOfData);
//Adjust endOfBuffer:
endOfBuffer-=startOfData;
break;
}
startOfData+= lengthFieldSize;
gtControl::gtMsg gtMessage;
if(!gtMessage.ParseFromArray(buffer+startOfData, protosize)) {
cerr << "Failed to parse gtMessage." << endl;
close(sock);
return;
}
// Move position pointer forward by one message (length+pbuf)
startOfData+= protosize;
PrintGtMessage(&gtMessage);
}
}
So basically I have a big loop which contains a receiving loop and a parsing loop. There's a character array being passed back and forth as I can't be sure to have received everything until I actually parse it. I'm trying to replicate this behaviour using "proper" C++ (i.e. std::string)
My first instinct was to go in a loop and append the received string to another string until I get everything, but this seems inefficient.
String concatenation is technically platform dependent, but probably str1 + str2 will require one dynamic allocation and two copies (from str1 and str2). That's sorta slow, but it's far faster than network access! So my first piece of advice would be to go with your first instinct, to find out whether it's both correct and fast enough.
If it's not fast enough, and your profiler shows that the redundant string copies are to blame, consider maintaining a list of strings (std::vector<string*>, perhaps) and joining all the strings together once at the end. This requires some care, but should avoid a bunch of redundant string copying.
But definitely profile first!

Concatenating strings into own protocol

I'm writing networking programming using socket.h to my studies. I have written server and client simple programs that can transfer files between them using buffer size given by user.
Server
void transfer(string name)
{
char *data_to_send;
ifstream myFile;
myFile.open(name.c_str(),ios::binary);
if(myFile.is_open))
{
while(myFile.eof))
{
data_to_send = new char [buffer_size];
myFile.read(data_to_send, buffer_size);
send(data_to_send,buffer_size);
delete [] data_to_send;
}
myFile.close();
send("03endtransmission",buffer_size);
}
else
{
send("03error",buffer_size);
}
}
Client
void download(string name)
{
char *received_data;
fstream myFile;
myFile.open(name.c_str(),ios::out|ios::binary);
if(myFile.is_open())
{
while(1)
{
received_data = new char[rozmiar_bufora];
if((receivedB = recv(sockfd, received_data, buffer_size,0)) == -1) {
perror("recv");
close(sockfd);
exit(1);
}
if(strcmp(received_data,"03endoftransmission") == 0)
{
cout<<"End of transmission"<<endl;
break;
}
else if (strcmp(received_data,"03error") == 0)
{
cout<<"Error"<<endl;
break;
}
myFile.write(received_data,buffer_size);
}
myFile.close();
}
The problem occurs, when I want to implement my own protocol- two chars (control), 32 chars hash, and the rest of package is data. I tried few times to split it and I end up with this code:
Server
#define PAYLOAD 34
void transfer(string name)
{
char hash[] = "12345678901234567890123456789012"; //32 chars
char *data_to_send;
ifstream myFile;
myFile.open(name.c_str(),ios::binary);
if(myFile.is_open))
{
while(myFile.eof))
{
data_to_send = new char [buffer_size-PAYLOAD];
myFile.read(data_to_send, buffer_size-PAYLOAD);
concatenation = new char[buffer_size];
strcpy(concatenation,"02");
strcat(concatenation,hash);
strcat(concatenation,data_to_send);
send(concatenation,buffer_size);
delete [] data_to_send;
delete [] concatenation;
}
myFile.close();
send("03endtransmission",buffer_size);
}
else
{
send("03error",buffer_size);
}
}
Client
void download(string name)
{
char *received_data;
fstream myFile;
myFile.open(name.c_str(),ios::out|ios::binary);
if(myFile.is_open())
{
while(1)
{
received_data = new char[buffer_size];
if((receivedB = recv(sockfd, received_data, buffer_size,0)) == -1) {
perror("recv");
close(sockfd);
exit(1);
}
if(strcmp(received_data,"03endoftransmission") == 0)
{
cout<<"End of transmission"<<endl;
break;
}
else if (strcmp(received_data,"03error") == 0)
{
cout<<"Error"<<endl;
break;
}
control = new char[3];
strcpy(control,"");
strncpy(control, received_data,2);
control[2]='\0';
hash = new char[33];
strcpy(hash,"");
strncpy(hash,received_data+2,32);
hash[32]='\0';
data = new char[buffer_size-PAYLOAD+1];
strcpy(data,"");
strncpy(data,received_data+34,buffer_size-PAYLOAD);
myFile.write(data,buffer_size-PAYLOAD);
}
myFile.close();
}
But this one inputs to file some ^# instead of real data. Displaying "data" to console looks the same on server and client. If you know how I can split it up, I would be very grateful.
You have some issues which may or may not be your problem.
(1) send/recv can return less than you requested. You may ask to receive 30 bytes but only get 10 on the recv call so all of these have to be coded in loops and buffered somewhere until you actually get the number you wanted. Your first set of programs was lucky to work in this regard and probably only because you tested on a limited amount of data. Once you start to push through more data your assumptions on what you are reading (and comparing) will fail.
(2) There is no need to keep allocating char buffers in the loops; allocate them before the loop or just use a local buffer rather than the heap. What you are doing is inefficient and in the second program you have memory leaks because you don't delete them.
(3) You can get rid of the strcpy/strncpy statements and just use memmove()
Your specific problem is not jumping out at me but maybe this will push in the right direction. More information what is being transmitted properly and exactly where in the data you are seeing problems would be helpful.
But this one inputs to file some ^# instead of real data. Displaying
"data" to console looks the same on server and client. If you know how
I can split it up, I would be very grateful.
You say that the data (I presume the complete file rather than the '^#') is the same on both client and server? If this is the case, then your issue is likely writing the data to file, rather than the actual transmission of the data itself.
If this is the case, you'll probably want to check assumptions about how the program writes to file - for example, are you passing in text data to be written to file, or binary data? If you're writing binary data, but it uses the NULL-terminated string, chances are it will quit early treating valid binary information as a NULL.
If it's text mode, you might want to consider initialising all strings with memset to a default character (other than NULL) to see if it's garbage data being out put.
If both server and client display the '^#' (or whatever data), binary based char data would be incompatible with the strcpy/strcat functions as this rely on NULL termination (where-as binary uses size termination instead).
I can't track down the specific problem, but maybe this might offer an insight or two that helps.