I'm trying to send a binary file through socket in C to an embedded platform, but when I run it after its sent it just gives me segfault (sending through ftp works fine, but its very slow).
Sending the binary file in same system works ok (the embedded is little-endian so I don't think its endian problem).
What can be the problem? the program is mft.cpp
You are assuming that every read returns the number of bytes that you want to read. That is incorrect. You should always check the read return value to see if you got as many bytes as you wanted.
This also means that you can rewrite your send loop as:
int bytesLeft = file_length;
char buf[1024]; //no need to reallocate it in the loop
while(bytesLeft > 0)
{
int to_read = 1024;
if(bytesLeft < to_read)
to_read = bytesLeft
int bytesRead = read(new_sock_id, buf, to_read);
if(error("reading file", false)) continue;
write(file, buf, bytesRead);
if(error("writing file", false)) continue;
bytesLeft -= bytesRead ;
}
Related
I have one question about socket programming in C++. Most of the tutorials I found on web assume that
(binding etc. is omitted)
there is a string at client process
it is saved to a file
then the file is sent to server by first reading the file into stream
server receives the stream and write it into another file.
Then, my question is that what if we can use stringstrem at step 2 instead of saving as a file? File I/O (in C++ ifstream and ofstream are typically used) is generally slow. Is it more efficient if I use stringstream directory?
Your Original Question:
"What if we can use stringstrem at step 2 instead of saving as a file?"
My Initial Response:
stringstream has nothing to do with server sockets and IO files.
You are lacking the fundamental idea of IO operations which is the concept of files for IO devices. There is no way around. You save nothing in a logical file stream. Your file bytes are buffered temporarily on your memory and flushed.
stringstream is a nice C++ library utility that let's you treat strings as file streams. Just like you read from an input file stream bytes after bytes until EOF/some other errors, or write into an output file stream bytes after bytes, using stringstream you can treat your string like the way you do to file streams. This is really helpful when you want to divide your string in small logical units. For example, suppose
you read a string line and want to read each word from that line by treating the string line as a stream of words.
Further Instructions To Guide You To The Right Direction:
Nothing is "saved" in a logical file stream. Every I/O operation is performed through "logical" files in any computer system. Socket connection has two file descriptors on both ends: one is a client file descriptor and another one is a server file descriptor (connected file descriptor). Server listens for connection requests through a listening file descriptor which actually stays around as long as the lifetime of the server, and when it accepts a connection request, it returns another file descriptor through accept function called connected file descriptor that stays around as long as the client-server connection/transaction is ongoing.
int accept(int listenfd, struct sockaddr *addr, int *addrlen);
If you want to read from or write into a file stream and also wish to buffer your file bytes, you exactly need to do that- buffer your bytes. This is also very important in the context of servers and short counts because your connection might time out or it might get interrupted by signals. There are several options and techniques that you might implement. However, such discussions are not possible in this small thread. What I'm going to do based on your question is give you an example of how you can buffer your file stream, avoid short count, and handle signal interruptions through following steps:
For example, following is a function that reads n bytes and doesn't buffer
ssize_t rio_readn(int fd, void *usrbuf, size_t n)
{
size_t nleft = n;
ssize_t nread;
char *bufp = usrbuf;
while (nleft > 0) {
if ((nread = read(fd, bufp, nleft)) < 0) {
if (errno == EINTR) /* Interrupted by sig handler return */
nread = 0;/* and call read() again */
else
return -1;/* errno set by read() */
}
else if (nread == 0)
break;/* EOF */
nleft -= nread;
bufp += nread;
}
return (n - nleft);/* Return >= 0 */
}
We can implement the following steps to do buffered and robust IO operations (note RIO means robust IO):
Step 1: Set up empty read buffer and associate an open file descriptor so that we can implement our robust IO operations
#define RIO_BUFSIZE 8192
typedef struct {
int rio_fd;/* Descriptor for this internal buf */
int rio_cnt;/* Unread bytes in internal buf */
char *rio_bufptr;/* Next unread byte in internal buf */
char rio_buf[RIO_BUFSIZE]; /* Internal buffer */
} rio_t;
//Initialize robust IO buffer
void rio_readinitb(rio_t *rp, int fd)
{
rp->rio_fd = fd;
rp->rio_cnt = 0;
rp->rio_bufptr = rp->rio_buf;
}
Step 2: A robust read utility function to handle short count
static ssize_t rio_read(rio_t *rp, char *usrbuf, size_t n)
{
int cnt;
while (rp->rio_cnt <= 0) {/* Refill if buf is empty */
rp->rio_cnt = read(rp->rio_fd, rp->rio_buf,
sizeof(rp->rio_buf));
if (rp->rio_cnt < 0) {
if (errno != EINTR) /* Interrupted by sig handler return */
return -1;
}
else if (rp->rio_cnt == 0)/* EOF */
return 0;
else
rp->rio_bufptr = rp->rio_buf; /* Reset buffer ptr */
}
/* Copy min(n, rp->rio_cnt) bytes from internal buf to user buf */
cnt = n;
if (rp->rio_cnt < n)
cnt = rp->rio_cnt;
memcpy(usrbuf, rp->rio_bufptr, cnt);
rp->rio_bufptr += cnt;
rp->rio_cnt -= cnt;
return cnt;
}
Step 3: A robust IO function for buffered reading
ssize_t rio_readnb(rio_t *rp, void *usrbuf, size_t n)
{
size_t nleft = n;
ssize_t nread;
char *bufp = usrbuf;
while (nleft > 0) {
if ((nread = rio_read(rp, bufp, nleft)) < 0) {
if (errno == EINTR) /* Interrupted by sig handler return */
nread = 0;/* Call read() again */
else
return -1;/* errno set by read() */
}
else if (nread == 0)
break;/* EOF */
nleft -= nread;
bufp += nread;
}
return (n - nleft);/* Return >= 0 */
}
I know how my packet looks like. It has 6 header fields (1 byte each, each header has 8 fields) and then it has the payload (data).
I would like to build a raw packet in C or C++ (it should look the same I think).
Here's what I think I should do:
unsigned char packet[11];
packet[0] = (0x81); // first header with 8 fields
packet[1] = (0x8c); // second header with 8 fields
packet[2] = (0xfe);
packet[3] = (0x84);
packet[4] = (0x1d);
packet[5] = (0x79);
packet[6] = (0x96); // payload, the 'h' letter, masked
packet[7] = (0xe1); // 'e'
packet[8] = (0x71); // 'l'
packet[9] = (0x15); // 'l'
packet[10] = (0x91);// 'o'
Where, for instance, 0x81 is the first byte (I simply converted every field (bit) of my first header to hex).
And then, simply, I want to send it to server: send(sockfd, packet, sizeof(packet), 0) to send it.
Receiving and printing the response:
unsigned char buffer[1024];
if ((recv(sockfd, buffer, len, 0)) == 0)
{
if (errno != 0)
{
exit(1);
}
}
int i;
for(i = 0; i<len; i++)
printf("%x ", buffer[i]);
Am I right?
Other than mishandling the return value from recv, your code looks okay.
if ((recv(sockfd, buffer, len, 0)) == 0)
{
if (errno != 0)
{
exit(1);
}
}
A zero return indicates normal close of the connection. There's no reason to check errno if it returns zero.
A return value of -1 indicates an error. In that case, it does make sense to check errno.
A value greater than zero indicates that number of bytes have been received. Be aware that it is perfectly normal for recv to return fewer bytes than you asked it for. If you want to receive exactly some number of bytes, you must call recv in a loop.
TCP is a byte-stream protocol and has no idea where your "packets" (really, messages) begin and end.
Your code will not appear to be error-prone!
But a good practice would be:
const std::uint32_t BUFFER_SIZE = 11;
std::vector<std::uint8_t> buffer;
buffer.reserve(BUFFER_SIZE)
buffer = {0x81,0x8c.....};
send( sockfd,
reinterpret_cast <const char*> ( buffer.data() ),
static_cast <int> ( buffer.size() ),
0
);
Doing so, your code gets more optimized, and avoids possible leaks, using the std vectors.
May also benefit from taking a look at ZeroMQ, as an example of a ready-made, high-performance asynchronous messaging library, aimed at use in distributed or concurrent applications.
I have a relatively simple web server I have written in C++. It works fine for serving text/html pages, but the way it is written it seems unable to send binary data and I really need to be able to send images.
I have been searching and searching but can't find an answer specific to this question which is written in real C++ (fstream as opposed to using file pointers etc.) and whilst this kind of thing is necessarily low level and may well require handling bytes in a C style array I would like the the code to be as C++ as possible.
I have tried a few methods, this is what I currently have:
int sendFile(const Server* serv, const ssocks::Response& response, int fd)
{
// some other stuff to do with headers etc. ........ then:
// open file
std::ifstream fileHandle;
fileHandle.open(serv->mBase + WWW_D + resource.c_str(), std::ios::binary);
if(!fileHandle.is_open())
{
// error handling code
return -1;
}
// send file
ssize_t buffer_size = 2048;
char buffer[buffer_size];
while(!fileHandle.eof())
{
fileHandle.read(buffer, buffer_size);
status = serv->mSock.doSend(buffer, fd);
if (status == -1)
{
std::cerr << "Error: socket error, sending file\n";
return -1;
}
}
return 0
}
And then elsewhere:
int TcpSocket::doSend(const char* message, int fd) const
{
if (fd == 0)
{
fd = mFiledes;
}
ssize_t bytesSent = send(fd, message, strlen(message), 0);
if (bytesSent < 1)
{
return -1;
}
return 0;
}
As I say, the problem is that when the client requests an image it won't work. I get in std::cerr "Error: socket error sending file"
EDIT : I got it working using the advice in the answer I accepted. For completeness and to help those finding this post I am also posting the final working code.
For sending I decided to use a std::vector rather than a char array. Primarily because I feel it is a more C++ approach and it makes it clear that the data is not a string. This is probably not necessary but a matter of taste. I then counted the bytes read for the stream and passed that over to the send function like this:
// send file
std::vector<char> buffer(SEND_BUFFER);
while(!fileHandle.eof())
{
fileHandle.read(&buffer[0], SEND_BUFFER);
status = serv->mSock.doSend(&buffer[0], fd, fileHandle.gcount());
if (status == -1)
{
std::cerr << "Error: socket error, sending file\n";
return -1;
}
}
Then the actual send function was adapted like this:
int TcpSocket::doSend(const char* message, int fd, size_t size) const
{
if (fd == 0)
{
fd = mFiledes;
}
ssize_t bytesSent = send(fd, message, size, 0);
if (bytesSent < 1)
{
return -1;
}
return 0;
}
The first thing you should change is the while (!fileHandle.eof()) loop, because that will not work as you expect it to, in fact it will iterate once too many because the eof flag isn't set until after you try to read from beyond the end of the file. Instead do e.g. while (fileHandle.read(...)).
The second thing you should do is to check how many bytes was actually read from the file, and only send that amount of bytes.
Lastly, you read binary data, not text, so you can't use strlen on the data you read from the file.
A little explanations of the binary file problem: As you should hopefully know, C-style strings (the ones you use strlen to get the length of) are terminated by a zero character '\0' (in short, a zero byte). Random binary data can contain lots of zero bytes anywhere inside it, and it's a valid byte and doesn't have any special meaning.
When you use strlen to get the length of binary data there are two possible problems:
There's a zero byte in the middle of the data. This will cause strlen to terminate early and return the wrong length.
There's no zero byte in the data. That will cause strlen to go beyond the end of the buffer to look for the zero byte, leading to undefined behavior.
I'm writing a file transfer client/server application
where the client is operating on windows7 and written in vb.net
and the server is operating on linux mint and written in c++ (I'm using vmware)
my problem is when i try to upload files to the server (such as images) the received data is missing many bytes which also represent the control characters (such as EOT, ETB,...) and I guess they're read as tcp control characters and ignored by the receiving OS.
I already tested the application with simple text files (size up to 4MB) without any problem.
is there a way to prevent the system from ignoring those bytes?
this is the c++ function that receives the file:
string readSockBytes(int port,int num,int size)
{
int dcmbuffSize = 1460;
int n;
stringstream temp;
string strBuffer,Sbuffer;
char Rbuffer[dcmbuffSize];
struct socketVar sockets;
sockets = setSocket(port);
sockets = sockListen(sockets);
cout<<"user connected\n";
strBuffer = readsock(sockets);
cout<<strBuffer.substr(0,strBuffer.find("$"))<<endl;
if(num == atoi(strBuffer.substr(0,strBuffer.find("$")).c_str()))
Sbuffer = "ready$";
else
{
Sbuffer = "exit$";
close(sockets.newsockfd);
close(sockets.sockfd);
}
n = writesock(sockets, Sbuffer, 100);
if (n < 0) error("ERROR writing to socket");
while(strBuffer.length() < fileSize)
{
n = read(sockets.newsockfd,Rbuffer,dcmbuffSize-1);
if (n < 0) error("ERROR reading from socket");
temp.str(Rbuffer);
strBuffer = strBuffer+temp.str();
}
strBuffer = strBuffer.substr(0,size);
return strBuffer;
}
The issue is most likely that you sent binary data. And binary data can contain zeros. And zeroes are the normal string terminator.
This means that when you do temp.str(Rbuffer) (assuming temp is a std::stringstream) then it only gets data from Rbuffer until the first zero.
Instead of using e.g. std::stringstream use std::string:
while(strBuffer.length() < fileSize)
{
char buffer[2048];
ssize_t n = read(sockets.newsockfd, buffer, sizeof(buffer));
if (n <= 0)
{
// An error, or connection closed
if (n < 0)
error("ERROR reading from socket");
break;
}
// Create a string of `n` bytes, including possible string terminators
// and add it to out current buffer
strBuffer += std::string(buffer, n);
}
The important thing to remember here is that you can't use the received data as a string! If it's binary data it will with most certainty contain the string terminator and so you have to treat is as binary data and not a string (even though you can store it in a std::string).
You also need to be aware that you can't print the data, as many binary values are either unprintable or will print as "garbage".
And lastly, if you read and write binary files, you need to open them in binary modes, or you will get errors with the bytes 0x0d and 0x0a (i.e. carriage-return and newline).
How do you loop read for c++ sockets.
stringstream ss;
while (1) {
char buf [2];
ssize_t e = read(fd, buf, 2);
ss << buf[0];
ss << buf[1];
cout << e << endl;
cout << buf[0] << endl;
cout << buf[1] << endl;
if (e <= 0) {
break;
}
}
string msg = "";
ss >> msg;
When I telnet and type hello and then enter to test this out, I get
2
h
e
2
l
l
2
o
1
And it just waits there, no looping. What is going on, and how do I read sockets?
How do I signify EOF using
telnet
buf in the context of write(fd, buf, bufSize);
What do you mean no looping? Typo some more, press enter and you'll see some more output..
If you finished, you have to press Ctrl+D to signal the end of input (EOF).
How do I signify EOF using
1) telnet
2) buf in the context of write(fd, buf, bufSize);
First, it's not EOF that you want to signal. It's "end of message" or "the data you have just received is a complete protocol data unit and you should now process it". And the answer is, however you want, so long as both sides agree.
For example, you could reserve the ~ character specifically to mark the end of a message. To indicate the end of a message, one side sends a ~. The other side knows that when it receives a ~, everything before that is a single message and everything after it is a new message.
You can also send the length of a message prior to sending the actual message. You can do it in text followed by a zero byte or by a line feed. Or you can make each message a line.
You can do it however makes the most sense in your specific application, but you actually have to do it.
There are two cases where TCP does do it for you:
1) When a side is completely done sending and knows the other side will not send anymore data anyway, it can use close on the socket. The other side will read the rest of the data already sent and then read will return 0.
2) When a side is not going to send any more data but the other side might send more data, it can shutdown its side of the connection. It can never send any more data though. The other side will know about this shutdown, and if specifically coded to do so, can continue to send data until it shuts down its half. Then the other side's read will return zero. (Note that if you do this to a program that wasn't designed to take it, it will likely get confused and not realize it can still send data to you.)
Otherwise, it's all up to you.
TCP packets are a stream, not a file. It is up to you how the stream is read.
In general, if you write 20 bytes at one end, you'll get 20 bytes at the other end in one read, barring the use of some common tcp/ip options.
There are assumptions made below. Some of the assumptions are:
Assume all the scaffolding code is there.
Assume I made mistakes and debug it yourself.
Assume all byte order issues are handled.
Assume you're smart enough not to send floating point types as binary.
Some programmers choose to use a length prefixed packet and read the length byte(2) and then read as many bytes as that length represents, like so.
unsigned char buffer[MAX_CHAR] = "";
unsigned char length = 0;
int bytesRead = 0;
read( fd, sizeof( length ), &length );
// Handle failure from read
while( bytesRead < length )
{
int readRv = read( fd, &buffer[bytesRead], length - bytesRead );
if( readRv <= 0 ) // 0 usually means socket was closed. -1 is an erro
{
// Handle socket error/closed socket
}
else if( readRv < length )
{
bytesRead += readRv;
}
else if( readRv == length )
{
bytesRead = readRv;
break;
}
}
Some programmers read what is available and look for an end of packet marker
unsigned char buffer[MAX_CHAR] = "";
unsigned char length = 0;
int bytesRead = 0;
int readRv = read( fd, buffer, sizeof( buffer ) );
int eopFound = 0;
if( (eopFound = findEndOfPacketMarker( buffer )) > 0 )
{
// Less than 0 = error.
}
else if( eopFound == 0 )
{
// No end of packet, must read more bytes here
}
else
{
// Found an end of packet marker
}
// Here you deal with bytes that are "over read"
// Either the
// 1) packet was bigger than buffer,
// 2) there were bytes left over after the EOP marker
// 3) Martians crapped in your code and screwed it all up. :)
Some programmers use a fixed sized set of packets and a packet type id
int packetType;
unsigned char buffer[MAX_CHAR] = "";
unsigned char length = 0;
int bytesRead = 0;
read( fd, sizeof( packetType ), &packetType );
read( fd, getLengthFromPacketType( packetType ), buffer );
That should cover most things...
If you have more questions, please ask in the comments with the #JimR thingy so it shows up as a message for me to read.