I'm writing a small sockets program (GNU libc). I have a loop which asks user for input (e.g. "MSG> "). When the user presses enter the message is sent (currently to a server on localhost).
Anyway, I want to read from stdin into a char buffer[256]. I'm currently using fgets() which doesn't do what I want. I'm not sure how write the code such that I ask the user and then get data 256 -1 bytes at a time so that I can send a c-string of 1000 bytes via several strings of 256 bytes.
EDIT: Add code
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#define BUFSIZE 256
int main(int argc, char *argv[])
{
char msg[BUFSIZE];
size_t msgLen;
msgLen = strlen(fgets(msg, BUFSIZE, stdin));
puts(msg);
// This simply checks whether we managed to fill the buffer and tries to get
// more input
while (msgLen == (BUFSIZE - 1))
{
memset (msg, '\0', BUFSIZE);
fread(msg, BUFSIZE, 1, stdin);
msg[BUFSIZE - 1] = '\0';
msgLen = strlen(msg);
puts(msg);
if (msgLen < (BUFSIZE - 1))
break;
}
return 0;
}
You're implementing a loop to ensure 1000 bytes are recieved, right? The loop should indicate that it's counting up to 1000, for the sake of legibility. Keep track of the number of bytes you read (using the += operator), and use that number in the loop condition.
You seem to assume that fread will read 255 bytes, but this is under the invalid assumption that 255 bytes are available. When less than 255 bytes are read, this doesn't necessarily indicate an error; keep reading! When the return value of fread is less than zero, then you should be worried. Make sure you handle those situations.
How about this:
fread(buffer, sizeof(buffer), 1, stdin);
If you're using fgets(), you're using the standard IO library. You're going to want to use fread() then (instead of, say, read(), which uses a file descriptor), to specify the number of bytes to read. See: http://www.cplusplus.com/reference/cstdio/fread/
You may consider using the read function for buffered input. It takes an open file descriptor (STDIN_FILENO for stdin), a pointer to a buffer (char *) and the count of bytes to read. See the manual entry for more details.
Related
I know C++ for quite long, but started using it for my purposes some what a year and a half ago.
I started learning network programming on C++ and the first networking project is "File Transfering between hosts over TCP/IP" which sounds kind easy but I am stuck with sending data.
I am trying to send small buffer less than 4KB, so buffer[4096] works fine for me, but I am planning to expand this. WSAStartup(), socket(), bind(), listen(), accept() functions work fine and values for them are initialised for both Server and Client, but I am dealing with other problems, maybe recv(), send() etc.
I still couldn't find the source of the problem.
Also it would be a ton helpful if somebody give me an example of transfering files over TCP/IP, but not in one packet, I want the file to be chunked and sent in parts or as it's called "ring model", but I couldn't find a working model;
P.S. This is first time I am asking here, pls give feedback about how well all of this is written, so that I could write more informative for community help, thanks)
Server
char* buffer = new char[4096];
ZeroMemory(buffer, sizeof(buffer));
ofstream file("a.txt", ios::binary);
int err = recv(conn, buffer, sizeof(buffer), 0);
file << buffer;
file.close();
if (err == 0)
{
printf("Client diconnected...\n");
}
printf("Quitting...\n");
delete[] buffer;
Client
ifstream file("a.txt", ios::binary);
file.seekg(0, ios::end);
int size = file.tellg();
file.seekg(0, ios::beg);
char* buffer = new char[size];
file.read(buffer, size);
file.close();
int err = send(client, buffer, size, 0);
if (err == 0)
{
printf("Disconnecting...\n");
}
printf("Quitting...\n");
delete[] buffer;
"a.txt" file on Client side is 45 bytes in here are 45 * 'a'
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
And this is what I get on Server side, file size is 14 bytes
aaaaaaaa ’pÈ/
In C++, sizeof(buffer) is the size of the pointer type.
You may want to read up on more modern (as in after 1998) C++. We have std::vector nowadays, and that has a convenient size method. It would return 4096 for your buffer. Also, vector handles new[] and delete[] for you.
The fact that you get 8 "a"'s suggests that you built for x64. The remaining bytes are garbage; you should check how many bytes recv actually wrote to buffer. You cannot assume that you got all the bytes you asked for (whether that's 8 or 4096).
I believe that sizeof(buffer) in this line -->
int err = recv(conn, buffer, sizeof(buffer), 0);
will return sizeof(char*) which is 4 bytes in a 32 bit program or 8 bytes in a 64 bits program instead of 4096 because it is not a static array as in you did not declare it as char buffer[4096]. So, either declare it as char buffer[4096] or convert the above code to
int err = recv(conn, buffer, 4096, 0);
Two additional points:
TCP is a streaming protocol (not "message based"), so there's no guarantee that a single recv() will get everything sent in a single send().
The server line file << buffer; assumes buffer is zero terminated.
MSDN state that :
If no error occurs, recv returns the number of bytes received and the
buffer pointed to by the buf parameter will contain this data
received. If the connection has been gracefully closed, the return
value is zero.
Otherwise, a value of SOCKET_ERROR is returned, and a specific error
code can be retrieved by calling WSAGetLastError.
https://learn.microsoft.com/en-us/windows/win32/api/winsock/nf-winsock-recv
test if you actually read 45 bytes and check if there's an error (WSAGetLastError function)
Introduction and general objective
I am trying to send an image from a child process (generated by calling popen from the parent) to the parent process.
The image is a grayscale png image. It is opened with the OpenCV library and encoded using imencode function of the same library. So the resulting encoded data is stored into a std::vector structure of type uchar, namely the buf vector in the code below.
No error in sending preliminary image information
First the child sends the following image information needed by the parent:
size of the buf vector containing the encoded data: this piece of information is needed so that the parent will allocate a buffer of the same size where to write the image information that it will receive from the child. Allocation is performed as follows (buf in this case is the array used to received data not the vector containing the encoded data):
u_char *buf = (u_char*)malloc(val*sizeof(u_char));
number of rows of the original image: needed by the parent to decode the image after all data have been received;
number of columns of the original image: needed by the parent to decode the image after all data have been received.
These data are written by the child on the standard output using cout and read by the parent using fgets system call.
This pieces of information are correctly sent and received so no problem until now.
Sending image data
The child writes the encoded data (i.e. the data contained in the vector buf) to the standard output using write system call while the parent uses the file-descriptor returned by popen to read the data. Data is read using read system call.
Data writing and reading is performed in blocks of 4096 bytes inside while loops. The writing line is the following:
written += write(STDOUT_FILENO, buf.data()+written, s);
where STDOUT_FILENO tells to write on standard output.
buf.data() returns the pointer to the first element in the array used internally by the vector structure.
written stores the number of bytes that have been written until now and it is used as index. s is the number of bytes (4096) that write will try to send each time.
write returns the number of bytes that actually have been written and this is used to update written.
Data reading is very similar and it is performed by the following line:
bytes_read = read(fileno(fp), buf+total_bytes, bytes2Copy);
fileno(fp) is telling from where to read data (fp is the filedescriptor returned by popen). buf is the array where received data is stored and total_bytes are the number of bytes read until now so it is used as index. bytes2Copy is the number of bytes expected to be received: it is wither BUFLEN (i.e. 4096) or for the last block of data the remaining data (if for example the total bytes are 5000 then after 1 block of 4096 bytes another block of 5000-4096 is expected).
The code
Consider this example. The following is a process launching a child process with popen
#include <stdlib.h>
#include <unistd.h>//read
#include "opencv2/opencv.hpp"
#include <iostream>
#define BUFLEN 4096
int main(int argc, char *argv[])
{
//file descriptor to the child process
FILE *fp;
cv::Mat frame;
char temp[10];
size_t bytes_read_tihs_loop = 0;
size_t total_bytes_read = 0;
//launch the child process with popen
if ((fp = popen("/path/to/child", "r")) == NULL)
{
//error
return 1;
}
//read the number of btyes of encoded image data
fgets(temp, 10, fp);
//convert the string to int
size_t bytesToRead = atoi((char*)temp);
//allocate memory where to store encoded iamge data that will be received
u_char *buf = (u_char*)malloc(bytesToRead*sizeof(u_char));
//some prints
std::cout<<bytesToRead<<std::endl;
//initialize the number of bytes read to 0
bytes_read_tihs_loop=0;
int bytes2Copy;
printf ("bytesToRead: %ld\n",bytesToRead);
bytes2Copy = BUFLEN;
while(total_bytes_read<bytesToRead &&
(bytes_read_tihs_loop = read(fileno(fp), buf+total_bytes_read, bytes2Copy))
)
{
//bytes to be read at this iteration: either 4096 or the remaining (bytesToRead-total)
bytes2Copy = BUFLEN < (bytesToRead-total_bytes_read) ? BUFLEN : (bytesToRead-total_bytes_read);
printf("%d btytes to copy\n", bytes2Copy);
//read the bytes
printf("%ld bytes read\n", bytes_read_tihs_loop);
//update the number of bytes read
total_bytes_read += bytes_read_tihs_loop;
printf("%lu total bytes read\n\n", total_bytes_read);
}
printf("%lu bytes received over %lu expected\n", total_bytes_read, bytesToRead);
printf("%lu final bytes read\n", total_bytes_read);
pclose(fp);
cv::namedWindow( "win", cv::WINDOW_AUTOSIZE );
frame = cv::imdecode(cv::Mat(1,total_bytes_read,0, buf), 0);
cv::imshow("win", frame);
return 0;
}
and the process opened by the above corresponds to the following:
#include <unistd.h> //STDOUT_FILENO
#include "opencv2/opencv.hpp"
#include <iostream>
using namespace std;
using namespace cv;
#define BUFLEN 4096
int main(int argc, char *argv[])
{
Mat frame;
std::vector<uchar> buf;
//read image as grayscale
frame = imread("test.png",0);
//encode image and put data into the vector buf
imencode(".png",frame, buf);
//send the total size of vector to parent
cout<<buf.size()<<endl;
unsigned int written= 0;
int i = 0;
size_t toWrite = 0;
//send until all bytes have been sent
while (written<buf.size())
{
//send the current block of data
toWrite = BUFLEN < (buf.size()-written) ? BUFLEN : (buf.size()-written);
written += write(STDOUT_FILENO, buf.data()+written, toWrite);
i++;
}
return 0;
}
The error
The child reads an image, encodes it and sends first the dimensions (size, #rows, #cols) to the parent and then the encoded image data.
The parent reads first the dimensions (no prob with that), then it starts reading data. Data is read 4096 bytes at each iteration. However when less than 4096 bytes are missing, it tries to read only the missing bytes: in my case the last step should read 1027 bytes (115715%4096), but instead of reading all of them it just reads `15.
What I got printed for the last two iterations is:
4096 btytes to copy
1034 bytes read
111626 total bytes read
111626 bytes received over 115715 expected
111626 final bytes read
OpenCV(4.0.0-pre) Error: Assertion failed (size.width>0 && size.height>0) in imshow, file /path/window.cpp, line 356
terminate called after throwing an instance of 'cv::Exception'
what(): OpenCV(4.0.0-pre) /path/window.cpp:356: error: (-215:Assertion failed) size.width>0 && size.height>0 in function 'imshow'
Aborted (core dumped)
Why isn't read reading all the missing bytes?
I am working on this image:
There might be errors also on how I am trying to decode back the image so any help there would be appreciated too.
EDIT
In my opinion as opposed to some suggestions the problem is not related to the presence of \n or \r or \0.
In fact when I print data received as integer with the following lines:
for (int ii=0; ii<val; ii++)
{
std::cout<<(int)buf[ii]<< " ";
}
I see 0, 10 and 13 values (the ASCII values of the above mentioned characters) in the middle of data so this makes me think it is not the problem.
fgets(temp, 10, fp);
...
read(fileno(fp), ...)
This cannot possibly work.
stdio routines are buffered. Buffers are controlled by the implementation. fgets(temp, 10, fp); will read an unknown number of bytes from the file and put it in a buffer. These bytes will never be seen by low level file IO again.
You never, ever, use the same file with both styles of IO. Either do everything with stdio, or do everything with low-level IO. The first option is the easiest by far, you just replace read with fread.
If for some ungodly reason known only to the evil forces of darkness you want to keep both styles of IO, you can try that by calling setvbuf(fp, NULL, _IOLBF, 0) before doing anything else. I have never done that and cannot vouch for this method, but they say it should work. I don't see a single reason to use it though.
On a possibly unrelated, note, your reading loop has some logic in its termination condition that is not so easy to understand and could be invalid. The normal way to read a file looks approximately as follows:
left = data_size;
total = 0;
while (left > 0 &&
(got=read(file, buf+total, min(chunk_size, left))) > 0) {
left -= got;
total += got;
}
if (got == 0) ... // reached the end of file
else if (got < 0) ... // encountered an error
The more correct way would be to try again if got < 0 && errno == EINTR, so the modified condition could look like
while (left > 0 &&
(((got=read(file, buf+total, min(chunk_size, left))) > 0) ||
(got < 0 && errno == EINTR))) {
but at this point readability starts to suffer and you may want to split this in separate statements.
You're writing binary data to standard output, which is expecting text. Newline characters (\n) and/or return characters (\r) can be added or removed depending on your systems encoding for end-of-line in text files. Since you're missing characters, it appears that you system is removing one of those two characters.
You need to write your data to a file that you open in binary mode, and you should read in your file in binary.
Updated Answer
I am not the world's best at C++, but this works and will give you a reasonable starting point.
parent.cpp
#include <stdlib.h>
#include <unistd.h>
#include <iostream>
#include "opencv2/opencv.hpp"
int main(int argc, char *argv[])
{
// File descriptor to the child process
FILE *fp;
// Launch the child process with popen
if ((fp = popen("./child", "r")) == NULL)
{
return 1;
}
// Read the number of bytes of encoded image data
std::size_t filesize;
fread(&filesize, sizeof(filesize), 1, fp);
std::cout << "Filesize: " << filesize << std::endl;
// Allocate memory to store encoded image data that will be received
std::vector<uint8_t> buffer(filesize);
int bufferoffset = 0;
int bytesremaining = filesize;
while(bytesremaining>0)
{
std::cout << "Attempting to read: " << bytesremaining << std::endl;
int bytesread = fread(&buffer[bufferoffset],1,bytesremaining,fp);
bufferoffset += bytesread;
bytesremaining -= bytesread;
std::cout << "Bytesread/remaining: " << bytesread << "/" << bytesremaining << std::endl;
}
pclose(fp);
// Display that image
cv::Mat frame;
frame = cv::imdecode(buffer, -CV_LOAD_IMAGE_ANYDEPTH);
cv::imshow("win", frame);
cv::waitKey(0);
}
child.cpp
#include <cstdio>
#include <cstdint>
#include <vector>
#include <fstream>
#include <cassert>
#include <iostream>
int main()
{
std::FILE* fp = std::fopen("image.png", "rb");
assert(fp);
// Seek to end to get filesize
std::fseek(fp, 0, SEEK_END);
std::size_t filesize = std::ftell(fp);
// Rewind to beginning, allocate buffer and slurp entire file
std::fseek(fp, 0, SEEK_SET);
std::vector<uint8_t> buffer(filesize);
std::fread(buffer.data(), sizeof(uint8_t), buffer.size(), fp);
std::fclose(fp);
// Write filesize to stdout, followed by PNG image
std::cout.write((const char*)&filesize,sizeof(filesize));
std::cout.write((const char*)buffer.data(),filesize);
}
Original Answer
There are a couple of issues:
Your while loop writing the data from the child process is incorrect:
while (written<buf.size())
{
//send the current block of data
written += write(STDOUT_FILENO, buf.data()+written, s);
i++;
}
Imagine your image is 4097 bytes. You will write 4096 bytes the first time through the loop and then try and write 4096 (i.e. s) bytes on the second pass when there's only 1 byte left in your buffer.
You should write whichever is the lesser of 4096 and bytes remaining in buffer.
There's no point sending the width and height of the file, they are already encoded in the PNG file you are sending.
There's no point calling imread() in the child to convert the PNG file from disk into a cv::Mat and then calling imencode() to convert it back into a PNG to send to the parent. Just open() and read the file as binary and send that - it is already a PNG file.
I think you need to be clear in your mind whether you are sending a PNG file or pure pixel data. A PNG file will have:
PNG header,
image width and height,
date of creation,
color type, bit-depth
compressed, checksummed pixel data
A pixel-data only file will have:
RGB, RGB, RGB, RGB
I have a client/server application in C++. The server sends a rather large file(27KB) to the client. The client reads from the socket fixed length of 1024 bytes, which I then concatenate to a string. However, when I use the += operator, it doesn't seem to allocate more than 4048 bytes and I end up with a 4KB file on the clients side.
Clients code:
#define BUFFER_SIZE 1024
string outStr="";
char buf[BUFFER_SIZE];
while(1){
int numread;
if ((numread = read(clientSocket, buf, sizeof(buf) -1)) == -1){
fprintf(stderr,"Error: reading from socket");
exit(1);
}
fprintf(stderr,"received answer with numread: %d\n",numread);
if (numread == 0){
break;
}
buf[numread] = '\0';
outStr+=buf;
}
fprintf(stderr,"Transmission is over with total length: %d\n",outStr.length());
The output i get is:
26 times:
received answer with numread: 1023
and after that:
received answer with numread: 246
received answer with numread: 0
transmission is over with total length: 4048
The outputs confirms that the whole file is transferred, but the concatenation does not let me append over the (system limit?) of 4048. However, the c++ string should automatically reallocate its memory when the content needs to be larger. So why is this happening?
Thank you for answers.
You could use str::append (overload no. 4) and explicitly provide the number of bytes to append. This will then properly also append null bytes. So, instead of:
buf[numread] = '\0';
outStr+=buf;
do
outStr.append(numread, buf);
Strings end at '\0' so if your byte array from the socket has something like that, when you concatenate at the end of your response string it will only concatenate until that point. So I think you should use a std::vector to store the whole response.
I'm having a problem with unix local sockets. While reading a message that's longer than my temp buffer size, the request takes too long (maybe indefinitely).
Added after some tests:
there is still problem with freeze at ::recv. when I send (1023*8) bytes or less to the UNIX socket - all ok, but when sended more than (1023*9) - i get freeze on recv command.
maybe its FreeBSD default UNIX socket limit or C++ default socket settings? Who know?
i made some additational tests and I am 100% sure that its "freeze" on the last 9th itteration when executing ::recv command, when trying to read message >= (1023*9) bytes long. (first 8th itterationg going well.)
What I'm doing:
The idea is to read in a do/while loop from a socket with
::recv (current_socket, buf, 1024, 0);
and check buf for a SPECIAL SYMBOL. If not found:
merge content of buffer to stringxxx += buf;
bzero temp buf
continue the ::recv loop
How do I fix the issue with the request taking too long in the while loop?
Is there a better way to clear the buffer? Currently, it's:
char buf [1025];
bzero(buf, 1025);
But I know bzero is deprecated in the new c++ standard.
EDIT:
*"Why need to clean the buffer*
I see questions at comments with this question. Without buffer cleanup on the next(last) itteration of reading to the buffer, it will contain the "tail" of first part of the message.
Example:
// message at the socket is "AAAAAACDE"
char buf [6];
::recv (current_socket, buf, 6, 0); // read 6 symbols, buf = "AAAAAA"
// no cleanup, read the last part of the message with recv
::recv (current_socket, buf, 6, 0);
// read 6 symbols, but buffer contain only 3 not readed before symbols, therefore
// buf now contain "CDEAAA" (not correct, we waiting for CDE only)
When your recv() enters an infinite loop, this probably means that it's not making any progress whatsoever on the iterations (i.e., you're always getting a short read of zero size immediately, so your loop never exits, because you're not getting any data). For stream sockets, a recv() of zero size means that the remote end has disconnected (it's something like read()ing from a file when the input is positioned at EOF also gets you zero bytes), or at least that it has shut down the sending channel (that's for TCP specifically).
Check whether your PHP script is actually sending the amount of data you claim it sends.
To add a small (non-sensical) example for properly using recv() in a loop:
char buf[1024];
std::string data;
while( data.size() < 10000 ) { // what you wish to receive
::ssize_t rcvd = ::recv(fd, buf, sizeof(buf), 0);
if( rcvd < 0 ) {
std::cout << "Failed to receive\n"; // Receive failed - something broke, see errno.
std::abort();
} else if( !rcvd ) {
break; // No data to receive, remote end closed connection, so quit.
} else {
data.append(buf, rcvd); // Received into buffer, attach to data buffer.
}
}
if( data.size() < 10000 ) {
std::cout << "Short receive, sender broken\n";
std::abort();
}
// Do something with the buffer data.
Instead of bzero, you can just use
memset(buf, 0, 1025);
These are 2 separate issues. The long time is probably some infinite loop due to a bug in your code and has nothing to do with the way you clear your buffer. As a matter of fact you shouldn't need to clear the buffer; receive returns the number of bytes read, so you can scan the buffer for your SPECIAL_SYMBOL up to that point.
If you paste the code maybe I can help. more.
Just to clarify: bzero is not deprecated in C++ 11. Rather, it's never been part of any C or C++ standard. C started out with memset 20+ years ago. For C++, you might consider using std::fill_n instead (or just using std::vector, which can zero-fill automatically). Then again, I'm not sure there's a good reason to zero-fill the buffer in this case at all.
I have:
char buf[320];
read(soc, buf, sizeof(buf));
//print buf;
However, sometimes the reply is much bigger then 320 characters, so I'm trying to run the read in a loop to avoid taking up too much memory space. I tried read(soc, buf, sizeof(buf)) but that only prints the same first x characters over again. How would I print the leftover characters that did not fit into the first 320 characters in a loop?
Thanks
Change your loop to something like:
int numread;
while(1) {
if ((numread = read(soc, buf, sizeof(buf) - 1)) == -1) {
perror("read");
exit(1);
}
if (numread == 0)
break;
buf[numread] = '\0';
printf("Reply: %s\n", buf);
}
for the reasons Nikola states.
Every time you call read( s, buf, buf_size ) the kernel copies min( buf_size, bytes_available ) into the buf, where bytes_available is the number of bytes already received and waiting in socket receive buffer. The read(2) system call returns the number of bytes placed into application buffer, or -1 on error, or 0 to signal EOF, i.e. a close(2) of the socket on the sending end. Thus when you reuse the buffer, only part of it might be overwritten with new data. Also note that -1 evaluates to true in C and C++. This is probably the case you are hitting.
printf(3) expects zero-terminated string for the %s format specifier. The bytes read from the socket might not contain the '\0' byte, thus letting printf(3) print till it finds zero further down somewhere. This might lead to buffer overrun.
The points here are:
Always check the value returned from read(2)
If you print strings read from a socket - always zero-terminate them manually.
Hope this helps.