I am currently working on a project for school and have ran into an issue with a large amount of data being sent via Telnet. If I send a message less than 10KB it is fine. However if I send a message that is above 10KB, I receive the following error "501 Syntax error - line too long" after a few minutes of it running.
Does anyone know of a better way to implement what I am trying to accomplish, that will preferably work with the send()? The data being sent is 5 pages (in Word) of an image in base64.
Thank you, any help is greatly appreciated.
Here is the code portions that I am currently using, which work, with small amounts of data.
char *MailContents = new char[20000000];
std::ifstream in("C:\\test.txt");
std::string MailData((std::istreambuf_iterator<char(in)),std::istreambuf_iterator<char>());
//The following streams in the data into MailData() from a .txt file.
memcpy(MailContents, MailData.c_str(), MailData.length()); //This takes the data and copies it to MailContents
strcat(MailContents, "\r\n");
send(Connection, MailContents, strlen(MailContents), 0); //The following line will take the data in MailContents and echo it to the Telnet data section to be sent.
send(Connection, ".\r\n", strlen(".\r\n"), 0); //This line terminates the data entry and sends it.
Related
I am writing a server-client application using Winsock in c++ for sending a file line by line and I have a problem in sending huge string. The line size is very huge.
For getting the message from the client by the server I use the code below.
int result;
char message[200];
while (true)
{
recv(newSd, (char*)&message, sizeof(message), 0);
cout << "The Message from client: " << message << ";";
}
The above code working fine if I send small length of the message. But, what I wanted is to send an unknown size of lines in a file.
How to send a big unknown string instead of char message[200];
TCP is a byte stream, it knows nothing about messages or lines or anything like that. When you send data over TCP, all it knows about is raw bytes, not what the bytes represent. It is your responsibility to implement a messaging protocol on top of TCP to delimit the data in some meaningful way so the receiver can know when the data is complete. There are two ways to do that:
send the data length before sending the actual data. The receiver reads the length first, then reads however many bytes the length says.
send a unique terminator after sending the data. Make sure the terminator never appears in the data. The receiver can then read until the terminator is received.
You are not handling either of those in your recv() code, so I suspect you are not handling either of them in your send() code, too (which you did not show).
Since you are sending a text file, you can either:
send the file size, such as in a uint32_t or uint64_t (depending on how large the file is), then send the raw file bytes.
send each text line individually as-is, terminated by a CRLF or bare-LF line break after each line, and then send a final terminator after the last line.
You are also ignoring the return value of recv(), which tells you how many bytes were actually received. It can, and usually does, return fewer bytes than requested, so you must be prepared to call recv() multiple times, usually in a loop, to receive data completely. Same with send().
first of all a little background on my situation:
- Qt/C++ UI desktop application
- embedded device (Stm32l4xx family) +ATWINC1500 wifi module
I'm developing the gui application in order to send commands and files to the emdedded device via sockets.
For simple commands I've done all successfully, but for sending files (text files in GCODE format) I am stuck with some issues.
The embedded device has already a socket management(not written by me, so I have not the possibility to modify the way sockets are managed, coming from third party company), and the reception of that type of files is managed in a way that the API waits for every single line of the file being sent, and then wrotes it into a reserved portion of the flash.
My problem is that when I send file from qt Application(by reading each line and and calling write() on the line, in reality my socket sends an entire chunk of the file, like 50 lines, resulting in my device not managing the file reception.
My sending code is this:
void sendGCODE(const QString fileName)
{
QFile *file = new QFile(fileName,this);
bool result = true;
if (file->open(QIODevice::ReadOnly))
{
while (!file->atEnd())
{
QByteArray bytes(file->readLine());
result = communicationSocket->write(bytes);
communicationSocket->flush();
if(result)
{
console->append("-> GCODE line sent:"+ QString(bytes));
}
else
{
console->append("-> Error sending GCODE line!");
}
}
file->close();
}
}
Have anyone of you guys any hints on what I am doing wrong?
I've already searched and someone suggests on other topic that for this purpose it should be better to use UDP instead of TCP sockets, but unfortunately I cannot touch the embedded-device-side code.
thank you all!
EDIT
After suggestions from comments, I've sniffed tcp packets and the packets are sent correctly(i.e. each packet contains a single line). BUT... at the receiver(device), I understood that there is something regarding memory which is not well managed. an example:
sender sends the line "G1 X470.492 Y599.623 F1000" ; receiver receives correctly the string "G1 X470.492 Y599.623 F1000"
next, if the line length is less than the previous sent, i.e. sending "G1 Z5", the receiver receives: "G1 Z5\n\n.492 Y599.623 F1000", so it is clear that the buffer used to store the data packet is not re-initialized from previous packet content, and the new part overwrites the previous values where the remaining part is from the previous packet
I'm trying to figure out how I could reset that part of memory.
This is all wrong. TCP is not a message-oriented protocol. There is no way to ensure that the TCP packets contain any particular amount of data. The receiver code on the device mustn't expect that either - you perhaps misunderstood the receiver's code, or are otherwise doing something wrong (or the vendor is). What the receiver must do is wait for a packet, add the packet's data to a buffer, then extract and process as many complete lines as it can, then move the remaining data to the beginning of the buffer. And repeat that on every packet.
Thus you're looking for the wrong problem at the wrong place, unless your device never ever had a chance of working. If that device works OK with other software, then your "packetized" TCP assumption doesn't hold any water.
Here's how to proceed:
If the device is commercially available and has been tested to work, then you're looking in the wrong place.
If the device is a new product and still in development, then someone somewhere did something particularly stupid and you either need to fix that stupidity, or have the vendor fix it, or hire a consultant to fix it. But just to be completely clear: that's not how TCP works, and you cannot just accept that "it's how it is".
i am trying to send data via tcp socket to a server. The idea behind that is a really simple chat programm.
The string I am trying to send looks like the following:
1:2:e9e633097ab9ceb3e48ec3f70ee2beba41d05d5420efee5da85f97d97005727587fda33ef4ff2322088f4c79e8133cc9cd9f3512f4d3a303cbdb5bc585415a00:2:xc_[z kxc_[z kxc_[z kxc_[==
As you can see there a few unprintable characters which I don't think are a problem here.
To send this data I am using the following code snippet.
bool tcp_client::send_data(string data)
{
if( send(sock , data.c_str(), strlen(data.c_str()) , 0) < 0)
{
perror("Send failed : ");
return false;
}
return true;
}
After a few minutes of trying things out I came up, that data.c_str() cuts my string of.
The result is:
1:2:e9e633097ab9ceb3e48ec3f70ee2beba41d05d5420efee5da85f97d97005727587fda33ef4ff2322088f4c79e8133cc9cd9f3512f4d3a303cbdb5bc585415a00:2:xc_[z
I think that there is some kind of null sequence inside my string which is a problem for the c_str() function.
Is there a way to send the whole string as I mentioned aboved without cutting it off?
Thanks.
Is there a way to send the whole string as I mentioned aboved without cutting it off?
What about:
send(sock , data.c_str(), data.size() , 0);
There are only two sane ways to send arbitrary data (such as a array of characters) over stream sockets:
On the server: close the socket after data was sent (like in ftp, http 0.9, etc). On the client - read until socket is closed in a loop.
On the server: prefix the data with fixed-length size (nowadays people usualy use 64 bit integers for size, watch out for endiannes). On the client - read the size first (in a loop!), than read the data until size bytes are read (in a loop).
Everything else is going to backfire sooner or later.
I have some code where in, there is a TCP connection between the client and the server. I need to send some xml data to the server and receive a response from it. I am trying to do it like this:
char request[MAX];
sprintf(request, "<attestationRequest><majorVersion>%d</majorVersion><minorVersion>%d</minorVersion></attestationRequest>", major, minor);
write(sockfd,request, length);
while(recv(sockfd, response, MAX, 0) >= 0)
{
cout << "response " << response;
//do something;
}
But the data received is (expected response + request).
If I try to fill the data using QString, the data received is proper.
QString request = QString("<attestationRequest>"
"<majorVersion>%1</majorVersion>"
"<minorVersion>%2</minorVersion>"
"</attestationRequest>")
.arg(major)
.arg(minor)
The rest of the code is same for both the cases. I receive more than 11000 bytes of data when I use sprintf, but 9000 bytes in case of QString. I am not able to understand if I am missing something.
IMO both solution are bad.
It is safer and better to use QXmlStreamWriter. Manual manipulation on text will in most cases end with incorrect encoding or missing escape sequence or other xml format violation.
Other problem is that you didn't gave any data how you perform communication. It is highly probable that there is an error causing your problem.
Most likely your problem is, that UTF-16 data is sent through the connection (that is QString.)
Maybe if you tried the w_char-versions of those commands, it might succeed.
Now the server (implemented with java) will send some stream data to me, my code is like below:
connect(socket, SIGNAL(readyRead()), this, SLOT(read_from_server()));
in the read_from_server():
{
while (socket->bytesAvailable())
{
QString temp = socket->readAll();
}
}
but I find that even the server sent me a string with only several characters, the data is truncated, and my function is called twice, thus temp is the never complete data that I want.
If server send me a longer string, my function may be called three or more times, making me diffficult to know at which time the data is transfered completely.
So anyone can tell me how to completely receive the data easily, without so many steps of bothering? I'm sorry if this is duplicated with some questions else, I couldn't get their answers work for me. Many thanks!
What you're seeing is normal for client-server communication. Data is sent in packets and the readyRead signal is informing your program that there is data available, but has no concept of what or how much data there is, so you have to handle this.
To read the data correctly, you will need a buffer, as mentioned by #ratchetfreak, to append the bytes as they're read from the stream. It is important that you know the format of the data being sent, in order to know when you have a complete message. I have previously used at least two methods to do this: -
1) Ensure that sent messages begin with the size, in bytes, of the message being sent. On receiving data, you start by reading the size and keep appending to your buffer until it totals the size to expect.
2) Send all data in a known format, such as JSON or XML, which can be checked for the end of the message. For example, in the case of JSON, all packets will begin with an opening brace '{' and end with a closing brace '}', so you could count braces and match up the data, or use QJsonDocument::fromRawData to verify that the data is complete.
Having used both of these methods, I recommend using the first; include the size of a message that is being sent.
you can use a buffer field to hold the unfinished data temporarily and handle packets as they complete:
{
while (socket->bytesAvailable())
{
buffer.append(socket->readAll());
int packetSize = getPacketSize(buffer);
while(packetSize>0)
{
handlePacket(buffer.left(packetSize);
buffer.remove(0,packetSize);
packetSize = getPacketSize(buffer);
}
}
}
If all of the data has not yet arrived then your while loop will exit prematurely. You need to use a message format that will let the receiving code determine when the complete message has been received. For example, the message could begin with a length element, or if you are dealing with text the message could end with some character used as a terminator.
Problem is that during tcp data transfer data are send in undefined chunks. If you are trying to read defined block size you have to know in advance expected chunk size ore have a way to determinate when your block ends (something like zero terminated c-string).
Check if this answer doesn't help you (there is a trick to wait for expected data block).