I'm having a really hard time understanding why is this piece of code making my computer beep. I've isolated this section of code to be the one producing the occasional beep, but I don't see what's the problem with it.
const int BUFFER_LENGTH = 8192;
char buffer [BUFFER_LENGTH + 1];
int recvResult;
do
{
recvResult = recv(webSocket, buffer, BUFFER_LENGTH, 0);
buffer[recvResult] = '\0';
printf("%s", buffer);
if (recvResult > 0)
{
sendResult = send(clientSocket, buffer, recvResult, 0);
}
}while (recvResult > 0);
shutdown(webSocket, SD_SEND);
To give a little bit of context, this is for a computer networks class in which we have to code a proxy. So what I'm doing is listening to the answer and simply forward it to the client.
I can't tell you how high I jumped out of my chair when I first heard the beeping noise...
The buffer probably contains a '\a' char which makes the computer beep. From 5.2.2 (Character display semantics) :
Alphabetic escape sequences representing nongraphic characters in the execution character set are intended to produce actions on display devices as follows:
\a (alert) Produces an audible or visible alert without changing the active position.
Nevermind, found it, it was actually the printf statement that was doing an occasionnal beep!
Agree with the '\a' beep explanation.
One more point about your code:
recvResult = recv(webSocket, buffer, BUFFER_LENGTH, 0);
buffer[recvResult] = '\0';
Note that recvResult will be -1 if there's an I/O error (or if you're working in the non-blocking mode and no data to read so far).
In such a case you'll write into forbidden memory, which is (damn, how I hate this phrase) undefined behavior. Simply speaking - memory overwrite, which is bad.
You should check for socket error before writing into buffer
Related
I've been working on a HTML / websocket server on a Wiznet W5100S-EVB-Pico, programmed in the Arduino IDE. It all worked fine up until now but I'm running into, I think, a string size limit. I guess it is in the way the code handles the const char but I don't know how to do it properly.
I hope someone is willing to help :)
Let me explain:
I convert the index.html to a index_html.h file containing a const char array:
const char c_index_html[] = {
0x3c,0x21,0x44,0x4f,0x43,..., ..., 0x6d,0x6c,0x3e};
In my code I include the index_html.h file:
#include "index_html.h"
Now the code that actually serves the "HTML"
if (web_client){
Serial.println("New client");
// an http request ends with a blank line
bool currentLineIsBlank = true;
while (web_client.connected()){
if (web_client.available()){
char c = web_client.read();
if (c == '\n' && currentLineIsBlank) // if you've gotten to the end of the line (received a newline
{ // character) and the line is blank, the http request has ended,
Serial.println(F("Sending response")); // so you can send a reply
String strData;
strData = c_index_html;
web_client.println(strData);
break;
}
if (c == '\n')
{
// you're starting a new line
currentLineIsBlank = true;
}
else if (c != '\r')
{
// you've gotten a character on the current line
currentLineIsBlank = false;
}
}
}
This is not the prettiest code, it's smashed together from examples and now the main culprit seems to be:
String strData;
strData = c_index_html;
web_client.println(strData);
When I add extra code to the HTML and view the page source, the code is incomplete. I tested reducing the HTML to a minimum and that solves the problem.
So my main question is:
How do I serve the 'const char c_index_html' without use of 'String'?
But also:
How could I prettify the whole 'if (web_client)' serving function?
Thank you very much for making it all the way through this post and if you have a suggestion I would very much appreciate it ;)
Edit: There is a bug in the ethernet library shown in this post.
I don't know if it affects you; you should look at your library implementation.
I'm assuming that web_client is an instance of EthernetClient from the Arduino libraries.
EthernetClient::println is inherited from Print via Stream and is defined in terms of write, which is:
size_t EthernetClient::write(const uint8_t *buf, size_t size)
{
if (_sockindex >= MAX_SOCK_NUM) return 0;
// This library code is not correct:
if (Ethernet.socketSend(_sockindex, buf, size)) return size;
setWriteError();
return 0;
}
So we see that it asks the socket to send the buffer up to some size. The socket can respond with a size or 0 (see edit); if it responds with 0 then there's an error condition to check.
Edit: This is how it's supposed to work. Since write is always returning the requested size and not telling you how much was written, you can't fix your problem using the print/write facilities and need to directly use socketSend.
You're not checking the result of this write (which is supposed to come through println) so you don't know whether the socket sent size bytes, 0 bytes, or some number in between.
In EthernetClient::connect we see that it's opening a TCP stream:
_sockindex = Ethernet.socketBegin(SnMR::TCP, 0);
When you call socketSend you're actually just copying your buffer into a buffer in the network stack. The TCP driver writes out that buffer when it can. If you're writing into that buffer faster than it's being flushed to the network then you'll fill it up and your socketSend calls will start returning < size bytes. See Does send() always send whole buffer?.
So you're probably right that your string is too long. What you need to do is spread your writes out. There are countless tutorials covering this on the web; it's roughly like this in your example:
...
size_t bytesRemaining = 0;
while (web_client.connected()){
if (bytesRemaining > 0) {
// Still responding to the last request
char const* const cursor = c_index_html
+ sizeof(c_index_html)
- bytesRemaining;
size_t const bytesWritten = web_client.write(cursor, bytesRemaining);
if (!bytesWritten) {
// check for error
}
bytesRemaining -= bytesWritten;
if (bytesRemaining == 0) {
// End the message. This might not write!
// We should add the '\n' to the source array so
// it's included in our write-until-finished routine.
web_client.println();
// Stop listening
break;
}
} else if (web_client.available()){
// Time for a new request
char c = web_client.read();
if (c == '\n' && currentLineIsBlank)
{
Serial.println(F("Sending response"));
// Start responding to this request
bytesRemaining = sizeof(c_index_html);
continue;
}
...
This is what I think is going on. I'm not an expert so I might be wrong, but it seems to make sense.
This is not an answer as in "solution" but I found out there is a 2k buffer size limit using the W5100S-EVB-Pico. And indeed, if I keep the HTML below 2k it works. Turns out that I actually got Matt Murphy's suggestion to work but the 2k limit was the problem. It looks like a hardware/library limitation, not completely sure on that.
For now I'll shrink my HTML and Javascript to a minimum and compact it even more with for example textfixer.com. I think I might write some python code to do that
Maybe there is a path to a solution in the link below but at this moment I'll try to live with the limitations
Link:
https://github.com/khoih-prog/EthernetWebServer/issues/7
I hate to ask this because I think this must be very trivial. But as someone who is used to high-level-languages this is a real problem.
I got a C++ program which uses PDFium to generate an Image to a PDF. And i have a C# program which communicates with the C++ program via Named Pipes. The PDF file (Which is saved as a byte-array) gets transmitted by the pipe. And here is my Problem.
On the 374th Position of the stream is a NUL byte (00) and im too stupid to somehow reach the data after it.
Here is my Code:
LPTSTR lpszPipename2 = TEXT("\\\\.\\pipe\\myNamedPipe2");
hPipe2=CreateFile(lpszPipename2, GENERIC_READ, 0,NULL,OPEN_EXISTING,FILE_FLAG_OVERLAPPED,NULL);
if(ReadFile( hPipe2, chBuf, dwBytesToRead, &cbRead, NULL))
{
PDFData = chBuf;
}
dwBytes to read is the size of the file and cbRead shows the correct number. But PDFData only contains the first 373 bytes. I checked that the data beyond the 373th position is there with the Immediate Window i just don't know how to process it.
I gotta put the Data into a char-array.
As I already said, i think this is very trivial. But although i know where the problem comes from, i have simply no idea how to fix it.
Many Thanks and Regards
Michael
Edit: The C#-Code. Its everything but perfect. But i'm very sure this Problem is on the C++ side.
public void SendRawData(byte[] data)
{
while (clientse == null || clientse.stream == null)
{ }
if (clientse.stream.CanWrite)
{
clientse.stream.Write(data, 0, data.Length);
clientse.stream.Flush();
}
}
private void ListenForClients()
{
while (true)
{
clientHandle = CreateNamedPipe(this.pipeName, DUPLEX | FILE_FLAG_OVERLAPPED, 0, 255, BUFFER_SIZE, BUFFER_SIZE, 0, IntPtr.Zero);
//could not create named pipe
if (clientHandle.IsInvalid)
return;
int success = ConnectNamedPipe(clientHandle, IntPtr.Zero);
//could not connect client
if (success == 0)
return;
clientse = new Client();
clientse.handle = clientHandle;
clientse.stream = new FileStream(clientse.handle, FileAccess.ReadWrite, BUFFER_SIZE, true);
if (ClientType == 0)
{
Thread readThread = new Thread(new ThreadStart(Read));
readThread.Start();
}
}
}
"Solution":
Actually this never was a real problem. I just got my wires crossed. While chBuf seemed after copying it into PDFData or when i read its value is VS to only have those 373 bytes. All ~20 kilobytes were copied to that position.
I knew that, but i didn't understand how the PDFium sources should know that if the string terminates after 373 chars.
Well... the PDFium-sources know it cause i have to pass the length. Which was determined by
size_t len = PDFData.length();
and was therefore of course only 373 bytes.
The null character '\0' is used by C/C++ to terminate char* strings. So any library function (i.e. strlen(), strncpy(), etc) will use the null character as an implicit end-of-string indicator. Your code is obviously doing this somewhere. Instead, use something more like memcpy() or a std::vector<char> with an explicit data length.
Have a look at string:assign (http://www.cplusplus.com/reference/string/string/assign/)
String assignment operator from char * uses the C-style end-of-string convention. You need the "buffer" assign call:
string& assign (const char* s, size_t n);
This will include any NULs.
That being said, vector of bytes may indeed be a better choice.
Actually this never was a real problem. I just got my wires crossed. While chBuf seemed after copying it into PDFData or when i read its value is VS to only have those 373 bytes. All ~20 kilobytes were copied to that position. I knew that, but i didn't understand how the PDFium sources should know that if the string terminates after 373 chars.
Well... the PDFium-sources know it cause i have to pass the length. Which was determined by
size_t len = PDFData.length();
and was therefore of course only 373 bytes.
I'm sorry that i bothered you with that stuff
I have an arduino board that is connected to a sensor. From Arduino IDE serial monitor, I see the readings are mostly 160, 150, etc. Arduino has a 10 bit ADC, so I assume the readings range from 0 to 1024.
I want to fetch that readings to my computer so that I can do further processing. It must be done this way up to this point. Now, I wrote a c++ program to read serial port buffer with Windows APIs (DCB). The transfer speed of the serial ports are set to 115200 on both the Arduino IDE and the c++ program.
I will describe my problem first: Since I want to send the readings to my computer, I expect the data looks like the following:
124
154
342
232
...
But now it looks like
321
43
5
2
123
...
As shown, the data are concatenated. I knew it because I tried to display them with [], and the data are truly messed up.
The section of the code that is doing the serial port reading on the computer is as here:
// Read
int n = 10;
char szBuff[10 + 1] = {0};
DWORD dwBytesRead = 0;
int i;
for (i = 0; i < 200; i++){
{
if(!ReadFile(hSerial, szBuff, n, &dwBytesRead, NULL)){
//error occurred. Report to user.
printf("Cannot read.\n");
}
else{
printf("%s\n" , szBuff);
}
}
}
The Arduino code that's doing the serial port sending is:
char buffer [10] = { 0 };
int analogIn = 0;
int A0_val = 0;
void setup() {
Serial.begin(115200);
}
void loop() {
A0_val = analogRead(analogIn);
sprintf(buffer, "%d", A0_val);
Serial.println(buffer);
}
I suspect that the messing up of the data is caused by different size of the buffer used to transmit and receive data in the serial port. What is the good suggestion for the size of the buffer and even better method to guarantee the successful transmission of valid data?
Thanks very much!
Your reciever code cannot assume a single read from the serial port will yield a complete line (i.e. the 2 or 3 digits followed by a '\n' that the arduino continuously sends).
It is up to the receiver to synthetize complete lines of text on reception, and only then try to use them as meaningful numbers.
Since the serial interface is extremely slow compared with your average PC computing power, there is little point in reading more than one character at a time: literally millions of CPU cycles will be spent waiting for the next character, so you really don't need to react fast to the arduino input.
Since in that particular case it will not hinder performances in the slightest, I find it more convenient to read one character at a time. That will save you the hassle of moving bits of strings around. At least it makes writing an educational example easier.
// return the next value received from the arduino as an integer
int read_arduino (HANDLE hserial)
{
char buffer[4]; // any value longer than 3 digits must come
// from a faulty transmission
// the 4th caracter is used for a terminating '\0'
size_t buf_index = 0; // storage position of received characters
for (;;)
{
char c; // read one byte at a time
if (!ReadFile(
hSerial,
&c, // 1 byte buffer
1, // of length 1
NULL, // we will read exactly one byte or die trying,
// so length checking is pointless
NULL)){
/*
* This error means something is wrong with serial port config,
* and I assume your port configuration is hard-coded,
* so the code won't work unless you modify and recompile it.
* No point in keeping the progam running, then.
*/
fprintf (stderr, "Dang! Messed up the serial port config AGAIN!");
exit(-1);
}
else // our read succeded. That's a start.
{
if (c == '\n') // we're done receiving a complete value
{
int result; // the decoded value we might return
// check for buffer overflow
if (buf_index == sizeof (buffer))
{
// warn the user and discard the input
fprintf (stderr,
"Too many characters received, input flushed\n");
}
else // valid number of characters received
{
// add a string terminator to the buffer
buffer[buf_index] = '\0';
// convert to integer
result = atoi (buffer);
if (result == 0)
{
/*
* assuming 0 is not a legit value returned by the arduino, this means the
* string contained something else than digits. It could happen in case
* of electricval problems on the line, typically if you plug/unplug the cable
* while the arduino is sending (or Mr Fluffy is busy gnawing at it).
*/
fprintf (stderr, "Wrong value received: '%s'\n", buffer);
}
else // valid value decoded
{
// at last, return the coveted value
return res; // <-- this is the only exit point
}
}
// reset buffer index to prepare receiving the next line
buf_index = 0;
}
else // character other than '\n' received
{
// store it as long as our buffer does not overflow
if (buf_index < sizeof (buffer))
{
buffer[buf_index++] = c;
/*
* if, for some reason, we receive more than the expected max number of
* characters, the input will be discarded until the next '\n' allow us
* to re-synchronize.
*/
}
}
}
}
}
CAVEAT: this is just code off the top of my head. I might have left a few typos here and there, so don't expect it to run or even compile out of the box.
A couple of basic problems here. First, it is unlikely that the PC can reliably keep up with 115,200 baud data if you only read 10 bytes at a time with ReadFile. Try a slower baud rate and/or change the buffer size and number of bytes per read to something that will get around 20 milliseconds of data, or more.
Second, after you read some data put a nul at the end of it
szBuf[dwBytesRead] = 0;
before you pass it to printf or any other C string code.
I'm having a problem with unix local sockets. While reading a message that's longer than my temp buffer size, the request takes too long (maybe indefinitely).
Added after some tests:
there is still problem with freeze at ::recv. when I send (1023*8) bytes or less to the UNIX socket - all ok, but when sended more than (1023*9) - i get freeze on recv command.
maybe its FreeBSD default UNIX socket limit or C++ default socket settings? Who know?
i made some additational tests and I am 100% sure that its "freeze" on the last 9th itteration when executing ::recv command, when trying to read message >= (1023*9) bytes long. (first 8th itterationg going well.)
What I'm doing:
The idea is to read in a do/while loop from a socket with
::recv (current_socket, buf, 1024, 0);
and check buf for a SPECIAL SYMBOL. If not found:
merge content of buffer to stringxxx += buf;
bzero temp buf
continue the ::recv loop
How do I fix the issue with the request taking too long in the while loop?
Is there a better way to clear the buffer? Currently, it's:
char buf [1025];
bzero(buf, 1025);
But I know bzero is deprecated in the new c++ standard.
EDIT:
*"Why need to clean the buffer*
I see questions at comments with this question. Without buffer cleanup on the next(last) itteration of reading to the buffer, it will contain the "tail" of first part of the message.
Example:
// message at the socket is "AAAAAACDE"
char buf [6];
::recv (current_socket, buf, 6, 0); // read 6 symbols, buf = "AAAAAA"
// no cleanup, read the last part of the message with recv
::recv (current_socket, buf, 6, 0);
// read 6 symbols, but buffer contain only 3 not readed before symbols, therefore
// buf now contain "CDEAAA" (not correct, we waiting for CDE only)
When your recv() enters an infinite loop, this probably means that it's not making any progress whatsoever on the iterations (i.e., you're always getting a short read of zero size immediately, so your loop never exits, because you're not getting any data). For stream sockets, a recv() of zero size means that the remote end has disconnected (it's something like read()ing from a file when the input is positioned at EOF also gets you zero bytes), or at least that it has shut down the sending channel (that's for TCP specifically).
Check whether your PHP script is actually sending the amount of data you claim it sends.
To add a small (non-sensical) example for properly using recv() in a loop:
char buf[1024];
std::string data;
while( data.size() < 10000 ) { // what you wish to receive
::ssize_t rcvd = ::recv(fd, buf, sizeof(buf), 0);
if( rcvd < 0 ) {
std::cout << "Failed to receive\n"; // Receive failed - something broke, see errno.
std::abort();
} else if( !rcvd ) {
break; // No data to receive, remote end closed connection, so quit.
} else {
data.append(buf, rcvd); // Received into buffer, attach to data buffer.
}
}
if( data.size() < 10000 ) {
std::cout << "Short receive, sender broken\n";
std::abort();
}
// Do something with the buffer data.
Instead of bzero, you can just use
memset(buf, 0, 1025);
These are 2 separate issues. The long time is probably some infinite loop due to a bug in your code and has nothing to do with the way you clear your buffer. As a matter of fact you shouldn't need to clear the buffer; receive returns the number of bytes read, so you can scan the buffer for your SPECIAL_SYMBOL up to that point.
If you paste the code maybe I can help. more.
Just to clarify: bzero is not deprecated in C++ 11. Rather, it's never been part of any C or C++ standard. C started out with memset 20+ years ago. For C++, you might consider using std::fill_n instead (or just using std::vector, which can zero-fill automatically). Then again, I'm not sure there's a good reason to zero-fill the buffer in this case at all.
I am using read function to read data from a socket, but when the data is more than 4k, read function just read part of the data, for example, less than 4k. Here is the key code:
mSockFD = socket(AF_INET, SOCK_STREAM, 0);
if (connect(mSockFD, (const sockaddr*)(&mSockAdd), sizeof(mSockAdd)) < 0)
{
cerr << "Error connecting in Crawl" << endl;
perror("");
return false;
}
n = write(mSockFD, httpReq.c_str(), httpReq.length());
bzero(mBuffer, BUFSIZE);
n = read(mSockFD, mBuffer, BUFSIZE);
Note than BUFSIZE is much larger than 4k.
When data is just a few hundred bytes, read function works as expected.
This is by design and to be expected.
The short answer to your question is you should continue calling "read" until you get all the data you expect. That is:
int total_bytes = 0;
int expected = BUFSIZE;
int bytes_read;
char *buffer = malloc(BUFSIZE+1); // +1 for null at the end
while (total_bytes < expected)
{
int bytes_read = read(mSockFD, buffer+total_bytes, BUFSIZE-total_bytes);
if (bytes_read <= 0)
break;
total_bytes += bytes_read;
}
buffer[total_bytes] = 0; // null terminate - good for debugging as a string
From my experience, one of the biggest misconceptions (resulting in bugs) that you'll receive as much data as you ask for. I've seen shipping code in real products written with the expectation that sockets work this way (and no one certain as to why it doesn't work reliably).
When the other side sends N bytes, you might get lucky and receive it all at once. But you should plan for receiving N bytes spread out across multiple recv calls. With the exception of a real network error, you'll eventually get all N bytes. Segmentation, fragmentation, TCP window size, MTU, and the socket layer's data chunking scheme are the reasons for all of this. When partial data is received, the TCP layer doesn't know about how much more is yet to come. It just passes what it has up to the app. It's up to the app to decide if it got enough.
Likewise, "send" calls can get conglomerated into the same packet together.
There may be ioctls and such that will make a socket block until all the expected data is received. But I don't know of any off hand.
Also, don't use read and write for sockets. Use recv and send.
Read this book. It will change your life with regards to sockets and TCP: