Arduino does not support messages larger than 65535 characters using websocket protocol? - c++

I'm using the following Arduino websocket library, I had a problem when trying to send messages over than 65535 characters, I got handshake fail error.
As long as the message doesn't exceeds this length, it worked perfectly
There's a note on the main web page of the library that states:
Because of limitations of the current Arduino platform (Uno at the time of this writing),
this library does not support messages larger than 65535 characters.
In addition, this library only supports single-frame text frames.
It currently does not recognize continuation frames, binary frames, or ping/pong frames.
In the client header file named WebSocketClient.h there's the following comment:
// Don't allow the client to send big frames of data. This will flood the arduino memory and might even crash it.
#ifndef MAX_FRAME_LENGTH
#define MAX_FRAME_LENGTH 256
#endif
I'm using this old library because it is the only one worked for me on my Arduino WIFI shield, I couldn't find other libraries that support WiFi shield since most of the webscket libraries are written for Arduino Eathernet Shield support, which I don't have.
My Arduino Code is
/*DS18 Libs*/
#include <dht.h>
#include <OneWire.h>
#include <DallasTemperature.h>
/*Websocket Libs*/
#include <WebSocketServer.h>
#include <WebSocketClient.h>
#include <sha1.h>
#include <MD5.h>
#include <global.h>
#include <Base64.h>
#include <SPI.h>
#include <WiFiUdp.h>
#include <WiFiServer.h>
#include <WiFiClient.h>
#include <WiFi.h>
#include <string.h>
char ssid[] = "AMM";
char pass[] = "027274792";
int status = WL_IDLE_STATUS;
IPAddress server(192, 168, 1, 3);
WiFiClient WiFiclient;
WebSocketClient WSclient;
// Data wire is plugged into port 2 on the Arduino
#define ONE_WIRE_BUS 2
// Setup a oneWire instance to communicate with any OneWire devices (not just Maxim/Dallas temperature ICs)
OneWire oneWire(ONE_WIRE_BUS);
// Pass our oneWire reference to Dallas Temperature.
DallasTemperature sensors(&oneWire);
//Humidture
dht DHT;
#define DHT11_PIN 4
void setup()
{
Serial.begin(9600);
while (!Serial) {
; // wait for serial port to connect. Needed for Leonardo only
}
//check for the presence of the shield:
if (WiFi.status() == WL_NO_SHIELD) {
Serial.println("WiFi shield not present");
// don't continue:
while (true);
}
// attempt to connect to Wifi network:
while (status != WL_CONNECTED) {
Serial.print("Attempting to connect to WPA SSID: ");
Serial.println(ssid);
// Connect to WPA/WPA2 network:
status = WiFi.begin(ssid, pass);
}
// you're connected now, so print out the data:
Serial.print("You're connected to the network");
/* Connect to the websocket server*/
if (WiFiclient.connect(server, 8080)) {
Serial.println("Connected");
}
else {
Serial.println("Connection failed.");
while (1) {
// Hang on failure
}
}
// Handshake with the server
WSclient.path = "/MyServer/endpoint/testtest/device/d6220ae7-caa9-48b5-92db-630c4c296ec4";
WSclient.host = "192.168.1.3:8080";
if (WSclient.handshake(WiFiclient)) {
Serial.println("Handshake successful");
}
else {
Serial.println("Handshake failed.");
while (1) {
// Hang on failure
}
}
/*DS18*/
sensors.begin();
}
void loop()
{
WSclient.sendData("{\"service_code\":\"89c4da72-a561-47db-bf62-8e63f8c4bbf0\",\"data\":[" + getHumidtureValue() + "],\"service_type\":\"TemperatureHumidityAnalysis\"}");
WSclient.sendData("{\"service_code\":\"bdc0f984-6550-4712-881f-b09071da5a73\",\"data\":" + getCBodyTempretureValue() + ",\"service_type\":\"TemperatureGaugeMonitor\"}");
//line-3 commented WSclient.sendData("{\"service_code\":\"8c212432-a86e-4c18-a956-9dc0dbb648d4\",\"data\":[" + getHumidtureValue() + "],\"service_type\":\"HumidityGaugeMonitor\"}");
}
String getCBodyTempretureValue()
{
sensors.requestTemperatures(); // Send the command to get temperatures
char charVal[10];
return dtostrf(sensors.getTempCByIndex(0), 4, 2, charVal);
}
String getHumidtureValue()
{
String str = "";
for (int i = 0; i < 2; i++)
{
int chk = DHT.read11(DHT11_PIN);
switch (chk)
{
case DHTLIB_OK:
Serial.println("OK,\t");
break;
case DHTLIB_ERROR_CHECKSUM:
Serial.println("Checksum error,\t");
break;
case DHTLIB_ERROR_TIMEOUT:
Serial.println("Time out error,\t");
break;
default:
Serial.println("Unknown error,\t");
break;
}
char charVal[10];
double tempF = (DHT.temperature * 9) / 5 + 32;
str = dtostrf(tempF, 3, 1, charVal);
str = str + "," + dtostrf(DHT.humidity, 3, 1, charVal);
Serial.println(str);
delay(200);
}
return str;
}
The code above works perfectly, when I uncomment the third send statement in the loop function, I got the handshake failed error.
-Is it safe to modify the value of MAX_FRAME_LENGTH for the new versions of Arduino board, considering this library is an old one?
-Is there any other libraries better than this one that can support websocket on WiFi shield?
Any solution or idea will appreciated.
Thanks in advance.

Without having looked at the code of the library it is likely not safe to change the max frame length, because the websocket protocol encodes the payload length differently depending on how long it is:
Payload length: 7 bits, 7+16 bits, or 7+64 bits.
The length of the "Payload data", in bytes: if 0-125, that is the payload length. If 126, the following 2 bytes interpreted as a 16-bit unsigned integer are the payload length. If 127, the following 8 bytes interpreted as a 64-bit unsigned integer (the most significant bit MUST be 0) are the payload length.
When the library says it doesn't support payload length above 65535 byte, it likely means that it has no implementation for the 64-bit length encoding.

After many trials and many times the program behaves very strangely, which drives me crazy, I found the problem is that I'm using too much strings in my program which makes the Arduino-Uno easily runs out of RAM.
The main reason I got handshake failed error is that the Arduino cannot read the "Sec-WebSocket-Accept" header of the handshake response message (as many other headers also) which I made sure that they are sent, by debugging code on the server.
Actually this problem and many other strange behaviors keep happening until I reduce the amount of the memory used during program run.

Related

No Matching Function Begin Multicast - Arduino/cpp

I was given a project to convert from ESP8266 to ESP32, and I'm definitely not a coder, but it needs to get done. So I was hoping I could get some guidance/help on how to solve this issue. I'm basically taking previous code from an old project using an ESP8266 WiFi module and converting it over to a more updated module using ESP32. Been compiling and troubleshooting code as I go the best I can, but this one has stumped me.
The error message I'm getting is "No matching function for call to 'WiFiUDP::beginMulticast(IPAddress, IPAddress&, const unsigned int&)'
There are several pages of code, but this is the .cpp file that is currently throwing up an error, and hoping it's able to be resolved with something on this page. Like I said, I'm not a coder, but I have the basics down. Any help on this would be greatly appreciated. Thanks!
#include "Switch.h"
#include <functional>
// Multicast declarations
IPAddress ipMulti(239, 255, 255, 250);
const unsigned int portMulti = 1900;
char packetBuffer[512];
#define MAX_SWITCHES 14
Switch switches[MAX_SWITCHES] = {};
int numOfSwitchs = 0;
//#define numOfSwitchs (sizeof(switches)/sizeof(Switch)) //array size
//<<constructor>>
UpnpBroadcastResponder::UpnpBroadcastResponder(){
}
//<<destructor>>
UpnpBroadcastResponder::~UpnpBroadcastResponder(){/*nothing to destruct*/}
bool UpnpBroadcastResponder::beginUdpMulticast(){
boolean state = false;
Serial.println("Begin multicast ..");
if(UDP.beginMulticast(WiFi.localIP(), ipMulti, portMulti)) {
Serial.print("Udp multicast server started at ");
Serial.print(ipMulti);
Serial.print(":");
Serial.println(portMulti);
state = true;
}
else{
Serial.println("Connection failed");
}
return state;
}
//Switch *ptrArray;
void UpnpBroadcastResponder::addDevice(Switch& device) {
Serial.print("Adding switch : ");
Serial.print(device.getAlexaInvokeName());
Serial.print(" index : ");
Serial.println(numOfSwitchs);
switches[numOfSwitchs] = device;
numOfSwitchs++;
}
void UpnpBroadcastResponder::serverLoop(){
int packetSize = UDP.parsePacket();
if (packetSize <= 0)
return;
IPAddress senderIP = UDP.remoteIP();
unsigned int senderPort = UDP.remotePort();
// read the packet into the buffer
UDP.read(packetBuffer, packetSize);
// check if this is a M-SEARCH for WeMo device
String request = String((char *)packetBuffer);
if(request.indexOf('M-SEARCH') > 0) {
if(request.indexOf("urn:Belkin:device:**") > 0) {
Serial.println("Got UDP Belkin Request..");
// int arrSize = sizeof(switchs) / sizeof(Switch);
for(int n = 0; n < numOfSwitchs; n++) {
Switch &sw = switches[n];
if (&sw != NULL) {
sw.respondToSearch(senderIP, senderPort);
}
}
}
}
}```
The Arduino UDP API as defined by the UDP base class in Arduino core has
uint8_t beginMulticast(IPAddress, uint16_t);
In esp8266 Arduino WiFi the UDP is modified and the first parameter of beginMulticast specifies the network interface to listen to. (Network interfaces are STA, SoftAP, Ethernet etc)
In esp32 Arduino WiFi library beginMulticast has only the standard parameters and listens on all network interfaces.
Your solution is to remove the first parameter in the beginMulticast call.

Nucleo F401RE - can’t receive a data from rx pin (response from ESP32 WiFi module)

I have a problem with the UART communication from Nucleo f401re to ESP-Wroom-32 (WiFi BLE Click wifi module mikroe). I’m able to send an AT command but not to receive. I’m using the latest mbed-os verion, also I’m sending the at command via the ATCmdParser Api that it’s included on the “mbed.h” file. I had tried to receive with the the recv() function but it returns False. I tried also with the BufferedSerial.h with the read() but nothinguuu. I’m positive that i’m sending the command because i tried to receive the response from the esp32 module with the samd21 dev board and i’m receiving it fine.
Here is the code:
#include “mbed.h”
#include
#define ESP32_DEFAULT_BAUD_RATE 115200
BufferedSerial _serial;
ATCmdParser _parser;
/////~ set the debug parameter as a true of the ATCmdParser ~////
using namespace std;
char bufRead[100], buffWrite[50];
int main()
{
_serial = new BufferedSerial(D8, D2, ESP32_DEFAULT_BAUD_RATE);
_serial->set_format(
/* bits / 8,
/ parity / BufferedSerial::None,
/ stop bit */ 1
);
_parser = new ATCmdParser(_serial);
_parser->debug_on( 1 );
_parser->set_delimiter( “\r\n” );
bool result = false;
while(1){
_parser->send("AT");
int x = _serial->read(bufRead, 40);
printf("Response %s\n", bufRead);
printf("-> %d\n", x);
thread_sleep_for(2000);
}
}

Strange characters after JSON

I have an application that gets the Arduino Nano information and sends it to ESP-01 via UART. The ESP-01 send this to MQTT.
NANO CODE:
#include "DHTesp.h"
#include <ArduinoJson.h>
#include <SoftwareSerial.h>
#define gasSensor A1
#define dhtPin 5
#define rain A2
#define soil A3
#define ldr A4
DHTesp dht;
void setup() {
Serial.begin(115200);
pinMode(gasSensor, INPUT);
pinMode(rain, INPUT);
pinMode(soil, INPUT);
pinMode(ldr, INPUT);
digitalWrite(dhtPin, LOW);
dht.setup(dhtPin, DHTesp::DHT11);
}
void loop() {
delay(dht.getMinimumSamplingPeriod());
float humidity = dht.getHumidity();
float temperature = dht.getTemperature();
DynamicJsonBuffer jBuffer;
JsonObject& measure = jBuffer.createObject();
JsonObject& data = jBuffer.createObject();
measure["gas"] = analogRead(gasSensor);
measure["humidity"] = humidity;
measure["temperature"] = temperature;
measure["heatindex"] = dht.computeHeatIndex(temperature, humidity, false);
measure["rain"] = analogRead(rain);
measure["soil"] = analogRead(soil);
measure["ldr"] = analogRead(ldr);
data["measure"] = measure;
data.printTo(Serial);
}
ESP-01 CODE:
#include <Arduino.h>
#include <ESP8266WiFi.h>
#include <PubSubClient.h>
const char* mqtt_server = "0.0.0.0";
WiFiClient espClient;
PubSubClient client(espClient);
char mystr[100];
void setup() {
Serial.begin(115200);
WiFi.begin("", "");
Serial.print("Connecting");
while (WiFi.status() != WL_CONNECTED) {
delay(1000);
Serial.print(".");
}
Serial.println();
Serial.print("Connected to IP: ");
Serial.println(WiFi.localIP());
client.setServer(mqtt_server, 1883);
}
void loop() {
if (client.connect("ESP")) {
Serial.println("STATUS MQTT-ESP: OK");
while (true) {
Serial.readBytes(mystr, 108);
client.publish("esp", mystr);
delay(1000);
}
} else {
Serial.println("STATUS MQTT-ESP: OFF");
}
}
But, I'm getting strange characters into MQTT:
And, at the connection between NANO <-uart-> Computer, the JSON is normal:
Someone could help me?
Thanks.
I am basing this answer on a bit of assumptions as I don't have an Arduino at hand.
You send your data like this:
data.printTo(Serial);
This sends the string holding the formatted JSON data. This does not include the terminating 0 byte.
Then you receive it like this:
char mystr[100];
...
while(true){
Serial.readBytes(mystr, 108);
client.publish("esp", mystr);
delay(1000);
}
This has multiple errors:
You do not care if you got any bytes at all. The method returns the number of bytes but you do not handle the return value at all. If you got 0 bytes within the timeout value, you will just send the previous message again instead of waiting for valid data.
You cannot put 108 bytes into memory location of 100 bytes.
You put an array of char into the publish method that does not contain any termination. How should that method know how many characters are really part of the string?
Try this:
while(true){
size_t num = Serial.readBytes(mystr, sizeof(mystr)-1);
mystr[num] = 0;
client.publish("esp", mystr);
delay(1000);
}
You do not have any protocol that allows you to detect when a message starts or ends. UART communiation is a byte stream without any boundaries for datagrams. You must ensure that you know what belongs to a message and what does not. This also means that you can never know if you have a complete message in your receive bufffer or an incomplete or even more than one message. It's up to you to detect and split messages. Fixing this is a bit more complex. I can only give some hints what you need to do.
a) Detect message boundaries:
You might send some terminator like \n, 0 or similar after each JSON string and scan for this extra byte on receiver side.
Or you can send the length of the string before you send the string.
You could also just check on receiver side when you have a matching pair of {}. This would not require any change on sender side.
b) Collect messages:
Call your read function as long as it takes until you detect the end of a string.
You might need to use intermediate buffer to collect multiple read buffers.
c) Forward messages:
As soon as you detected the end of a message, forward it via publish function.
Then move the remaining content of your collection buffer to the start of that buffer.
If you immediately find the end of another message, repeat that step.
If you don't find any complete message, continue to collect more data until the next message is complete.

How to store response from html request in char array on arduino?

I have an Arudino Uno with an Adafruit CC3000 wifi shield attached.
I am trying to send multiple http requests and store the results from the GET requests. I can make and receive the requests successfully, but space on the arduino (in the buffer?) runs out when I try to store more than one of the responses.
I'm happy to store one string at a time so I figured that instead of using the arduino String class if I use a char array instead and allocate memory, I can then free the memory afterwards. That way I could use the memory as required, hopefully not cause any issues in running the rest of the code. I know this also depends on how long the incoming response is, but let's assume the response size is small enough. Feel free to shoot me down if there are flaws in my logic... (likely)
I tried variations of creating the char array without having to define the size beforehand and then using strcpy or strcat to append the new characters, but with no success.
I want to do something in the process of: create char array, fill it, use it, free it from memory.
In the past I've used this method in such a form:
char *array = new char[size_wanted];
strcpy(array,some_char_array);
strcat(array,some_other_char_array);
This of course works well when you know what size_wanted is. I don't until I read the buffer, but once I've read the buffer I've read it, so cannot read it again. Am I missing a trick here?! Is there a simpler way to do this using the Arduino String class? Am I missing the obvious or just not understanding how this works? Any ideas would be greatly appreciated.
My code:
/***************************************************
Adafruit CC3000 Wifi Breakout & Shield Example
****************************************************/
#include <Adafruit_CC3000.h>
#include <ccspi.h>
#include <SPI.h>
#include <string.h>
#include "utility/debug.h"
// These are the interrupt and control pins
#define ADAFRUIT_CC3000_IRQ 2 // MUST be an interrupt pin!
// These can be any two pins
#define ADAFRUIT_CC3000_VBAT 5
#define ADAFRUIT_CC3000_CS 10
// Use hardware SPI for the remaining pins
// On an UNO, SCK = 13, MISO = 12, and MOSI = 11
Adafruit_CC3000 cc3000 = Adafruit_CC3000(ADAFRUIT_CC3000_CS, ADAFRUIT_CC3000_IRQ, ADAFRUIT_CC3000_VBAT,
SPI_CLOCK_DIVIDER); // you can change this clock speed
#define WLAN_SSID "wifi"
#define WLAN_PASS "passoword"
#define WLAN_SECURITY WLAN_SEC_WPA2
#define IDLE_TIMEOUT_MS 3000
// What page to grab!
#define WEBSITE "www.adafruit.com"
#define WEBPAGE "/testwifi/index.html"
uint32_t ip;
int n = 1;
char* result;
void setup(void)
{
Serial.begin(115200);
Serial.println(F("Hello, CC3000!\n"));
Serial.print("Free RAM: "); Serial.println(getFreeRam(), DEC);
/* Initialise the module */
Serial.println(F("\nInitializing..."));
if (!cc3000.begin())
{
Serial.println(F("Couldn't begin()! Check your wiring?"));
while(1);
}
Serial.print(F("\nAttempting to connect to ")); Serial.println(WLAN_SSID);
if (!cc3000.connectToAP(WLAN_SSID, WLAN_PASS, WLAN_SECURITY)) {
Serial.println(F("Failed!"));
while(1);
}
Serial.println(F("Connected!"));
/* Wait for DHCP to complete */
Serial.println(F("Request DHCP"));
while (!cc3000.checkDHCP())
{
delay(100); // ToDo: Insert a DHCP timeout!
}
/* Display the IP address DNS, Gateway, etc. */
while (! displayConnectionDetails()) {
delay(1000);
}
ip = 0;
// Try looking up the website's IP address
Serial.print(WEBSITE); Serial.print(F(" -> "));
while (ip == 0) {
if (! cc3000.getHostByName(WEBSITE, &ip)) {
Serial.println(F("Couldn't resolve!"));
}
delay(500);
}
cc3000.printIPdotsRev(ip);
String r1, r2, r3, r4, r5;
r1 = connect_to_webclient();
r2 = connect_to_webclient();
r3 = connect_to_webclient();
r4 = connect_to_webclient();
r5 = connect_to_webclient();
/*
Serial.println("RESULTS:");
Serial.println("r1:"); Serial.println(r1);
Serial.println("r2:"); Serial.println(r2);
Serial.println("r3:"); Serial.println(r3);
Serial.println("r4:"); Serial.println(r4);
Serial.println("r5:"); Serial.println(r5);
*/
/* You need to make sure to clean up after yourself or the CC3000 can freak out */
/* the next time your try to connect ... */
Serial.println(F("\n\nDisconnecting"));
cc3000.disconnect();
}
void loop(void)
{
delay(1000);
}
bool displayConnectionDetails(void)
{
uint32_t ipAddress, netmask, gateway, dhcpserv, dnsserv;
if(!cc3000.getIPAddress(&ipAddress, &netmask, &gateway, &dhcpserv, &dnsserv))
{
Serial.println(F("Unable to retrieve the IP Address!\r\n"));
return false;
}
else
{
Serial.print(F("\nIP Addr: ")); cc3000.printIPdotsRev(ipAddress);
Serial.print(F("\nNetmask: ")); cc3000.printIPdotsRev(netmask);
Serial.print(F("\nGateway: ")); cc3000.printIPdotsRev(gateway);
Serial.print(F("\nDHCPsrv: ")); cc3000.printIPdotsRev(dhcpserv);
Serial.print(F("\nDNSserv: ")); cc3000.printIPdotsRev(dnsserv);
Serial.println();
return true;
}
}
String connect_to_webclient() {
/* Try connecting to the website.
Note: HTTP/1.1 protocol is used to keep the server from closing the connection before all data is read.
*/
Serial.print("\nConnection number: ");
Serial.println(n);
Adafruit_CC3000_Client www = cc3000.connectTCP(ip, 80);
if (www.connected()) {
Serial.println("Connected succeeded");
www.fastrprint(F("GET "));
www.fastrprint(WEBPAGE);
www.fastrprint(F(" HTTP/1.1\r\n"));
www.fastrprint(F("Host: ")); www.fastrprint(WEBSITE); www.fastrprint(F("\r\n"));
www.fastrprint(F("\r\n"));
www.println();
} else {
Serial.println(F("Connection failed"));
return;
}
Serial.println(F("-------------------------------------"));
/* Read data until either the connection is closed, or the idle timeout is reached. */
unsigned long lastRead = millis();
while (www.connected() && (millis() - lastRead < IDLE_TIMEOUT_MS)) {
while (www.available()) {
char c = www.read();
Serial.print(c);
//strcat(result, c);
lastRead = millis();
}
}
www.close();
Serial.println(F("-------------------------------------"));
n++;
return result;
}

Serial communication between pc and arduino via RS232 using c++

I am trying to communicate with my arduino duemilanove via an RS232 cord. I simply want to be able to send a byte (or char) to my arduino from a desktop application. The Arduino is plugging into USB COM5 on my computer. I have the RS232 plugged into COM1, and then I have pins 2 3 and 5 on the other end of the RS232 connected to arduino pins TX, RX, and GND, respectively.
I found a serial comm class for c++ at the following link:
http://playground.arduino.cc/Interfacing/CPPWindows
I have added the .h and .cpp files from the above example as Serial.h and Serial.cpp (i think the example uses SerialClass.h and SerialClass.cpp, I just changes the names).
On my arduino, I have the following code running:
// ARDUINO
char incomingByte = 0;
void setup() {
Serial.begin(9600);
}
void loop() {
// send data only when you receive data:
if (Serial.available() > 0) {
// read the incoming byte:
incomingByte = Serial.read();
// say what you got:
Serial.print("I received: ");
Serial.println(incomingByte, HEX);
}
}
And my c++ program is the following:
// C++
#include <iostream>
#include <Windows.h>
#include "Serial.h"
using namespace std;
int main(void)
{
Serial port("COM1");
char* msg = "Hello Arduino!";
int msgLen = strlen(msg);
bool writeSuccess = port.WriteData(msg, msgLen);
cout << "\n\n";
system("PAUSE");
}
When I use the Arduino's serial port viewer to see what is bring printed, I'm getting very strange values that don't match what I'm sending (as far as I can tell).
When I send "Hello Arduino!", the arduino prints the following:
I received: FFFFFFAB
I received: 3A
I received: 3A
I received: A
I received: FFFFFFFA
I received: FFFFFFEB
I received: 6D
I received: 37
I received: 15
I received: 2D
I received: 23
I received: 21
I received: FFFFFFBD
I received: 0
This does not appear to be the correct hex for "Hello Arduino!", but I have no idea why it's not correct. Does anyone have any clue what I'm doing wrong?
Arduino used TTL logic for Serial connection. It expects values at 0 and 5V. RS232 used a different voltage -V to +V. You may need a converter.
Ehm... No! pull up and pull down are not for this reason..
TTL = low: 0V, high: 5V
RS232 = low: +3:+15V, high: -3:-15V
Consequently.. You need a voltage converter (and inverter), like David Skogan correctly pointed out.
Examples:
Using discrete components (has automatic echo feature, i.e. on the PC you will see the data you send): http://project.irone.org/simple-rs232-to-ttl-level-converter.html or http://circuit-diagram.hqew.net/Simple-TTL$2dRS232-Level-Converter-Using-Transistor_2757.html
Common circuit with a MAX232 (or equivalent) and four capacitors
Instead of using a USB-RS232 converter use a USB-UART one, using for instance a FT232 or something like that. This does not need any interface
Or.. simply use the USB port on the Arduino, which already has a FT232 on it.
Personal comment: i'd avoid solution 1...