Parsing NMEA sentences from serial - c++

I want to use TinyGPS++ on an Arduino to parse NMEA data and display information on an OLED display. But, instead of using software serial and the TX/RX pins, the NMEA data will be received by USB.
I followed the examples from TinyGPS++, but i encountered two problems:
1)
Only the first 64 characters are received by the Arduino, when i send one NMEA sentence over the serial monitor (Windows, Arduino 1.6.9). How can I overcome this restriction? I help myself by deleting a couple of decimal places, but this is not the preferred way to go.
2)
In the TinyGPS++ BasicExample a sample NMEA string is defined in the read-only memory:
// A sample NMEA stream.
const char *gpsStream =
"$GPRMC,045103.0,A,3014.0,N,09748.0,W,36.88,65.02,030913,,,A*7C\r\n"
"$GPGGA,045104.0,3014.0,N,09749.0,W,1,09,1.2,211.6,M,-22.5,M,,*62\r\n"
"$GPRMC,045200.0,A,3014.0,N,09748.0,W,36.88,65.02,030913,,,A*77\r\n"
"$GPGGA,045201.0,3014.0,N,09749.0,W,1,09,1.2,211.6,M,-22.5,M,,*6C\r\n"
"$GPRMC,045251.0,A,3014.0,N,09748.0,W,36.88,65.02,030913,,,A*7D\r\n"
"$GPGGA,045252.0,3014.0,N,09749.0,W,1,09,1.2,211.6,M,-22.5,M,,*6F\r\n";
and parsed by
while (*gpsStream) {
Serial.print(*gpsStream);
gps.encode(*gpsStream++);
}
I receive my NMEA (unfortunately only one line) this way:
if (Serial.available()) {
while (Serial.available() > 0) {
if(index < 80)
{
inChar = Serial.read();
inData[index] = inChar;
index++;
inData[index] = '\0';
}
}
}
and try to parse it by:
index = 0;
while (index < 80) {
gps.encode(inData[index]);
Serial.print(inData[index]);
index++;
}
But this does not work as desired. Checking if the location isValid() always returns not to be true.
Unfortunately, i have several possible sources for this undesired behavior.
The too short sentences (unlikely)
Incorrect way of reading the data over serial.
I only submit one line.
Something else.
I am not that experienced with neither NMEA, nor the serial data communication, and i have only little experience with Arduino/C. Can you point me into a direction how to solve for this (these) problems?

Basically, you do not need to accumulate NMEA characters. Just feed them to the GPS library as you receive them. You don't provide the entire loop, but it is very common to have a problem there, too.
After struggling with several GPS libraries and their examples, I eventually wrote NeoGPS. It is faster and smaller than all other libraries, it validates the checksum, and the examples are structured correctly. Unlike other libraries, NeoGPS does not store GPS values as floating-point values, so it is able to retain the full accuracy of your GPS device.
If you'd like to try it, be sure to follow the Installation instructions. The NMEA.ino example will emit one line of info (CSV format) for each batch of GPS sentences that you send, ending with the default RMC sentence. Be sure to modify it to use the Serial object instead of gps_port, or simply define it that way:
#define gps_port Serial
It will also show the number of characters that have been parsed, how many good sentences have been received, and how many sentences had checksum errors. That could help with debugging if you are not generating the checksum correctly. This site is useful, too.
Those CSV lines will be sent back over the USB port (to the PC), but you can easily change it to send specific fields to the OLED (see NMEAloc.ino).
Although it is possible to develop something on a PC and then port it to an embedded environment like the Arduino, you have to be careful about (1) linear program structure and (2) ignoring resource limits (program size, MCU speed and RAM). There are a number of quirks with the Arduino environment that usually make it frustrating to port a "sketch" to/from a PC. :P

Related

Converting String for Delphi to std::string for C++

I am developing a device (ESP32) and there is a rxValue variable that defined as below;
std::string rxValue = pCharacteristic->getValue(); <- ESP32 (C++) side
But I need to send data (string or character) from Delphi App via SetValueAs... method but when I use
Characteristics.SetValueAsString('b'); <- Delphi side (but problematic)
EDIT: I'm adding SetValueAsString procedure here
procedure TBluetoothGattCharacteristic.SetValueAsString(const AValue: string; IsUTF8: Boolean);
begin
if IsUTF8 then
Value := TEncoding.UTF8.GetBytes(AValue)
else
Value := TEncoding.Unicode.GetBytes(AValue);
end;
Arduino serial monitor shows me a strange character (square) and command cannot be executed.
for (int i = 0; i < rxValue.length(); i++) { <- ESP32 (C++) side
Serial.print(rxValue[i]);
}
if (rxValue.find("a") != -1) {
digitalWrite(LED, HIGH);
}
As a result, what is the way of sending a string from Delphi to C++ (ESP32 board) as std::string. In addition (I didn't want to ask a little question separately.) What is the way of exact comparing rxValue with string? I mean if rxValue is exact same as abcdef execute a command?
Thanks right now..
The problem has become more complex after diving into deep. The main problem was 8-Bit / 16-Bit string incompatibility as #RemyLebeau pointed out and #David Heffernan was correct because you cannot use be used std::string for interop across programming languages.
So I've decided to convert my data to 8Bit AnsiString and send but this time I saw that Delphi doesn't support AnsiChar and Ansistring in mobile compilers anymore (you can read here and here).
So I cannot use 8-Bit strings in Delphi mobile (there are many people they ask "why" to Embarcadero because this means that you cannot use devices that use 8-Bit (many of IoT devices) ) but thank goodness, there are some good people solve this problem with a custom library (you can find the source here).
After adding this library to my app and execute the command like below, problem has been solved;
Characteristics.SetValueAsString(RawByteString('b'));
BUT answer is: You can't use std::string this way, it's not suitable for interop.

C++ Winsock Download File Cut off HTTP Header

I'm downloading the bytes of a file from the web using winsock2. so good so far.
I have the problem that I download my bytes including the http header which I don't need and which causes troubles in my files bytecodes.
Example:
I know I can find the position where the header is ending by finding "\r\n\r\n".
But somehow I can't find or at least cut it... :(
int iResponseBytes = 0;
ofstream ofDownloadedFile;
ofDownloadedFile.open(pathonclient, ios::binary);
do {
iResponseBytes = recv(this->Socket, responseBuffer, pageBufferSize, 0);
if (iResponseBytes > 0) // if bytes received
{
ofDownloadedFile.write(responseBuffer, pageBufferSize);
}
else if (iResponseBytes == 0) //Done
{
break;
}
else //fail
{
cout << "Error while downloading" << endl;
break;
}
} while (iResponseBytes > 0);
I tried searching the array / the pointer using strncmp etc.
Hopefully someone can help me.
Best greetings
You have no guarantees, whatsoever, that the \r\n\r\n sequence will be received completely within a single recv() call.
For example, the first recv() call could end up reading everything up until the first two characters of the sequence, \r\n, then your code runs around the loop again, and the second time recv() gets called it receives the remaining \r\n for the initial two bytes received (followed by the first part of the actual content). A small possibility that this might happen, but it cannot be ignored, and must be correctly handled.
If your goal is to trim everything up until the \r\n\r\n, your current approach is not going to work very well.
Instead, what you should do is invest some time studying how file stream buffering actually works. Pontificate, for a moment, how std::istream/std::ostream read/write large chunks of data at a time, but they provide a character-oriented interface. std::istream, for example, reads a buffer's full of file data at a time, placing it into an internal buffer, which your code can then retrieve one character at a time (if it wishes to). How does that work? Think about it.
To do this correctly, you need to implement the same algorithm yourself: recv() from the socket a buffer at a time, then provide a byte-oriented interface, to return the received contents one byte at a time.
Then, the main code becomes a simple loop, reading the streamed socket contents one byte at a time, at which point discarding everything up until the code sees \r\n\r\n becomes trivial (although there are still a few non-obvious gotchas in doing this right, but that can be a new question).
Of course, once the \r\n\r\n gets processed, it is certainly possible to optimize things going forward, by flushing out whatever's still buffered internally, to the output file, and then resume reading from the socket a whole buffer-at-a-time, and copying it to the output file without burning CPU cycles dealing with the byte-oriented interface.

Sending numeric Array via UDP using winsock vc++

RESOLVED: Problem was primarily with the simulink blockset that was reading in the UDP packet rather than data transmission.
I am trying to send a 20 byte numerical array out of a c++ program by using winsock. The issue I am running into is data packaging, in the end I want each number in the array to go out as its own byte so that my simulink model that is receiving these values does not need an additional processing script.
My array contains 14 boolean values (0|1) and then 6 values that range from -100 to 100. These are reporting the status of a controller input. for example the array would look like
int array msgint[20] = [1,0,1,0,0,1,1,0,0,0,0,0,0,0,50,80,-90,40,90,-20];
I have tried using typecasting and sending multiple strings but all just appear to rearrange the gibberish I am getting or cause a socket error. Currently my sendto function looks like
sendto(sd,message,80,0,(struct sockadd *) &server,server_length)
I know this line works as the packet makes it through it just does not appear as I would like it to. In the send to, message is the formatted string I am trying to create to properly send all of contents of the array. Currently is is arbitrary and has little significance I have it in for debugging purposes essentially.
you are starting at the wrong point. Network communications should start with the design of the wire protocol.
How will you represent something on the wire. Binary or text. Most 'modern' protocols use text (json or xml). A few years ago binary was hot (asn1/ber/der). I suggest json
Then how will you wrap up the payload. Do you need to say 'here is a set of xxxs. now here is a set of yyyys'. I dont know what you are doing so its hard to say what you need
If you want to send 20 bytes, and you know that every integer value in your array will be in the [-100, +100] range, you should not use the int type, which usually contains either 32-bit or 64-bit values on modern platforms.
You might instead want to use the char type, which usually represents a 8-bit value.
For even more certainty, if you can use C++11 features, you should use the <cstdint> header, which defines the int8_t type, guaranteed to be a signed 8-bit type. See http://en.cppreference.com/w/cpp/types/integer.
Your array should look like:
#include <cstdint>
std::int8_t msgint[20] = {1,0,1,0,0,1,1,0,0,0,0,0,0,0,50,80,-90,40,90,-20};
or
char msgint[20] = {1,0,1,0,0,1,1,0,0,0,0,0,0,0,50,80,-90,40,90,-20};
and your sendto will be:
sendto(sd,msgint,20,0,(struct sockadd *) &server,server_length);
"in the end I want each number in the array to go out as its own byte"
Then change your array to
char msgint[20] = ...

Send Hex data to a serial port?

I have a GPS connected to USB0 and I can read everything perfectly but now I am trying to write data to it. It works with sending and receiving Hexadecimal data. I tried to send data to the GPS. Let's say I want to send 0xB6 0x62 to the port how could I do it in C++. I used this but I don't know weatehr I am right or wrong could someone help me
int main()
{
unsigned char bytestosend[2] = {0xB5, 0x62};
write(fd,&bytestosend,2);
}
of course I open the port to the file descriptor fd.
basically, what you're doing is right given your GPS reads actually hexadecimal bytes. But it may as well read string encoded hexadecimal values, so be sure of that. So from what you tell, I'd say you're writing to it correctly.
And you don't say how you open the serial port, but be careful to use termios and set the connection up correctly, or you may get issues. You shall not just open the port like a standard file. If you haven't done it yet, have a good read of this!

PWM Audio with reading data from SD card

Currently I'm working on this Arduino/Nanode project where we want to play a collection of WAV-files stored on a SD-card, with PWM on clock OCR0.
- I'm able to play the PWM perfectly, starting from the sketch from Michael Smith on the Arduino website: http://www.arduino.cc/playground/Code/PCMAudio
- I'm able to read the SD-card correctly and convert the data to 8bit integers that look correct when I print them to the serial window.
The problem I have is when I feed in these integers to the PWM value of the clock.
As I said, when I'm using the original PWM Audio file with my own WAV-file converted to a .h-file (through wav2c) it works and it sounds good. When I'm reading the SD-card it shows me the correct values. It shows correct when I'm reading the WAV-files direct and also (what I'm trying in my latest version posted here) as I convert them to text-files and read these. When I'm feeding in the integers from the text file, I hear a horn-like sound, like if the PWM uses the wrong values to output.
I'm guessing the problem is somewhere in the casting of the data into the byte data the Atmega uses. But I don't have any clue where to look or how to solve it. I noticed that the original file uses unsigned char's where I'm using uint_t8. I tried to cast them but it's not working.
Does anyone has some experience in this? Or any clue how I could possible solve this?
Many thanks for you help and time!
Jeroen
PS: Below is the piece of my code where I read through the text files and convert them to integers. They always consist of 3 characters; value 21 for example is printed as 021 in the file, and seperated with a comma which the script skips with the 4th myFile.read()
myFile = SD.open(FileName);
char sampleTMP[4];
sampleTMP[0] = (myFile.read());
sampleTMP[1] = (myFile.read());
sampleTMP[2] = (myFile.read());
sampleTMP[3] = 0;
myFile.read();
unsigned char ss;
ss = atoi(sampleTMP);
Serial.println(ss, DEC);
OCR0A = ss;
OCR0B = ss;