Arduino substring doesn't work - c++

I have a static method that searches (and returns) into String msg the value between a TAG
this is the code function:
static String genericCutterMessage(String TAG, String msg){
Serial.print("a-----");
Serial.println(msg);
Serial.print("b-----");
Serial.println(TAG);
if(msg.indexOf(TAG) >= 0){
Serial.print("msg ");
Serial.println(msg);
int startTx = msg.indexOf(TAG)+3;
int endTx = msg.indexOf(TAG,startTx)-2;
Serial.print("startTx ");
Serial.println(startTx);
Serial.print("endTx ");
Serial.println(endTx);
String newMsg = msg.substring(startTx,endTx);
Serial.print("d-----");
Serial.println(newMsg);
Serial.println("END");
Serial.println(newMsg.length());
return newMsg;
} else {
Serial.println("d-----TAG NOT FOUND");
return "";
}
}
and this is output
a-----[HS][TS]5132[/TS][TO]5000[/TO][/HS]
b-----HS
msg [HS][TS]5132[/TS][TO]5000[/TO][/HS]
startTx 4
endTx 30
d-----
END
0
fake -_-'....go on! <-- print out of genericCutterMessage
in that case I want return the string between HS tag, so my expected output is
[TS]5132[/TS][TO]5000[/TO]
but I don't know why I receive a void string.
to understand how substring works I just followed tutorial on official Arduino site
http://www.arduino.cc/en/Tutorial/StringSubstring
I'm not an expert in C++ and Arduino but this looks like a flushing or buffering problem, isn't it?
Any idea?

Your code is correct, this should not happen. Which forces you to consider the unexpected ways that this could possibly fail. There is really only one candidate mishap I can think of, your Arduino is running out of RAM. It has very little, the Uno only has 2 kilobytes for example. It doesn't take a lot of string munching to fill that up.
This is not reported in a smooth way. All I can do is point you to the relevant company page. Quoting:
If you run out of SRAM, your program may fail in unexpected ways; it will appear to upload successfully, but not run, or run strangely. To check if this is happening, you can try commenting out or shortening the strings or other data structures in your sketch (without changing the code). If it then runs successfully, you're probably running out of SRAM. There are a few things you can do to address this problem:
If your sketch talks to a program running on a (desktop/laptop) computer, you can try shifting data or calculations to the computer, reducing the load on the Arduino.
If you have lookup tables or other large arrays, use the smallest data type necessary to store the values you need; for example, an int takes up two bytes, while a byte uses only one (but can store a smaller range of values).
If you don't need to modify the strings or data while your sketch is running, you can store them in flash (program) memory instead of SRAM; to do this, use the PROGMEM keyword.
That's not very helpful in your specific case, you'll have to look at the rest of the program for candidates. Or upgrade your hardware, StackExchange has a dedicated site for Arduino enthusiasts, surely the best place to get advice.

Related

Parallel Writes to NFS-backed File

UPDATE: I had each node write to a separate file, and when the separate files were concatenated together the result was correct. I also updated the code to attempt a channel flush and file sync after each write of a single record, but there are still issues between nodes 0 and 1, now. If I make Node 0 sleep for a few seconds before it starts its iteration of the coforall loop, the records come out correct. If not, the last few hundred bytes of Node 0's records seem to be reliably overwritten with NULL bytes, up to the start of Node 1's records. The issues between Node 1 and Node 2, and Node 2 and Node 3, seem to not show up anymore.
Additionally, if I suppress either Node 0 or Node 1 from writing, I see the fully-formed records from the un-suppressed node written correctly to the file. In the case that Node 1 is suppressed, I see 9,997 100B records (or 999,700) correct bytes followed by NULL bytes in the file where Node 1's suppressed records would go. In the case that Node 0 is suppressed, I see exactly 999,700 NULL bytes in the file, after which Node 1's records begin.
Original Post:
I'm trying to troubleshoot an issue with parallel writes from different nodes to a shared NFS-backed file on disk. At the moment, I suspect that something is wrong with the way writes to the disk happen on the NFS server.
I'm working on adapting MPI+C code that uses pwrite to write to coordinated chunks of a file. If I try to have the equivalent locales in Chapel write to the file inside of a coforall loop, I end up with the bits of the file around the node boundaries messed up - usually the final few hundred bytes of each node's data are garbled. However, if I have just one locale iterate through the data on all locales and write it, the data comes out correctly. That is, I use the same data structures to calculate the offsets, but only Locale 0 seeks to that offset and performs the writes.
I've verified that the offsets into the file that each locale runs do not overlap, and I'm using a single channel per task, defined from within the on loc do block, so that tasks don't share a single channel.
Are there known issues with writing to a file from different locales? A lot of the documentation makes it seem like this is known to be safe, but an unsubstantiated guess seems to indicate that there are issues with caching of file contents; when examining the incorrect data, the bits that are incorrect seem to be the original data from the file in that location at the beginning of the program.
I've included the relevant routine below, in case you easily spot something I missed. To make this serial, I convert the coforall loc in Locales and on loc do block into a for j in 0..numLocales-1 loop, and replace here.id with j. Please let me know what else would help get to the bottom of this. Thanks!
proc write_share_of_data(data_filename: string, ref record_blocks) throws {
coforall loc in Locales {
on loc do {
var data_file: file = open(data_filename, iomode.cwr);
var data_writer = data_file.writer(kind=ionative, locking=false);
var line: [1..100] uint(8);
const indices = record_blocks[here.id].D;
var local_record_offset = + reduce record_blocks[0..here.id-1].D.size;
writeln("Loc ", here.id, ": record offset is ", local_record_offset);
var local_bytes_offset = terarec.terarec_width_disk * local_record_offset;
data_writer.seek(start=local_bytes_offset);
for i in indices {
var write_rec: terarec_t = record_blocks[here.id].records[i];
line[1..10] = write_rec.key;
line[11..98] = write_rec.value;
line[99] = 13; // \r
line[100] = 10; // \n
data_writer.write(line);
lines_written += 1;
}
data_file.fsync();
data_writer.close();
data_file.close();
}
}
return;
}
Adding an answer here that solved my particular problem, though it doesn't explain the behavior seen. I ended up changing the outer loop from coforall loc in Locales to for loc in Locales. This isn't too big of an issue since it's all writing to one file anyway - I doubt that multiple locales can actually make much headway in all attempting to write concurrently to a single file on an NFS server. As a result, the change still allows nodes to write the data they have locally to NFS, rather than forcing Node 0 to collect and then write the data on behalf of all locales. This amounts to only adding idle time to the write operation commensurate with the time it takes Locale 0 to start the remote task on other nodes when the previous node has finished writing, which for the application at hand is not a concern.
Have you tried specifying start/end in file.writer instead of using seek? Does that change anything? What about specifying the end offset for the channel.seek call? Does it matter if the file is created and has the appropriate size before you start?
Other than that, I wonder if this issue would appear for both NFS and Lustre. If it appears for both it might well be a Chapel bug. It sounds from your description that the C program was using this pattern, which points to it being a bug. But, have you run C code doing this on your setup? If it being a Chapel bug seems most likely after further investigation, we would appreciate a bug report issue with a reproducer.
I know that NFS does not always do what one would like, in terms of data consistency. It's my understanding that it has "close to open" semantics but it's unclear to me what that means in the context of opening a file and writing to a particular region within it, in parallel from different locales.
From Why NFS Sucks by Olaf Kirch:
An NFS client is permitted to cache changes locally and send them to
the server whenever it sees fit. This sort of lazy write-back greatly
helps write performance, but the flip side is that everyone else will
be blissfully unaware of these change before they hit the server. To
make things just a little harder, there is also no requirement for a
client to transmit its cached write in any particular fashion, so
dirty pages can (and often will be) written out in random order.
I read two implications from this paragraph that are relevant to your situation here:
The writes you do on different locales can be observed by the NFS server in an arbitrary order. (However as I understand it, the data should be sent to the server by the time your fsync call returns).
These writes are done at an OS page granularity (usually 4k). (Note that this is more a hypothesis I am making than it is a fact. It should be tested or further investigated).
It would be interesting to check if 2. is a plausible explanation for the behavior you are seeing. For example, you could explore having each locale operate on a multiple of 4096 records (or potentially try writing records of 4096 bytes each) and see if that changes the behavior. If 2 is indeed the explanation, it should be possible to create a C program that demonstrates the behavior as well.

Weird output for RGB

I am trying to manage some LED strips with my mobile device using bluetooth + Adafruit NeoPixel. I almost had the sketch finished but I have found the numbers for RGB are not appearing as I expected and I cannot find what I am doing wrong.
Imagine that from my mobile I have sent the following RGB code "0,19,255", when I check the console I see the following result:
As you can see the first two lines are ok, but the third one we can see 2550. The 0 should no be there and I cannot figure it out the problem.
So I decided isolate the code and try to keep the minimum to identify the root cause and this is the code:
#include <SoftwareSerial.h>
#include <Adafruit_NeoPixel.h>
SoftwareSerial BT (10, 11);
#define PIN 2
#define NUMPIXELS 144
Adafruit_NeoPixel pixels(NUMPIXELS, PIN, NEO_GRBW + NEO_KHZ800);
int red = "";
void setup() {
Serial.begin(9600);
Serial.println("Ready");
BT.begin(38400);
pixels.begin();
pixels.show();
pixels.setBrightness(20);
}
void loop() {
while (BT.available()>0){
red = BT.parseInt();
Serial.println(red);
}
}
You describe that for the shown output you sent via mobile "0,19,255".
Yet in the shown output that obviously is only the second part of a longer sequence of numbers sent, which starts with "0,21".
Assuming that what you send is always of the format you have described, i.e. three numbers, separated by two ",", the shown output is most likely the result of you sending first "0,21,255" and then another triplet "0,19,255".
These two messages together would end up in an input buffer "0,21,2550,19,255".
Now I have to do some speculation. Most parsers, when told to look for numbers within the buffer, will look for digits followed by non-digits. They would end up yielding "0,21,2550".
Without knowing details of the parsers working it is hard to say how to fix your problem.
I would however definitly experiment with sending triplets which end in a non-digit.
For example:
"0,21,255,"
or
"0,21,255 "
or
"0,21,255;"
If none of them work you might need to explicitly expect the non-digit, i.e. between triplets of numbers read a character and either ignore it or compare it to " ,", " " or ";" for additional self-checking features.
(Writing this I rely on user zdf not intending to make an answer, because while I did spot the "2550" as "255""0", zdf spotted the only two "," inside the question body in sample input, which I missed. I will of course adapt my answer to one created by zdf, to not use their contribution without their consent.)

Arduino random() affecting string output

I am currently working on an arduino project using a Teensy 3.2.
The project uses some encryption code that exists in the library folder. When building the encryption code, any padding that was needed was hard-coded until the rest of the encryption worked. With the encryption working, I moved to randomize the padding process.
I am currently using the random() function from Wprogram.h.
When using manual padding, the output seems fine. However as soon as the random number generator is used, the output becomes wild and unintelligible.
When using manual padding:
//st is defined as byte[4][4]
st[0][0]=message; //character from original message, usually 'a' for testing
st[0][1]='1';
st[0][2]='2';
st[0][3]='3';
...
st[3][3]='k';
//Output
//set state: a123CDEF89fghijk //state prior to encryption
//encrypted: ⸮$2⸮⸮k6⸮tʠ&⸮⸮ //this should look funny
//decrypted to: a123CDEF89fghijk //matches plaintext
When using random padding, I've tried a few different ways:
//starting off with one random char for ease of testing
//random value seeded by randomSeed(analogRead(7));
st[0][0]=message;
st[0][1]=byte(random(33,127));//using 33 to 127 to avoid non-print chars
st[0][2]='2';
st[0][3]='3';
...
st[3][3]='k';
//-----Also Trying------
//It almost seemed like the data from the random number generator was flawed
//So I put in a scheme to ensure a hard coded character
if(random(100)%2)
st[0][1]='a';
else
st[0][1]='b';
//--------Also trying----------
//Since it seems like it is simply the existance of the random value that is
// problematic, I tried calling it without using it
int randval=random(100);
st[0][0]=message;
st[0][1]='1';
st[0][2]='2';
st[0][3]='3';
...
st[3][3]='k';
//Typical Output
//set state: a123CDEF89fghijk //state prior to encrypting(random char shows)
//encrypted: *3}3⸮⸮⸮⸮⸮⸮⸮⸮⸮N //this should look funny
//decrypted to: ⸮⸮⸮⸮⸮⸮⸮⸮k⸮-⸮⸮ //this should not look funny
From trying several different ways to create a random character and from also seeing that just the existence of the random value is causing trouble. I am suspecting that there is something buggy about how the random() function works on arduino/teensy.
Something else I noticed after setting up the random number and not using it, is that when I remove this line of code I have to recompile a couple of times before the code works correctly again. This is causing me to suspect the function even more or being buggy.
As I noted in the code samples, I am seeding the random number generator with a unused pin. Even though I would suspect it would be an issue here, I have tried a few different pins just to be safe.
So I guess my questions here might be: does anybody know what might be causing this problem or can possibly suggest and alternate way to get a random value?

C++ serial communication in Linux

I am trying to send data from my BeagleBone black board to Arduino Uno. The baud rate I have selected is 300. I am using the serialib library which is located here: http://serialib.free.fr/html/classserialib.html#ac8988727fef8e5d86c9eced17876f609 you can scroll all the way to the bottom to view the two files (serialib.h and serialib.cpp), however I have posted the main snippets here too. I read some reviews saying that this library is not reliable however I would first want to check my code before really suspecting anything else.
This is the program I have written in C++ on my BeagleBone:
#include <iostream>
#include "serialib.h"
#ifdef __linux__
#define DEVICE_PORT "/dev/ttyO1"
#endif
using namespace std;
int main()
{
serialib LS; //the main class to access
int Ret;
Ret= LS.Open(DEVICE_PORT,300);
if (Ret!=1)
{
cout<<"cant open port\n";
return 0;
}
else{cout<<"port now open \n";}
string xval="650X450Y";
for(int i=0;i<500;i++)//send xval 500 times
{
for(int j=0;j<xval.length();j++)//send each character separately
{
Ret=LS.WriteChar(xval[j]);
LS.Close();
LS.Open(DEVICE_PORT,300);
}
if (Ret!=1){cout<<"cannot write\n";}
else{cout<<"done writing\n";}
}
LS.Close();
cout<<"Transmission complete\n";
}
My code on the Arduino Uno is as follows:
#include <SoftwareSerial.h>
SoftwareSerial uart(10,11);
void setup() {
// put your setup code here, to run once:
Serial.begin(9600); //baud rate for Serial communication
uart.begin(300);
}
char x;
String data="";
void loop()
{
if(uart.available()>0)//check if data is coming in
{
while(uart.available())
{
x=uart.read();//read the incoming byte
data+=x;//append the string with incoming bytes
}
if(x=='Y')//received all the bits
{
Serial.println(data); //display received information
}
}
}
C++ - I am first opening up my UART port on the BeagleBone and sending the string "650X450Y" character by character which is repeated 500 times to see if my communication system is robust or not. As you can see that within the 'for' loop I am closing and opening this port after sending every character because without this, it sends quite a lot of wrong data and if this 'for' loop is very big then the writing process even stops (not sure why it behaves like that) thus after closing and opening this port every time, I have managed to reduce the errors significantly but there are still a few errors:
I sent this string 500 times but one sample I received on the Uno was "660X450Y", wrong value.
About 2-3 times out of 500, I am receiving "650X450Y650X450Y650X450Y" i.e. repetitions, this is definitely not the string length then how come it can send this data?.
The rest of the strings I received on the Uno are perfect.
On the Uno as you can see that I am reading in character by character and appending it to my string named data and printing out this data as soon as the byte 'Y' is detected which denotes the end of the string. I previously used the WriteString() function in my C++ code however that gave a number of errors, The code I have provided is the closest I have come so far in the last few days to make this system 100% reliable and robust after lots of debugging, however I'm really not sure why the system is still not 100%.
I saw the source code of both the files in the library and observed the WriteChar(char Byte) function which is defined at line 210 in serialib.cpp (link already provided above) and I see that this is the function transmitting the characters:
if (write(fd,&Byte,1)!=1) // Write the char
return -1; // Error while writting
return 1; // Write operation successfull
I don't see anything wrong with this function then why can't I receive the data with 100% accuracy, Is there anything wrong in both my source codes or either one?, should I opt for a different serial library?, in case I opt for other libraries and I still don't get my results then I think I may have to transmit this info in a wireless manner for e.g using bluetooth modules. If anyone has any suggestions/improvements regarding this problem then do let me know :), till then I'll try other methods to achieve a 100% accuracy.
The easiest just to check if quickly can be done in the linux terminal and no library is used for that.
Set the baudrate to 300 for the UART1 (stty -F /dev/ttyO1 raw & stty -F /dev/ttyO1 300)
You can sent data to the UART1 simply by writing(or reading) to the device file for the UART1 which is in your case /dev/ttyO1
Writing: echo "650X450Y" > /dev/ttyO1
Reading: cat /dev/ttyO1
If you want to do this in C Code you can use the open/read/write/close syscall functions.

Efficient and stable YAML parser for cocos2d-x

I am developing a game using cocos2d-x and C++, and I need to load a bunch of YAML files for this application. I tried using the yaml-cpp library with quite good results.
The problem is that this library seems to be very unstable (at least under cocos2d-x on iOS), since almost 20% of the time it fails loading the same YAML file throwing "end of map not found", "invalid map element", or errors like these ones.
I followed the HowToParseADocument guide, so I think I got it correct. But, since it's not 100% reliable, I am looking for something more stable. Eg:
long size = 0;
unsigned char *yaml = FileUtils::getInstance()->getFileData("file.yml", "r", &size);
std::stringstream is;
is << yaml;
YAML::Parser parser(is);
YAML::Node doc;
while(parser.GetNextDocument(doc)) {
instance->settings = doc.Clone();
}
The parser usally breaks at the parser.GetNextDocument(doc) call. The document I am trying to read is plain YAML with key: value lists in this simple form:
# Comment
section1:
param1: value1
param2: value2
# Comment
section2:
param1: value1
param2: value2
Edit
I am not allowed to disclose the content of the original YAML file, but I can give you some information:
It only contains maps, and not arrays, aliases or other particular constructs
Those values are integers, float or strings
It has been linted with this free tool, with success.
The code I used to read it, posted up there, it's always in that form, and I do not modify it to make the app run correctly. It's just that the app starts and works or starts and does not work. Since I am changing nothing in the middle, I really do not understand what's happening.
It's a bit hard to guess at the solution because you won't provide an actual example, but:
Who owns the data at the unsigned char* returned by getFileData? If that function itself owns the data, then it is no longer valid after the function returns, and so all sorts of crazy stuff might happen.
To validate what's happening here (beyond looking at the implementation of getFileData), you could print out is.string() before calling YAML::Parser parser(is); and see if that prints the expected YAML.