fast reading constant data stream from serial port in C++.net - c++

I'm trying to establish a SerialPort connection which transfers 16 bit data packages at a rate of 10-20 kHz. Im programming this in C++/CLI. The sender just enters an infinte while-loop after recieving the letter "s" and constantly sends 2 bytes with the data.
A Problem with the sending side is very unlikely, since a more simple approach works perfectly but too slow (in this approach, the reciever sends always an "a" first, and then gets 1 package consisting of 2 bytes. It leads to a speed of around 500Hz).
Here is the important part of this working but slow approach:
public: SerialPort^ port;
in main:
Parity p = (Parity)Enum::Parse(Parity::typeid, "None");
StopBits s = (StopBits)Enum::Parse(StopBits::typeid, "1");
port = gcnew SerialPort("COM16",384000,p,8,s);
port->Open();
and then doing as often as wanted:
port->Write("a");
int i = port->ReadByte();
int j = port->ReadByte();
This is now the actual approach im working with:
static int values[1000000];
static int counter = 0;
void reader(void)
{
SerialPort^ port;
Parity p = (Parity)Enum::Parse(Parity::typeid, "None");
StopBits s = (StopBits)Enum::Parse(StopBits::typeid, "1");
port = gcnew SerialPort("COM16",384000,p,8,s);
port->Open();
unsigned int i = 0;
unsigned int j = 0;
port->Write("s"); //with this command, the sender starts to send constantly
while(true)
{
i = port->ReadByte();
j = port->ReadByte();
values[counter] = j + (i*256);
counter++;
}
}
in main:
Thread^ readThread = gcnew Thread(gcnew ThreadStart(reader));
readThread->Start();
The counter increases (much more) rapidly at a rate of 18472 packages/s, but the values are somehow wrong.
Here is an example:
The value should look like this, with the last 4 bits changing randomly (its a signal of an analogue-digital converter):
111111001100111
Here are some values of the threaded solution given in the code:
1110011001100111
1110011000100111
1110011000100111
1110011000100111
So it looks like the connection reads the data in the middle of the package (to be exact: 3 bits too late). What can i do? I want to avoid a solution where this error is fixed later in the code while reading the packages like this, because I don't know if the the shifting error gets worse when I edit the reading code later, which I will do most likely.
Thanks in advance,
Nikolas
PS: If this helps, here is the code of the sender-side (an AtMega168), written in C.
uint8_t activate = 0;
void uart_puti16(uint16_t val) //function that writes the data to serial port
{
while ( !( UCSR0A & (1<<UDRE0)) ) //wait until serial port is ready
nop(); // wait 1 cycle
UDR0 = val >> 8; //write first byte to sending register
while ( !( UCSR0A & (1<<UDRE0)) ) //wait until serial port is ready
nop(); // wait 1 cycle
UDR0 = val & 0xFF; //write second byte to sending register
}
in main:
while(1)
{
if(active == 1)
{
uart_puti16(read()); //read is the function that gives a 16bit data set
}
}
ISR(USART_RX_vect) //interrupt-handler for a recieved byte
{
if(UDR0 == 'a') //if only 1 single data package is requested
{
uart_puti16(read());
}
if(UDR0 == 's') //for activating constant sending
{
active = 1;
}
if(UDR0 == 'e') //for deactivating constant sending
{
active = 0;
}
}

At the given bit rate of 384,000 you should get 38,400 bytes of data (8 bits of real data plus 2 framing bits) per second, or 19,200 two-byte values per second.
How fast is counter increasing in both instances? I would expect any modern computer to keep up with that rate whether using events or directly polling.
You do not show your simpler approach which is stated to work. I suggest you post that.
Also, set a breakpoint at the line
values[counter] = j + (i*256);
There, inspect i and j. Share the values you see for those variables on the very first iteration through the loop.

This is a guess based entirely on reading the code at http://msdn.microsoft.com/en-us/library/system.io.ports.serialport.datareceived.aspx#Y228. With this caveat out of the way, here's my guess:
Your event handler is being called when data is available to read -- but you are only consuming two bytes of the available data. Your event handler may only be called every 1024 bytes. Or something similar. You might need to consume all the available data in the event handler for your program to continue as expected.
Try to re-write your handler to include a loop that reads until there is no more data available to consume.

Related

Serial Read - Arduino - DELAY

I am a new programmer, so I am having a bit of problem with Serial communication of Arduino.
I am trying to read data from serial Input, sent by a simulation as characters and I need to store it as integer to write it with my servo.
I found this https://forum.arduino.cc/t/serial-input-basics-updated/382007 tutorial and example 4 does the job,
However, the simulation sends the data so fast that Arduino bottlenecks and the data pile up in the serial port and even if I stop the simulation the Arduino continues to perform the messages.
How can I slow down the data receiving like read data every 0.3 seconds instead. I tried to put some delays but it seems like it doesn't work.
Also, how can I change the code in a way that it stops performing new thing when there is no new serial messages and cancel the ones in the queue?
const byte numChars = 32;
char receivedChars[numChars]; // an array to store the received data
boolean newData = false;
//SERVO//
#include <Servo.h>
Servo myservo; // create servo object to control a servo
////////////////////////
int dataNumber = 0; // new for this version
void setup() {
Serial.begin(9600);
pinMode(LED_BUILTIN, OUTPUT);
myservo.attach(9); // attaches the servo on pin 9 to the servo object
Serial.println("<Arduino is ready>");
}
void loop() {
recvWithEndMarker();
showNewNumber();
}
void recvWithEndMarker() {
static byte ndx = 0;
char endMarker = '\n';
char rc;
if (Serial.available()> 0) {
rc = Serial.read();
if (rc != endMarker) {
receivedChars[ndx] = rc;
ndx++;
if (ndx >= numChars) {
ndx = numChars - 1;
}
}
else {
receivedChars[ndx] = '\0'; // terminate the string
ndx = 0;
newData = true;
delay(1);
}
}
}
void showNewNumber() {
if (newData == true) {
dataNumber = 0; // new for this version
dataNumber = atoi(receivedChars); // new for this version
Serial.print("This just in ... ");
Serial.println(receivedChars);
Serial.print("Data as Number ... "); // new for this version
Serial.println(dataNumber); // new for this version
myservo.write(dataNumber); // sets the servo position according to the scaled value
delay(50);
newData = false;
}
}
Thanks!
Welcome to the forum.
I'll admit that I don't know about your arduino set-up, but I hope that I can help.
Serial ports are asynchronous sources of data.
For base 3 wire RS-232, the receiver can't control the speed at which data is received other than baud rate hence received data is copied into a buffer (array) before it is processed.
This is to give your code time to process the received data before more messages arrive and cause what is known as a buffer overrun, corrupting data already received. Think of the serial link as being a hose pipe filling a bucket (the buffer) with water and you emptying it using a cup (your processing).
If your processing code is running too slowly and you are losing data then one option is to increase the size of the reception buffer, say from 32 to 255.
Adding delays to the reception code will only make matters worse.
An important point to make is that you must ensure that any processed data is removed from the buffer otherwise it will be processed again.
If your processing is fast enough then a nasty method is to just clear the buffer of all data by setting all array values to 0.
Another method is to use is keep a records (index value) of the next available location to write to and read from.
Data from the serial port is written into the buffer address using the write index value saved previously as a starting point. It is updated to take into account the size of the data written and incremented to indicate where to start the next write operation.
Your processing reads from the buffer using the last read index until it detects your end of message indicator and increments the read index to indicate the next location to read from.
It may be that your arduino serial port supports hardware flow control raising a Ready To Receive line when the hardware buffer (in the serial port itself) is full. This would be set before you open it.
Code:
Remove Delay calls - they only slow down your code.
Sending data out Serial.print, Serial.println commands take time,
place those after myservo.write
Remove Serial.print type commands that aren't strictly necessary.

Writing to uart serial port & receiving response, losing bytes when using nonblocking mode

I made a simple c++ program for armv7 architecture (compiled with linaro gnueabihf using raspi rootfs) that takes in arguments with baud rate, data, serial port etc and sends it to the selected serial port and receives the response. At least that's the goal of it.
I'm currently using it to send a command to disable/enable backlight on an industrial screen through an UART port. The screen takes a simple text command ended with crlf and returns a response. The specification of the screen says it uses 9600 baud, no parity, 8 data bits and 1 stop bit for communication, so pretty much standard.
While the sending works flawlessly - I cannot seem to find a way to properly receive the response. I tried configuring the termios port structure in multiple different ways (disabling hardware control, using cfmakeraw, configuring the VMIN and VTIME values) but without luck.
First thing is that, I'm receiving all the input byte by byte (so each read() call returns exactly 1 byte..), but that wouldn't be a problem.
When using nonblocking mode without select() I'm receiving all bytes, but I don't know when to stop receiving (and I want it to be universal, so I send a command, expect a simple response and if there is no more data then just exit). I made a time counter since the last message, so if nothing was received in last ~500ms then I assume nothing more will come. But this sometimes loses some bytes of the response and I don't know why.
When using blocking mode, I receive correct bytes (still byte by byte though), but I don't know when to stop and the last call to read() leaves the program hanging, because nothing else comes in the input.
When adding select() to the blocking call, to see if input is readable, I get very frequent data loss (sometimes just receiving a few bytes), and sometimes select returns 1, but read() blocks, and I'm left hanging.
When I just send data without doing any reading, and look at the input using cat -v < /dev/ttyS3 I can actually see correct input on the serial port all the time, however when I run both cat and my program as receivers, only one of them gets the data (or cat receives a few bytes and my program a few), this suggests me that something is "stealing" my bytes the same way when I try to read it, but what could it be, and why is it like that?
My current code (using the nonblocking read + 500ms timeout), that still loses some bytes from time to time:
#include <stdio.h>
#include <fcntl.h>
#include <termios.h>
#include <unistd.h>
#include <string.h>
#include <stdlib.h>
#include <errno.h>
#include <time.h>
int open_port(char* portname)
{
int fd; // file description for the serial port
fd = open(portname, O_RDWR | O_NOCTTY | O_NDELAY);
if(fd == -1) // if open is unsucessful
{
printf("Error: open_port: Unable to open %s. \n", portname);
}
else
{
//fcntl(fd, F_SETFL, 0);
fcntl(fd, F_SETFL, FNDELAY);
}
return(fd);
}
int configure_port(int fd, int baud_rate)
{
struct termios port_settings;
tcgetattr(fd, &port_settings);
cfsetispeed(&port_settings, baud_rate); // set baud rates
cfsetospeed(&port_settings, baud_rate);
cfmakeraw(&port_settings);
port_settings.c_cflag &= ~PARENB; // no parity
port_settings.c_cflag &= ~CSTOPB; // 1 stop bit
port_settings.c_cflag &= ~CSIZE;
port_settings.c_cflag |= CS8; // 8 data bits
tcsetattr(fd, TCSANOW, &port_settings); // apply the settings to the port
return(fd);
}
/**
* Convert int baud rate to actual baud rate from termios
*/
int get_baud(int baud)
{
switch (baud) {
case 9600:
return B9600;
case 19200:
return B19200;
case 38400:
return B38400;
case 57600:
return B57600;
case 115200:
return B115200;
case 230400:
return B230400;
case 460800:
return B460800;
case 500000:
return B500000;
case 576000:
return B576000;
case 921600:
return B921600;
case 1000000:
return B1000000;
case 1152000:
return B1152000;
case 1500000:
return B1500000;
case 2000000:
return B2000000;
case 2500000:
return B2500000;
case 3000000:
return B3000000;
case 3500000:
return B3500000;
case 4000000:
return B4000000;
default:
return -1;
}
}
unsigned char* datahex(char* string) {
if(string == NULL)
return NULL;
size_t slength = strlen(string);
if((slength % 2) != 0) // must be even
return NULL;
size_t dlength = slength / 2;
unsigned char* data = (unsigned char*)malloc(dlength);
memset(data, 0, dlength);
size_t index = 0;
while (index < slength) {
char c = string[index];
int value = 0;
if(c >= '0' && c <= '9')
value = (c - '0');
else if (c >= 'A' && c <= 'F')
value = (10 + (c - 'A'));
else if (c >= 'a' && c <= 'f')
value = (10 + (c - 'a'));
else {
free(data);
return NULL;
}
data[(index/2)] += value << (((index + 1) % 2) * 4);
index++;
}
return data;
}
int main(int argc, char **argv) {
int baud_rate = B9600;
baud_rate = get_baud(atoi(argv[1]));
if(baud_rate == -1) {
printf("Error: Cannot convert baud rate %s, using 9600\n", argv[1]);
baud_rate = B9600;
}
bool convertHex = false;
char portName[24] = "/dev/ttyS0";
bool debug = false;
bool noreply = false;
for(int i = 3; i < argc; i++) {
if(!strcmp(argv[i], "hex"))
convertHex = true;
else if(strstr(argv[i], "/dev/") != NULL)
strncpy(portName, argv[i], sizeof(portName));
else if(!strcmp(argv[i], "debug"))
debug = true;
else if(!strcmp(argv[i], "no-reply"))
noreply = true;
}
unsigned char* data = nullptr;
size_t len = 0;
if(convertHex) {
data = datahex(argv[2]);
if((int)data == (int)NULL) {
convertHex = false;
printf("Error: Couldn't convert hex value! Needs to be even length (2 chars per byte)\n");
}
else
len = strlen(argv[2])/2;
}
if(!convertHex) {
data = (unsigned char*)argv[2];
len = strlen(argv[2]);
}
int fd = open_port(portName);
if(fd == -1) {
printf("Error: Couldn't open port %s\n", portName);
if(convertHex)
free(data);
return 0;
}
configure_port(fd, baud_rate);
if(debug) {
printf("Sending data (raw): ");
for(int i =0; i< len; i++) {
printf("%02X", data[i]);
}
printf("\n");
}
size_t writelen = write(fd, data, len);
if(debug)
printf("Sent %d/%d bytes\n", writelen, len);
if(writelen != len)
printf("Error: not all bytes were sent (%d/%d)\n", writelen, len);
else if(noreply)
printf("WRITE OK");
if(!noreply) {
unsigned char ibuff[512] = {0};
int curlen = 0; // full length
clock_t begin_time = clock();
while(( float(clock() - begin_time) / CLOCKS_PER_SEC) < 0.5 && curlen < sizeof(ibuff)) {
int ret = read(fd, ibuff+curlen, sizeof(ibuff)-curlen-1);
if(ret < 0) {
ret = 1;
continue;
}
if(ret > 0) {
curlen += ret;
begin_time = clock();
}
}
if(curlen > 0) {
ibuff[curlen] = 0; // null terminator
printf("RESPONSE: %s", ibuff);
}
}
if(fd)
close(fd);
if(convertHex)
free(data);
return 0;
}
I launch the program like ./rs232 9600 [hex string] hex debug
The scren should return a response like #BLIGHT_ON!OK, but sometimes I receive for example #BLI_ON!O
What can be the cause of this? I made some serial communcation earlier with QtSerial <-> STM32 controller and had no such issues that would cause data loss.
First thing is that, I'm receiving all the input byte by byte (so each
read() call returns exactly 1 byte..) [...]
That's not surprising. The response is coming back at 9600 baud, which is likely much slower per byte than one iteration of the loop requires. It would also arise directly from some configurations of the serial driver. It should be possible to tune this by manipulating VMIN and VTIME, but do note that that requires disabling canonical mode (which you probably want to do anyway; see below).
When using nonblocking mode without select() I'm receiving all bytes,
but I don't know when to stop receiving (and I want it to be
universal, so I send a command, expect a simple response and if there
is no more data then just exit). I made a time counter since the last
message, so if nothing was received in last ~500ms then I assume
nothing more will come. But this sometimes loses some bytes of the
response and I don't know why.
It's all in the details, which you have not presented for that case. We cannot therefore speak to your particular data losses.
Generally speaking, if you're working without flow control, then you have to be sure to read each byte before the next one arrives, on average, else pretty soon, new bytes will overwrite previously-received ones. VMIN and VTIME can help with that, or one can try other methods for tune read timing, but note well that a 9600 baud response will deliver bytes at a rate exceeding one per millisecond, so a 500 ms delay between read attempts is much too long. Supposing that the particular responses you are trying to read are relatively short, however, this will not explain the data losses.
When using blocking mode, I receive correct bytes (still byte by byte
though), but I don't know when to stop and the last call to read()
leaves the program hanging, because nothing else comes in the input.
So the command is required to be CRLF-terminated, but the response cannot be relied upon to be likewise terminated? What a rude device you're working with. If it terminated its responses the same way it required terminated commands, then you could probably work in canonical mode, and you could definitely watch for the terminator to recognize end-of-transmission.
When adding select() to the blocking call, to see if input is
readable, I get very frequent data loss (sometimes just receiving a
few bytes), and sometimes select returns 1, but read() blocks, and I'm
left hanging.
I cannot suggest what the problem may be in that case without any relevant code to analyze, but you really shouldn't need select() for this.
When I just send data without doing any reading, and look at the input
using cat -v < /dev/ttyS3 I can actually see correct input on the
serial port all the time,
That's a good test.
however when I run both cat and my program
as receivers, only one of them gets the data (or cat receives a few
bytes and my program a few),
That's exactly as I would expect. Once a program reads a byte from the port, it is no longer available for any other program to read. Thus, if multiple programs try to read from the same port at the same time then the data available will be partitioned among them in some unspecified and not necessarily consistent fashion.
this suggests me that something is
"stealing" my bytes the same way when I try to read it, but what could
it be, and why is it like that?
That seems unlikely, considering that cat is not affected the same way when you run it alone, nor (you report) are some versions of your own program.
In the first place, if the device supports flow control then I would enable it. Hardware flow control in preference to software flow control if both are viable. This is mainly a fail-safe, however -- I don't see any reason to think that flow control is likely to actually trigger if your program is well written.
Mainly, then, in addition to setting the serial line parameters (8/n/1), you should
Disable canonical mode. This is necessary because you (apparently) cannot rely on the response to be terminated by a line terminator, among other reasons.
Disable echo.
Avoid enabling non-blocking mode on the file.
(Optional) read the first response byte with VMIN == 1 and VTIME == 0; this allows for an arbitrary delay before the device starts sending the response. Alternatively, if you have a reliable upper bound on the time you're willing to wait for the device to start sending the response then you can probably skip this step by using a suitable VTIME in the next one. Or perhaps use a a larger VTIME for this first byte to accommodate a delay before start of transmission, yet not hang if the device fails to respond.
Do read the remaining response bytes with VTIME == 1 (or larger) and VMIN == 0. This probably gets you the whole remainder of the response in one call, but do repeat the read() until it returns 0 (or negative). The 0 return indicates that all available bytes have been transferred and no new ones were received for VTIME tenths of a second -- much longer than the inter-character time in a 9600-baud transmission even for VTIME == 1. Do note that the larger you make VTIME, the longer will be the delay between the device sending the last byte of its response and the program detecting end-of-transmission.
Do not implement any artificial delay between successive read attempts.
You should not need non-blocking mode at the fcntl level, and you should not need select(). There may be other termios settings you could apply to better tune your program for the particular device at the other end of the serial link, but the above should be enough for single-command / single-response pairs with ASCII-only data and no control characters other than carriage returns and newlines.

How to capture and store sensor data from Arduino into C++

I am writing a code to capture serial readings from the Arduino to C++
Is there a way to capture the readings line by line and then store it into an array? I have read another post similar to mine, but I am still unable to apply it.
Any help is greatly appreciated, thank you.
Environment setup:
Arduino UNO
ADXL 335 accelerometer
Ubuntu 16.04
C++
[Updated] applied solution from Bart
Cpp file
The reason why I added the "for-loop with print and break" is to analyze the array contents.
#include <stdio.h>
#include <string.h>
#include <iostream>
#include <unistd.h>
using namespace std;
char serialPortFilename[] = "/dev/ttyACM0";
int main()
{
char readBuffer[1024];
FILE *serPort = fopen(serialPortFilename, "r");
if (serPort == NULL)
{
printf("ERROR");
return 0;
}
while(1)
{
usleep(1000); //sync up Linux and Arduino
memset(readBuffer, 0, 1024);
fread(readBuffer, sizeof(char),1024,serPort);
for(int i=0; i<1024; i++){
printf("%c",readBuffer[i]);
}
break;
}
return 0;
}
Ino file
Fetching data from the Accelerometer
#include <stdio.h>
const int xPin = A0;
const int yPin = A1;
const int zPin = A2;
void setup() {
Serial.begin(9600);
}
void loop() {
int x = 0, y = 0, z = 0;
x = analogRead(xPin);
y = analogRead(yPin);
z = analogRead(zPin);
char buffer[16];
int n;
n = sprintf(buffer,"<%d,%d,%d>",x,y,z);
Serial.write(buffer);
}
Results
Running the code for three times
Click Here
The ideal outputs should be
<a,b,c><a,b,c><a,b,c>...
but right now, some of the outputs has the values inside "corrupted" (please see the fourth line from the top).
Even if use the start and end markers to determine a correct dataset, the data within the set is still wrong. I suspect the issue lies with the char array from C++, due to it being unsynchronized with Arduino. Else I need to send by Bytes from Arduino (not really sure how)
When dealing with two programs running on different processors they will never start sending/receiving at the same time. What you likely see is not that the results are merged wrong it is more likely the reading program started and stopped half way through the data.
When sending data over a line it is best that you:
On the Arduino:
First frame the data.
Send the frame.
On Linux:
Read in data in a buffer.
Search the buffer for a complete frame and deframe.
1. Framing the data
With framing the data I mean that you need a structure which you can recognize and validate on the receiving side. For example you could add the characters STX and ETX as control characters around your data. When the length of your data varies it is also required to send this.
In the following example we take that the data array is never longer than 255 bytes. This means that you can store the length in a single byte. Below you see pseudo code of how a frame could look like:
STX LENGTH DATA_ARRAY ETX
The total length of the bytes which will be send are thus the length of the data plus three.
2. Sending
Next you do not use println but Serial.write(buf, len) instead.
3. Receiving
On the receiving side you have a buffer in which all data received will be appended.
4. Deframing
Next each time new data has been added search for an STX character, assume the next character is the length. Using the length +1 you should find a ETX. If so you have found a valid frame and you can use the data. Next remove it from the buffer.
for(uint32_t i = 0; i < (buffer.size() - 2); ++i)
{
if(STX == buffer[i])
{
uint8_t length = buffer[i+2];
if(buffer.size() > (i + length + 3) && (ETX == buffer[i + length + 2]))
{
// Do something with the data.
// Clear the buffer from every thing before i + length + 3
buffer.clear(0, i + length + 3);
// Break the loop as by clearing the data the current index becomes invalid.
break;
}
}
}
For an example also using a Cyclic Redundancy Check (CRC) see here

DMA write to SD card (SSP) doesn't write bytes

I'm currently working on replacing a blocking busy-wait implementation of an SD card driver over SSP with a non-blocking DMA implementation. However, there are no bytes actually written, even though everything seems to go according to plan (no error conditions are ever found).
First some code (C++):
(Disclaimer: I'm still a beginner in embedded programming so code is probably subpar)
namespace SD {
bool initialize() {
//Setup SSP and detect SD card
//... (removed since not relevant for question)
//Setup DMA
LPC_SC->PCONP |= (1UL << 29);
LPC_GPDMA->Config = 0x01;
//Enable DMA interrupts
NVIC_EnableIRQ(DMA_IRQn);
NVIC_SetPriority(DMA_IRQn, 4);
//enable SSP interrupts
NVIC_EnableIRQ(SSP2_IRQn);
NVIC_SetPriority(SSP2_IRQn, 4);
}
bool write (size_t block, uint8_t const * data, size_t blocks) {
//TODO: support more than one block
ASSERT(blocks == 1);
printf("Request sd semaphore (write)\n");
sd_semaphore.take();
printf("Writing to block " ANSI_BLUE "%d" ANSI_RESET "\n", block);
memcpy(SD::write_buffer, data, BLOCKSIZE);
//Start the write
uint8_t argument[4];
reset_argument(argument);
pack_argument(argument, block);
if (!send_command(CMD::WRITE_BLOCK, CMD_RESPONSE_SIZE::WRITE_BLOCK, response, argument)){
return fail();
}
assert_cs();
//needs 8 clock cycles
delay8(1);
//reset pending interrupts
LPC_GPDMA->IntTCClear = 0x01 << SD_DMACH_NR;
LPC_GPDMA->IntErrClr = 0x01 << SD_DMACH_NR;
LPC_GPDMA->SoftSReq = SD_DMA_REQUEST_LINES;
//Prepare channel
SD_DMACH->CSrcAddr = (uint32_t)SD::write_buffer;
SD_DMACH->CDestAddr = (uint32_t)&SD_SSP->DR;
SD_DMACH->CLLI = 0;
SD_DMACH->CControl = (uint32_t)BLOCKSIZE
| 0x01 << 26 //source increment
| 0x01 << 31; //Terminal count interrupt
SD_SSP->DMACR = 0x02; //Enable ssp write dma
SD_DMACH->CConfig = 0x1 //enable
| SD_DMA_DEST_PERIPHERAL << 6
| 0x1 << 11 //mem to peripheral
| 0x1 << 14 //enable error interrupt
| 0x1 << 15; //enable terminal count interrupt
return true;
}
}
extern "C" __attribute__ ((interrupt)) void DMA_IRQHandler(void) {
printf("dma irq\n");
uint8_t channelBit = 1 << SD_DMACH_NR;
if (LPC_GPDMA->IntStat & channelBit) {
if (LPC_GPDMA->IntTCStat & channelBit) {
printf(ANSI_GREEN "terminal count interrupt\n" ANSI_RESET);
LPC_GPDMA->IntTCClear = channelBit;
}
if (LPC_GPDMA->IntErrStat & channelBit) {
printf(ANSI_RED "error interrupt\n" ANSI_RESET);
LPC_GPDMA->IntErrClr = channelBit;
}
SD_DMACH->CConfig = 0;
SD_SSP->IMSC = (1 << 3);
}
}
extern "C" __attribute__ ((interrupt)) void SSP2_IRQHandler(void) {
if (SD_SSP->MIS & (1 << 3)) {
SD_SSP->IMSC &= ~(1 << 3);
printf("waiting until idle\n");
while(SD_SSP->SR & (1UL << 4));
//Stop transfer token
//I'm not sure if the part below up until deassert_cs is necessary.
//Adding or removing it made no difference.
SPI::send(0xFD);
{
uint8_t response;
unsigned int timeout = 4096;
do {
response = SPI::receive();
} while(response != 0x00 && --timeout);
if (timeout == 0){
deassert_cs();
printf("fail");
return;
}
}
//Now wait until the device isn't busy anymore
{
uint8_t response;
unsigned int timeout = 4096;
do {
response = SPI::receive();
} while(response != 0xFF && --timeout);
if (timeout == 0){
deassert_cs();
printf("fail");
return;
}
}
deassert_cs();
printf("idle\n");
SD::sd_semaphore.give_from_isr();
}
}
A few remarks about the code and setup:
Written for the lpc4088 with FreeRTOS
All SD_xxx defines are conditional defines to select the right pins (I need to use SSP2 in my dev setup, SSP0 for the final product)
All external function that are not defined in this snippet (e.g. pack_argument, send_command, semaphore.take() etc.) are known to be working correctly (most of these come from the working busy-wait SD implementation. I can't of course guarantee 100% that they are bugless, but they seem to be working right.).
Since I'm in the process of debugging this there are a lot of printfs and hardcoded SSP2 variables. These are of course temporarily.
I mostly used this as example code.
Now I have already tried the following things:
Write without DMA using busy-wait over SSP. As mentioned before I started with a working implementation of this, so I know the problem has to be in the DMA implementation and not somewhere else.
Write from mem->mem instead of mem->sd to eliminate the SSP peripheral. mem->mem worked fine, so the problem must be in the SSP part of the DMA setup.
Checked if the ISRs are called. They are: first the DMA IRS is called for the terminal count interrupt, and then the SSP2 IRS is called. So the IRSs are (probably) setup correctly.
Made a binary dump of the entire sd content to see if it the content might have been written to the wrong location. Result: the content send over DMA was not present anywhere on the SD card (I did this with any change I made to the code. None of it got the data on the SD card).
Added a long (~1-2 seconds) timeout in the SSP IRS by repeatedly requesting bytes from the SD card to make sure that there wasn't a timeout issue (e.g. that I tried to read the bytes before the SD card had the chance to process everything). This didn't change the outcome at all.
Unfortunately due to lack of hardware tools I haven't been able yet to verify if the bytes are actually send over the data lines.
What is wrong with my code, or where can I look to find the cause of this problem? After spending way more hours on this then I'd like to admit I really have no idea how to get this working and any help is appreciated!
UPDATE: I did a lot more testing, and thus I got some more results. The results below I got by writing 4 blocks of 512 bytes. Each block contains constantly increasing numbers module 256. Thus each block contains 2 sequences going from 0 to 255. Results:
Data is actually written to the SD card. However, it seems that the first block written is lost. I suppose there is some setup done in the write function that needs to be done earlier.
The bytes are put in a very weird (and wrong) order: I basically get alternating all even numbers followed by all odd numbers. Thus I first get even numbers 0x00 - 0xFE and then all odd numbers 0x01 - 0xFF (total number of written bytes seems to be correct, with the exception of the missing first block). However, there's even one exception in this sequence: each block contains 2 of these sequences (sequence is 256 bytes, block is 512), but the first sequence in each block has 0xfe and 0xff "swapped". That is, 0xFF is the end of the even numbers and 0xFE is the end of the odd series. I have no idea what kind of black magic is going on here. Just in case I've done something dumb here's the snippet that writes the bytes:
uint8_t block[512];
for (int i = 0; i < 512; i++) {
block[i] = (uint8_t)(i % 256);
}
if (!SD::write(10240, block, 1)) { //this one isn't actually written
WARN("noWrite", proc);
}
if (!SD::write(10241, block, 1)) {
WARN("noWrite", proc);
}
if (!SD::write(10242, block, 1)) {
WARN("noWrite", proc);
}
if (!SD::write(10243, block, 1)) {
WARN("noWrite", proc);
}
And here is the raw binary dump. Note that this exact pattern is fully reproducible: so far each time I tried this I got this exact same pattern.
Update2: Not sure if it's relevant, but I use sdram for memory.
When I finally got my hands on a logic analyzer I got a lot more information and was able to solve these problems.
There were a few small bugs in my code, but the bug that caused this behaviour was that I didn't send the "start block" token (0xFE) before the block and I didn't send the 16 bit (dummy) crc after the block. When I added these to the transfer buffer everything was written successfully!
So this fix was as followed:
bool write (size_t block, uint8_t const * data, size_t blocks) {
//TODO: support more than one block
ASSERT(blocks == 1);
printf("Request sd semaphore (write)\n");
sd_semaphore.take();
printf("Writing to block " ANSI_BLUE "%d" ANSI_RESET "\n", block);
SD::write_buffer[0] = 0xFE; //start block
memcpy(&SD::write_buffer[1], data, BLOCKSIZE);
SD::write_buffer[BLOCKSIZE + 1] = 0; //dummy crc
SD::write_buffer[BLOCKSIZE + 2] = 0;
//...
}
As a side note, the reason why the first block wasn't written was simply because I didn't wait until the device was ready before sending the first block. Doing so fixed the problem.

How to reduce cpu usage during data transfer on TCP ports realtime

I have a socket program which acts like both client and server.
It initiates connection on an input port and reads data from it. On a real time scenario it reads data on input port and sends the data (record by record ) on to the output port.
The problem here is that while sending data to the output port CPU usage increases to 50% while is not permissible.
while(1)
{
if(IsInputDataAvail==1)//check if data is available on input port
{
//condition to avoid duplications while sending
if( LastRecordSent < LastRecordRecvd )
{
record_time temprt;
list<record_time> BufferList;
list<record_time>::iterator j;
list<record_time>::iterator i;
// Storing into a temp list
for(i=L.begin(); i != L.end(); ++i)
{
if((i->recordId > LastRecordSent) && (i->recordId <= LastRecordRecvd))
{
temprt.listrec = i->listrec;
temprt.recordId = i->recordId;
temprt.timestamp = i->timestamp;
BufferList.push_back(temprt);
}
}
//Sending to output port
for(j=BufferList.begin(); j != BufferList.end(); ++j)
{
LastRecordSent = j->recordId;
std::string newlistrecord = j->listrec;
newlistrecord.append("\n");
char* newrecord= new char [newlistrecord.size()+1];
strcpy (newrecord, newlistrecord.c_str());
if ( s.OutputClientAvail() == 1) //check if output client is available
{
int ret = s.SendBytes(newrecord,strlen(newrecord));
if ( ret < 0)
{
log1.AddLogFormatFatal("Nice Send Thread : Nice Client Disconnected");
--connected;
return;
}
}
else
{
log1.AddLogFormatFatal("Nice Send Thread : Nice Client Timedout..connection closed");
--connected; //if output client not available disconnect after a timeout
return;
}
}
}
}
// Sleep(100); if we include sleep here CPU usage is less..but to send data real time I need to remove this sleep.
If I remove Sleep()...CPU usage goes very high while sending data to out put port.
}//End of while loop
Any possible ways to maintain real time data transfer and reduce CPU usage..please suggest.
There are two potential CPU sinks in the listed code. First, the outer loop:
while (1)
{
if (IsInputDataAvail == 1)
{
// Not run most of the time
}
// Sleep(100);
}
Given that the Sleep call significantly reduces your CPU usage, this spin-loop is the most likely culprit. It looks like IsInputDataAvail is a variable set by another thread (though it could be a preprocessor macro), which would mean that almost all of that CPU is being used to run this one comparison instruction and a couple of jumps.
The way to reclaim that wasted power is to block until input is available. Your reading thread probably does so already, so you just need some sort of semaphore to communicate between the two, with a system call to block the output thread. Where available, the ideal option would be sem_wait() in the output thread, right at the top of your loop, and sem_post() in the input thread, where it currently sets IsInputDataAvail. If that's not possible, the self-pipe trick might work in its place.
The second potential CPU sink is in s.SendBytes(). If a positive result indicates that the record was fully sent, then that method must be using a loop. It probably uses a blocking call to write the record; if it doesn't, then it could be rewritten to do so.
Alternatively, you could rewrite half the application to use select(), poll(), or a similar method to merge reading and writing into the same thread, but that's far too much work if your program is already mostly complete.
if(IsInputDataAvail==1)//check if data is available on input port
Get rid of that. Just read from the input port. It will block until data is available. This is where most of your CPU time is going. However there are other problems:
std::string newlistrecord = j->listrec;
Here you are copying data.
newlistrecord.append("\n");
char* newrecord= new char [newlistrecord.size()+1];
strcpy (newrecord, newlistrecord.c_str());
Here you are copying the same data again. You are also dynamically allocating memory, and you are also leaking it.
if ( s.OutputClientAvail() == 1) //check if output client is available
I don't know what this does but you should delete it. The following send is the time to check for errors. Don't try to guess the future.
int ret = s.SendBytes(newrecord,strlen(newrecord));
Here you are recomputing the length of the string which you probably already knew back at the time you set j->listrec. It would be much more efficient to just call s.sendBytes() directly with j->listrec and then again with "\n" than to do all this. TCP will coalesce the data anyway.