QCustomPlot Huge Amount of Data Plotting - c++

I am trying to plot some serial data on my Qt Gui program using qcustomplot class. I had no trouble when I tried to plot low sampling frequency datas like 100 data/second. The graph was really cool and was plotting the data smoothly. But at high sampling rates such 1000data/second, plotter makes a bottleneck for serial read function. It slow downs serial there was a huge delay like 4-5 seconds apart from device. Straightforwardly, plotter could not reach the data stream speed. So, is there any common issue which i dont know about or any recommendation?
I thougth these scenarious,
1- to devide whole program to 2 or 3 thread. For example, serial part runs in one thread and plotting part runs in another thread and two thread communicates with a QSemaphore
2- fps of qcustom plot is limited. but there should be a solution because NI LABVIEW plots up to 2k of datas without any delay
3- to desing a new virtual serial device in usb protocol. Now, I am using ft232rl serial to usb convertor.
4- to change programming language. What is the situation and class support in C# or java for realtime plotting? (I know it is like a kid saying, but this is a pretex to be experienced in other languages)
My serial device send data funct(it is foo device for experiment there is no serious coding) is briefly that:
void progTask()
{
DelayMsec(1); //my delay function, milisecond
//read value from adc13
Adc13Read(adcValue.ui32Part);
sendData[0] = (char)'a';
sendData[1] = (char)'k';
sendData[2] = adcValue.bytes[0];
sendData[3] = (adcValue.bytes[1] & 15);
Qt Program read function is that:
//send test data
UARTSend(UART6_BASE,&sendData[0],4);
}
union{
unsigned char bytes[2];
unsigned int intPart;
unsigned char *ptr;
}serData;
void MedicalSoftware::serialReadData()
{
if(serial->bytesAvailable()<4)
{
//if the frame size is less than 4 bytes return and
//wait to full serial receive buffer
//note: serial->setReadBufferSize(4)!!!!
return;
}
QByteArray serialInData = serial->readAll();
//my algorithm
if(serialInData[0] == 'a' && serialInData[1] == 'k')
{
serData.bytes[0] = serialInData[2];
serData.bytes[1] = serialInData[3];
}else if(serialInData[2] == 'a' && serialInData[3] == 'k')
{
serData.bytes[0] = serialInData[0];
serData.bytes[1] = serialInData[1];
}
else if(serialInData[1] == 'a' && serialInData[2] == 'k')
{
serial->read(1);
return;
}else if(serialInData[0] == 'k' && serialInData[3] == 'a')
{
serData.bytes[0] = serialInData[1];
serData.bytes[1] = serialInData[2];
}
plotMainGraph(serData.intPart);
serData.intPart = 0;
}
And qcustom plot setting fuction is:
void MedicalSoftware::setGraphsProperties()
{
//MAIN PLOTTER
ui->mainPlotter->addGraph();
ui->mainPlotter->xAxis->setRange(0,2000);
ui->mainPlotter->yAxis->setRange(-0.1,3.5);
ui->mainPlotter->xAxis->setLabel("Time(s)");
ui->mainPlotter->yAxis->setLabel("Magnitude(mV)");
QSharedPointer<QCPAxisTickerTime> timeTicker(new QCPAxisTickerTime());
timeTicker->setTimeFormat("%h:%m:%s");
ui->mainPlotter->xAxis->setTicker(timeTicker);
ui->mainPlotter->axisRect()->setupFullAxesBox();
QPen pen;
pen.setColor(QColor("blue"));
ui->mainPlotter->graph(0)->setPen(pen);
dataTimer = new QTimer;
}
And the last is plot function:
void MedicalSoftware::plotMainGraph(const quint16 serData)
{
static QTime time(QTime::currentTime());
double key = time.elapsed()/1000.0;
static double lastPointKey = 0;
if(key-lastPointKey>0.005)
{
double value0 = serData*(3.3/4096);
ui->mainPlotter->graph(0)->addData(key,value0);
lastPointKey = key;
}
ui->mainPlotter->xAxis->setRange(key+0.25, 2, Qt::AlignRight);
counter++;
ui->mainPlotter->replot();
counter = 0;
}

Quick answer:
Have you tried:
ui->mainPlotter->replot(QCustomPlot::rpQueuedReplot);
according to the documentation it can improves performances when doing a lot of replots.
Longer answer:
My feeling on your code is that you are trying to replot as often as you can to get a "real time" plot. But if you are on a PC with a desktop OS there is no such thing as real time.
What you should care about is:
Ensure that the code that read/write to the serial port is not delayed too much. "Too much" is to be interpreted with respect to the connected hardware. If it gets really time critical (which seems to be your case) you have to optimize your read/write functions and eventually put them alone in a thread. This can go as far as reserving a full hardware CPU core for this thread.
Ensure that the graph plot is refreshed fast enough for the human eyes. You do not need to do a full repaint each time you receive a single data point.
In your case you receive 1000 data/s which make 1 data every ms. That is quite fast because that is beyond the default timer resolution of most desktop OS. That means you are likely to have more than a single point of data when calling your "serialReadData()" and that you could optimize it by calling it less often (e.g call it every 10ms and read 10 data points each time). Then you could call "replot()" every 30ms which would add 30 new data points each time, skip about 29 replot() calls every 30ms compared to your code and give you ~30fps.
1- to devide whole program to 2 or 3 thread. For example, serial part
runs in one thread and plotting part runs in another thread and two
thread communicates with a QSemaphore
Dividing the GUI from the serial part in 2 threads is good because you will prevent a bottleneck in GUI to block the serial communication. Also you could skip using semaphore and simply rely on Qt signal/slot connections (connected in Qt::QueuedConnection mode).
4- to change programming language. What is the situation and class
support in C# or java for realtime plotting? (I know it is like a kid
saying, but this is a pretex to be experienced in other languages)
Changing the programming language, in best case, won't change anything or could hurt your performances, especially if you go toward languages which are not compiled to native CPU instructions.
Changing the plotting library on the other hand could change the performances. You can look at Qt Charts and Qwt. I do not know how they compare to QCustomPlot though.

Related

Rising edge interrupt triggering multiple times on STM32 Nucleo

I am using the STM32 NUCLEO-F401RE microcontroller board.
I have a speaker programmed to change frequency by a set amount when the joystick is pushed up/down. My issue is that sometimes (more often than not) when the joystick is pushed up/down the frequency increases/decreases multiple times, implying that the ISR is executing multiple times. Also, the InterruptIn object is set to trigger on the rising edge however sometimes it also executes on the falling edge (when the joystick is returend to neutral after pushing up/down). Any help for getting over this?
void upISR()
{
if (greenLED.getStatus())
{
myTicker.detach();
frequency+=200;
myTicker.attach(callback(&spkr, &Speaker::toggle), 0.5/frequency);
}
}
'
int main()
{
InterruptIn up(A2);
InterruptIn down(A3);
InterruptIn fire(D4);
up.rise(&upISR);
down.rise(&downISR);
fire.rise(&toggleISR);
redLED.on();
while (1){}
}
Mechanical switch bounce is a feature of all mechanical switches to a lesser or greater extent. It is often necessary to implement "debouncing" in software especially if the switch is directly driving an interrupt as in this case.
A quick Google search for software denounce techniques yields some rather poor techniques IMO. I seen it done poorly more times than well unfortunately.
I suggest that in the switch ISR you start (or restart in the event of a "bounce") a hardware timer for a period of say 20ms or so (longer than the switch bounce time, but shorter than the time you could possibly to genuinely release the switch). Then in the timer ISR, you test the state of the switch and change the frequency accordingly:
Pseudocode:
void upISR()
{
debounceTimerRestart() ;
}
void downISR()
{
debounceTimerRestart() ;
}
void debounceTimerISR()
{
debounceTimerStop() ;
tDirection dir = getJoystickDir() ;
swithc( dir )
{
case UP :
{
increaseFrquency() ;
}
break ;
case DN :
{
decreaseFrquency() ;
}
break ;
}
}
What this does is trigger a timer interrupt shortly ("debounce time") after the switch stops bouncing. Note the timer is "single-shot" not periodic.
Below I present an enhancement at #BenVoigt's suggestion (in comments). I am keeping it separate to make it clear it was his work. The above will generally work, but if you have a particularly poor switch the following would resolve issues, and at little cost, so you may as well:
void debounceTimerISR()
{
debounceTimerStop() ;
static tDirection previous_dir = CENTRE ;
tDirection dir = getJoystickDir() ;
// If the state changed...
if( previous_dir != dir )
{
previous_dir = dir ;
switch( dir )
{
case UP :
{
increaseFrquency() ;
}
break ;
case DN :
{
decreaseFrquency() ;
}
break ;
}
}
}
Simple do not use EXTI for mechanical yousticks and buttons.
Use regular interrupt (for example systick) to poll the status of the pins.
We clearly believe this is the normal and expected bouncing of the switch. Mechanically a switch is some piece of metal that when acted on moves that metal from one pole to another, even if they do not resemble a wiper and two poles. The metal that moves will collide and bounce, the electrical connection will show that. The bouncing is often slow enough for a processor to get multiple interrupts, although that may be an under-sampling of all the bounces possibly seen electrically. If you try to look at it on a scope the scope itself may not-intentionally be filtering some of it (but so will your chip).
One way to see the problem is as with anything, research first then write the application later. This is not a solution but a way to characterize the problem for your system
switch_isr ( void )
{
...
some_global_variable <<= 1;
some_global_variable |= (pin_state_register>>pin_number)&1;
...
}
main ( void )
{
...
some_local_variable = 0;
while(1)
{
if(some_local_variable != some_global_variable)
{
some_local_variable = some_global_variable;
primitive_hex_print(some_local_variable);
}
}
}
No reason to expect to see every state change in the shifted variable, but you should see some and get a feel for the problem. Another way is to just have a counter increment on every interrupt, print periodically in the foreground and you will see one button press may result in multiple counts. And from the time it takes for the printouts to stop changing roughly in human time the settling time.
Filtering is all about state changes per unit time though and you have to have some flavor of time, be it a loop in the foreground that polls some information set by the interrupt (up/down counters, etc), or state changes relative to a timer/clock.
I do not know what the complete rules are for your assignment, if you can only have an interrupt for each switch and not a timer, or preferably a timer instead, I do not see a clean solution that will actually work. You would have to filter in the foreground but all that is doing is polling a copy of the pin state collected by the interrupt and is that any different than not using the interrupt? You cannot use Clifford's answer if you cannot set a timer interrupt, if you could use a timer and an interrupt then you could just periodically sample the switch states with that interrupt or a copy of the pin state collected by the pin state change interrupts and filter in the timer interrupt. Not the same as Clifford's but in all cases you need state change history relative to time to see when the thing settles.
Without a time reference and states not changing with respect to time (which a pin interrupt cannot show since the state has not changed) you cannot filter out the bounces. Instead work on your dexterity and how you flick the joystick up and down.

Switching the GPIO of a Raspberry Pi at a fixed frequency

I have a 208 - 232 Bit long binary signal that I want to send via a GPIO of the raspberry pi.
The delay between the bits needs to be constant.
How can I achieve this ?
The simplest solution that came to my mind was this (pseudocode):
send(gpio, message, delay){
for(int i = 0; i < lenght(message); i++){
if (message[i] == 1){
gpio.high()
}
else{
gpio.low()
}
sleep(delay)
}
}
But the frequency at which I want to send this message is around 40kHz so the delay between two bits is only 25us.
How can I assure it is exactly&constantly that much delay.
In userspace there is no way to garantee that anything is happening in "real time".
Because of this I decided using a seperate Microcontroller that is responsible for time critical features that communicates via I2C or UART with the RaspberryPi.
This way the RaspberryPi can do high abstraction level decisions and show animation to a user while being able to send messages over ultrasound.
Another way I came up with is to create a file similar to .wav and then use DMA or other techniques like with audio. But as I haven't tried it I don't know if the Pi is sufficient to do this at higher Samplingrates.

Problem with programming a basic hardware

I have an animation shown on LEDs. When the button is pressed, the animation has to stop and then continue after the button is pressed again.
There is a method that processes working with the button:
void checkButton(){
GPIO_PinState state;
state = HAL_GPIO_ReadPin(GPIOC, GPIO_PIN_15);
if (state == GPIO_PIN_RESET) {
while(1){
state = HAL_GPIO_ReadPin(GPIOC, GPIO_PIN_15);
if (state == GPIO_PIN_SET){
break;
}
}
//while (state == GPIO_PIN_RESET) {
//state = HAL_GPIO_ReadPin(GPIOC, GPIO_PIN_15);
//}
}
}
GPIO_PIN_SET is the default button position. GPIO_PIN_RESET is the condition when the button is pressed. The commented section is what I tried instead of the while(1){...} loop. The checkButton() method is called in the main loop from time to time to be run. The program runs on STM32 with an extension module (here the type of an extension module does not matter).
The fact is that this method stops animation just for a moment and does not work as I would like it to. Could you correct anything about this program to make it work properly?
Could you correct anything about this program to make it work
properly?
My guess is that you are trying to add a 'human interaction' aspect to your design. Your current approach relies on a single (button position) sample randomly timed by a) your application and b) a human finger. This timing is simply not reliable, but the correction is possibly not too difficult.
Note 1: A 'simple' mechanical button will 'bounce' during it's activation or release (yes, either way). This means that the value which the software 'sees' (in a few microseconds) is unpredictable for several (tbd) milliseconds(?) near the button push or release.
Note 2: Another way to look at this issue, is that your state value exists two places: in the physical button AND in the variable "GPIO_PinState state;". IMHO, a state value can only reside in one location. Two locations is always a mistake.
The solution, then (if you believe) is to decide to keep one state 'record', and eliminate the other. IMHO, I think you want to keep the button, which seems to be your human input. To be clear, you want to eliminate the variable "GPIO_PinState state;"
This line:
state = HAL_GPIO_ReadPin(GPIOC, GPIO_PIN_15);
samples the switch state one time.
HOWEVER, you already know that this design can not rely on the one read being correct. After all, your user might have just pressed or released the button, and it is simply bouncing at the time of the sample.
Before we get to accumulating samples, you should be aware that the bouncing can last much more than a few microseconds. I've seen some switches bounce up to 10 milliseconds or more. If test equipment is available, I would hook it up and take a look at the characteristics of your button. If not, well, you can try the adjusting the controls of the following sample accumulator.
So, how do we 'accumulate' enough samples to feel confident we can know the state of the switch?
Consider multiple samples, spaced-in-time by short delays (2 controls?). I think you can simply accumulate them. The first count to reach tbr - 5 (or 10 or 100?) samples wins. So spin sample, delay, and increment one of two counters:
stateCount [2] = {0,0}; // state is either set or reset, init both to 0
// vvv-------max samples
for (int i=0; i<100; ++i) // worst case how long does your switch bounce
{
int sample = HAL_GPIO_ReadPin(GPIOC, GPIO_PIN_15); // capture 1 sample
stateCount[sample] += 1; // increment based on sample
// if 'enough' samples are the same, kick out early
// v ---- how long does your switch bounce
if (stateCount[sample] > 5) break; // 5 or 10 or 100 ms
// to-be-determined --------vvv --- how long does switch bounce
std::this_thread::sleep_for(1ms); // 1, 3, 5 or 11 ms between samples
// C++ provides, but use what is available for your system
// and balanced with the needs of your app
}
FYI - The above scheme has 3 adjustments to handle different switch-bounce durations ... You have some experimenting to do. I would start with max samples at 20. I have no recommendation for sleep_for ... you provided no other info about your system.
Good luck.
It has been a long time, but I think I remember the push-buttons on a telecom infrastructure equipment bounced 5 to 15 ms.

Effective way of reducing data for real-time plot

I am developing scientific application in Windows Forms (VC++ 2010), which controls relatively new, electronic device. I control it by additional, wrapped library written in C. After initial setup of all parameters, this application triggers a measurement in the device. Then, it sends to my app a huge data of over 200k samples of int at significant rate – let’s assume it’s 50 datasets per second.
Now, I need to plot my data at the real-time pace using Windows Forms chart. It would be perfect to have 750 samples plotted inside chart at rate of about 30 FPS. The problem I encountered lies in the algorithm of reducing database in a fast way without losing reliability of plot.
My ideas (data is oscilating around value = 127):
Choose 750 points just by selecting every (200 000/ 750) th point
Group the data and calculate mean value
Group the data and select maximum or minimum (based on overall group placement – if most of them is above 127 – select minimum, else maximum).
Which one (if any) of those solution is the best considering I have to plot data at real-time speed and plot should not miss spots, where we have any significant signal (looking like a kind of narrowed, modulated sine wave)? Is there any better approach?
And the last question: should I consider using table of pointers to my huge data buffer or data copies as data for plot considering I always have the same buffer of collected data (device just overwrites this buffer constantly with new data)?
This is my first post, so please inform me if there will be anything wrong in the style of post.
I developed an application that reads data at 256Hz (256 samples / second) from 16 channels and displays it in 16 different charts. The best way of plotting all data in real time was using a separate thread to updoat the plots. Here is the solution (in c#) that might be useful for you too.
When new data is read, data is stored in a list or array. Since it is real-time data, the timestamps are also generated here. Using the sample rate of the data acquired: timeStamp = timeStamp + sampleIdx/sampleRate;
public void OnDataRead(object source, EEGEventArgs e)
{
if ((e.rawData.Length > 0) && (!_shouldStop))
{
lock (_bufferRawData)
{
for (int sampleIdx = 0; sampleIdx < e.rawData.Length; sampleIdx++)
{
// Append data
_bufferRawData.Add(e.rawData[sampleIdx]);
// Calculate corresponding timestamp
secondsToAdd = (float) sampleIdx/e.sampleRate;
// Append corresponding timestamp
_bufferXValues.Add( e.timeStamp.AddSeconds(secondsToAdd));
}
}
Then, create a thread that sleeps every N ms (100ms is suitable for me for a 2 seconds display of data, but if I wanna display 10 seconds, I need to increase to 500ms of sleep time for the thread)
//Create thread
//define a thread to add values into chart
ThreadStart addDataThreadObj = new ThreadStart(AddDataThreadLoop);
_addDataRunner = new Thread(addDataThreadObj);
addDataDel += new AddDataDelegate(AddData);
//Start thread
_addDataRunner.Start();
And finally, update the charts and make the thread sleep every N ms
private void AddDataThreadLoop()
{
while (!_shouldStop)
{
chChannels[1].Invoke(addDataDel);
// Sleeep thread for 100ms
Thread.Sleep(100);
}
}
Data will be added to the chart every 100ms
private void AddData()
{
// Copy data stored in lists to arrays
float[] rawData;
DateTime[] xValues;
if (_bufferRawData.Count > 0)
{
// Copy buffered data in thread-safe manner
lock (_bufferRawData)
{
rawData = _bufferRawData.ToArray();
_bufferRawData.Clear();
xValues = _bufferXValues.ToArray();
_bufferXValues.Clear();
}
for (int sampleIdx = 0; sampleIdx < rawData.Length; sampleIdx++)
{
foreach (Series ptSeries in chChannels[channelIdx].Series)
// Add new datapoint to the corresponding chart (x, y, chartIndex, seriesIndex)
AddNewPoint(xValues[sampleIdx], rawData[sampleIdx], ptSeries);
}
}
}

running a background process on arduino

I am trying to get my arduino mega to run a function in the background while it is also running a bunch of other functions.
The function that I am trying to run in the background is a function to determine wind speed from an anemometer. The way it processes the data is similar to that of an odometer in that it reads the number of turns that the anemometer makes during a set time period and then takes that number of turns over the time to determine the wind speed. The longer time period that i have it run over the more accurate data i receive as there is more data to average.
The problem that i have is there is a bunch of other data that i am also reading in to the arduino which i would like to be reading in once a second. This one second time interval is too short for me to get accurate wind readings as not enough revolutions are being completed by the anemometer to give high accuracy wind data.
Is there a way to have the wind sensor function run in the background and update a global variable once every 5 seconds or so while the rest of my program is running simultaneously and updating the other data every second.
Here is the code that i have for reading the data from the wind sensor. Every time the wind sensor makes a revolution there is a portion where the signal reads in as 0, otherwise the sensor reads in as a integer larger than 0.
void windmeterturns(){
startime = millis();
endtime = startime + 5000;
windturncounter = 0;
turned = false;
int terminate = startime;
while(terminate <= endtime){
terminate = millis();
windreading = analogRead(windvelocityPin);
if(windreading == 0){
if(turned == true){
windturncounter = windturncounter + 1;
turned = false;
}
}
else if(windreading >= 1){
turned = true;
}
delay(5);
}
}
The rest of the processing of takes place in another function but this is the one that I am currently struggling with. Posting the whole code would not really be reasonable here as it is close to a 1000 lines.
The rest of the functions run with a 1 second delay in the loop but as i have found through trial and error the delay along with the processing of the other functions make it so that the delay is actually longer than a second and it varies based off of what kind of data i am reading in from the other sensors so a 5 loop counter for timing i do not think will work here
Let Interrupts do the work for you.
In short, I recommend using a Timer Interrupt to generate a periodic interrupt that measures the analog reading in the background. Subsequently this can update a static volatile variable.
See my answer here as it is a similar scenario, detailing how to use the timer interrupt. Where you can replace the callback() with your above analogread and increment.
Without seeing how the rest of your code is set up, I would try having windturncounter as a global variable, and add another integer that is iterated every second your main program loops. Then:
// in the main loop
if(iteratorVariable >= 5){
iteratorVariable = 0;
// take your windreading and implement logic here
} else {
iteratorVariable++;
}
I'm not sure how your anemometer stores data or what other challenges you might be facing, so this may not be a 100% solution, but it would allow you to run the logic from your original post every five seconds.