Slow Serial Connection between Arduino and Python - python-2.7

I currently have a project where my goal is to graph (in real time) the air flow going through an air flow sensor. The sensor is connected via IIC to an Arduino Uno, which is then collects the data and sends it to my COM3 Serial line with a baudrate of 1,000,000. In python, I read, line by line, the Serial line, and then graph the data. My goal is to graph about 500 data points/second (500hz), but it seems that Python is reading in the data too slowly.
Here is a small 'test' I performed:
I blew on the air flow sensor. From the Serial Monitor on my Arduino program, I immediately see the readings spike. However, from Python's console output (which is just reading the serial line and printing out the contents), the spike from my blowing is significantly delayed. The longer the program runs, the more backed up Python gets with keeping up with the Serial line.
From the Arduino side:
Serial.println((double)average, 4);
//Serial.print(" , ");
// Serial.println((double)Flow, 4);
delay(2);
}
And on the Python side:
print("PRESS 's' TO BEGIN RECORDING DATA!")
while not pressedStart:
if keyboard.is_pressed('s'):
print("Timer and Program Started!")
startTime = hour.time()
pressedStart = True
while pressedStart: #Program begins reading Serial Line when 's' is pressed.
while flowData.inWaiting()==0:
pass
dataArray = flowData.readline().split(",")
currentAverage = float(dataArray[0])
averageArray.append(currentAverage)
print(currentAverage) #this line is as far as my question goes. Why does Python print out this number
#more slowly than Arduino sends it out????

Python is a scripting language. It runs through an interpreter and therefore runs very slowly. Although and Arduino micro-controller is very under-powered, it runs faster than your python because python is that much slower. Unless you are willing to have less frequent data points by increasing the Arduino delay, I don't think this can be solved. Good luck still!

Related

QSerialPort continuous reading accumulative delay

I am trying to do communication from QT Application to Arduino. The flow is like this: QT Application sends a '1' and Arduino is expected to respond with some data(the data String length is huge, around 300). QT Application is sending '1' at the rate of around 5Hz(every 200ms).
The problem I am facing is, there is an accumulative delay between the Arduino to QT communication. That is, the data I receive from Arduino is not recent data but the frequency of data coming of Arduino is 5Hz only(which is as expected), just the data coming is not recent. This delay keeps on increasing with time. I believe there is some problem with buffer or something.
What I tried:
QSerialPort serialPort; is my device port
serialPort.clear()
serialPort.flush()
Increasing and decreasing Baud Rate from both ends.
Reduce character length from Arduino, here delay reduces significantly but the accumulated delay is observed after a long time.
to clear serial communication buffer, but the issue still persists.
Here is my code snippet:
connect(timer_getdat, SIGNAL(timeout()), this, SLOT(Rec()));
timer_getdat->start(200);
where Rec() is the function where I do communication part.
In Rec():
serialPort.write("1", 2);
// serialPort.waitForBytesWritten(100);
long long bytes_available = serialPort.bytesAvailable();
if (bytes_available >= 1)
{
serialPort.readLine(temp, 500);
serialPort.flush(); // no change
serialPort.clear(); // no change by .clear() also
}
I have been stuck on this issue for a quite long time. The above code snippet is what I think is necessary but if anyone needs more clarification, I may reveal more of the code.
I also encountered with the same issue, and yes QSerialPort.clear() and QSerialPort.flush() doesn't help. Try doing readAll()
So change the part in your Rec() function to something like this:
serialPort.write("1", 2);
long long bytes_available = serialPort.bytesAvailable();
if (bytes_available >= 1)
{
serialPort.readLine(temp, 500);
serialPort.readAll(); // This reads all the data in buffer at once and clears the queue.
}
Even on QT forums, I didn't find the answer to this, was playing with all functions available with QSerialPort class and readAll() seems to work.
About readAll(), Qt documentation says:
Reads all remaining data from the device, and returns it as a byte
array.
My explanation for the resolution is that readAll captures all of the data from the communication buffer and empties it.
This should be the job of clear() function but apparently readAll() seems to work.

Pyserial/Eye-tracker: Reading from serial port whilst displaying stimuli

I am using Pyserial (+Python 2.7) to read in eye-tracking coordinates taken from an eye-tracker (CRS Live-Track set as serial port). Using the code below I am able to successfully start the tracker, read/save a line of coordinates, stop the tracker, and close. My problem is I need to continuously read in the coordinates whilst carrying out other tasks such as stimuli display. As it stands, I can't do anything whilst I'm reading in data ('trial1 = ser.readline'). I have to wait until I've read in the data before I continue. Is there a way to read in the data from the serial port continuously whilst I am displaying stimuli/collecting responses etc.?
I need to turn the tracker on, collect the data for the duration of the trial, and then tracker off.
import serial, time
ser = serial.Serial(
port='COM3',
baudrate=9600,
parity=serial.PARITY_ODD,
stopbits=serial.STOPBITS_TWO,
bytesize=serial.SEVENBITS,
)
x = ser.is_open #check port is open
if x:
print "port is open"
print "port name is: %s" %(ser.name) #check which port
ser.flushInput()
ser.flushOutput()
running = True
while running:
ser.write('$Raw\r') #start eye-tracker
trial1 = ser.readline() #read a line
###i need to do stuff here###
ser.write('$Stop\r') #stop eye-tracker
running = False
ser.flushInput()
ser.flushOutput()
print trial1 #print coordinates output
ser.close()
Cheers,
Steve
ioHub by Sol Simpson provides asynchronous device monitoring from within PsychoPy. i.e it operates on an entirely separate process so you don't have to pause while monitoring the port in the main PsychoPy thread.
Serial port documentation in ioHub seems scarce but there is a demo here: https://github.com/psychopy/psychopy/blob/master/psychopy/demos/coder/iohub/serial/customparser.py
Another option would be to try threads. You could put your eye tracker in one thread and the rest of your code in another. There is a stack overflow answer about multithreading (not using psychopy and eye trackers) from 2010. If you scroll down you will see that someone links to a blog post about an updated approach that they think is easier.

Why Serial Communications from Arduino stops after some time

I'm building a growbox/terrarium with arduino uno as the temperature controller. Simple sketch for arduino: if DS18B20 sensor giv less than 25'C than turn on relay, which the heating cable is connected to. Loop 30s, every time Serial.print(temperature) to the PC, where I'm collecting data and make timelapse photos. ---> Here is the problem.
After some time (from 15 min up to 4 hours). Serial communication with PC stops. When I'm trying to upload a new sketch to arduino I got an error msg:
avrdude: ser_open(): can't set com-state for "\.\COM3"
I need to unplug and plug in again USB cable (or turn it off and on in Windows Device Manager), also restart Python app that is collecting data. (Very unsatisfactory fix).
So my questions are:
1. Why?
2. How to fix it?
3. Or how to make some workaround, for example reset COM port from code (desirably python2.7)
PS. Example of what I'm doing and how it works(and do not works) here: Life of pepper
PS2. My goal is to make controllable habitate for plants, where I can see behaviours differs depending on the temp, day-duration, light-intensity, humidity.
Please help me :-) .
ARDUINO UNO SKETCH
#include <OneWire.h>
#include <DallasTemperature.h>
// Data wire is plugged into port 2 on the A
rduino
#define ONE_WIRE_BUS 2
#define PIN_HEATING 6
float temperatura = 30.0;
// Setup a oneWire instance to communicate with any OneWire devices (not just Maxim/Dallas temperature ICs)
OneWire oneWire(ONE_WIRE_BUS);
// Pass our oneWire reference to Dallas Temperature.
DallasTemperature sensors(&oneWire);
void setup(void)
{
// start serial port
Serial.begin(9600);
// Start up the library
sensors.begin();
pinMode(PIN_HEATING, OUTPUT);
}
void loop(void)
{
delay(30000);
if(temperatura<25.0){
digitalWrite(PIN_HEATING, HIGH);
}
else{
digitalWrite(PIN_HEATING, LOW);
}
sensors.requestTemperatures(); // Send the command to get temperatures
// After we got the temperatures, we can print them here.
// We use the function ByIndex, and as an example get the temperature from the first sensor only.
temperatura = sensors.getTempCByIndex(0);
Serial.print("#");
Serial.print(temperatura);
}
PYTHON2.7 RECEIVING PART
import serial #pySerial
ser = serial.Serial()
ser.port = "COM3"
ser.open()
data = ser.read_all().split("#")
datasize = len(data)
if datasize>1:
temp = data[datasize-1]
tempstr = " / " + str(temp) + "'C"
else:
tempstr=" / ----- "
I've experienced a similar problem in the past where the serial port would become unresponsive. The solution in my case wasn't very intuitive but worked nonetheless. Here's what might help you too:
Make sure the USB port you're using to connect the arduino isn't shared with other devices (for example using a USB multipoint hub). If you're using a desktop with multiple USB ports (usually found at the back of the desktop case), unplug all other USB devices (keyboard, mouse etc) and reconnect them to a different USB. Use the fastest USB port available available on your computer (USB 3.0 if present) and have it dedicated for communication with the arduino. The reason the communication port goes unresponsive sometimes is because of the momentary fluctuations in the voltage and/or current if you have several devices connected to it.
Make sure you have a stable power supply for you computer, voltage drifts due to heavy loading can sometimes cause PC peripherals to disconnect/reconnect.
If this doesn't work, try this quick software fix. Just before the delay(30000) statement in your code, add the following:
if(!Serial) { //check if Serial is available... if not,
Serial.end(); // close serial port
delay(100); //wait 100 millis
Serial.begin(9600); // reenable serial again
}
This might at least ensure continued operation operation of your application with out requiring your intervention.
So in my case the problem was solved by connecting my computer through proper surge supressor, formerly i had connectected it to wall with a simple cable, and I have old 2 wire, non-grounded, electric installation in my flat.

Android usb host input bulktransfer fails to read randomly when data available

The following code is inside a thread and reads input data coming over usb. Approximately every 80 readings it misses one of the packets coming from an stm32 board. The board is programmed to send data packets to the android tablet every one second.
// Non Working Code
while(running){
int resp = bulktransfer(mInEp,mBuf,mBuf.lenght,1000);
if(resp>0){
dispatchMessage(mBuf);
}else if(resp<0)
showsBufferEmptyMessage();
}
I was looking the Missile Launcher example in android an other libraries on the internet and they put a delay of 50ms between each poll. Doing this it solves the missing package problem.
//Working code
while(running){
int resp = bulktransfer(mInEp,mBuf,mBuf.lenght,1000);
if(resp>0){
dispatchMessage(mBuf);
}else if(resp<0)
showsBufferEmptyMessage();
try{
Thread.sleep(50);
}catch(Exception e){}
}
Does anyone knows the reason why the delay works. Most of the libraries on github has this delay an as I mention before the google example too.
I am putting down my results regarding this problem. After all seems that the UsbConnection.bulkTransfer(...) method has some bug when called continuously. The solution was to use the asynchronous API, UsbRequest class. Using this method I could read from the input endpoint without delay and no data was lost during the whole stress test. So the direction to take is asynchronous UsbRequest instead of synchronously bulktransfer.

How to read the temperature from sensors on the motherboard?

Trying to get the temperature of the processor.
Have already tried using WQL (WMI class MSAcpi_ThermalZoneTemperature), but apparently it is not implemented for all the platforms yet. On most of the machines it simply returns every existing error message via HRESULT return value. The temperature itself is not returned.
The idea is to read this temperature directly via the bus port. I have found the library, which gives the functionality of outp and inp functions, and with it managed to start initiate the connection (NTPort), however, the question becomes which port to connect to in order to read the data. The microcontroller is IT8728F. SpeedFan (the application that is able to read the temperature) in its logs says that it reads the data from port 0x290. However, when connecting to it, the data that comes back does not look like temperature (it always returns 29).
So what the next thing that was tried was to read from whatever port, trying to determine if any data that came through looked like temperature. However, from every port some data came through that was either too low or too high to be the real temperature.
int CPU_TEMP;
Outp(INDEX, BANK_SET);
Outp(DATA, Inp(DATA)|0x01);
for(CPU_TEMP=0x0;CPU_TEMP<0x999;CPU_TEMP++)
{
Outp(INDEX, CPU_TEMP);
printf("CPU temp: %iC\n", Inp(DATA));
}
maybe your processor is 29 degrees Celsius and stable. That seems legit for a processor.
Also: is that 0x00000029 or DEC 29? that makes a big difference