How can I print out the output of an external command by characters or at least by lines?
This code prints it in one block after the command returns.
import subprocess as sub
output, errors = sub.Popen(command, stdout=sub.PIPE, stderr=sub.PIPE, shell=True).communicate()
print output + errors
You can access to the standart output stream as
p = sub.Popen("cmd", stdout=sub.PIPE, stderr=sub.PIPE)
print(p.stdout.readline()) # This read a line
You can perform any operation o the file streams.
When you use readline in a standart output of a proccess the main thread of the app wait for that proccess to write some thing on the ouput. When that process write to the output you program continue.
You must know that before read a line from the proceess you need to call flush() on the stream. Because the streams have a cache time before the real values are written to it.
You can see this post, this is a good explanation of how this work on python
http://www.cyberciti.biz/faq/python-execute-unix-linux-command-examples/
import subprocess
p = subprocess.Popen(cmd, shell=True, stderr=subprocess.PIPE)
while True:
out = p.stderr.read(1)
if out == '' and p.poll() != None:
break
if out != '':
sys.stdout.write(out)
sys.stdout.flush()
It prints out the output of the cmd character by character.
Related
I am having a raspberry pi 4 (RPI) with python 2.7 installed. Within a python script, I am executing a shell script, which flashes a µController (µC) connected to the pi. The shell script prints some stuff and reaches idle after printing "Connecting". Note that the script does not finish at this point!
Now I want to use subprocess (or any other function) to forward me all the prints from the shell script. I do then want to check if the keyphrase "Connecting" has been printed. Besides, I need a timeout if the shell script gets stuck before printing "Connecting".
However, I am quite new to python, thus I dont know how to use the subprocess correctly to be able to retrieve the prints from the shell script and set a timeout for the script as well.
Here is some sort of pseudo code:
output = subprocess.Popen(["./prebuilt/bin/bbb_cc13xx-sbl /dev/ttyACM0 {hexfileName} cc13x2 -e -p -v"], \
stdout=subprocess.PIPE, stderr = subprocess.PIPE, shell = True)
expression_found = False
for i in range(5)
if(output.stdout.find('Expression') != -1):
expression_found = True
break
time.sleep(1)
if(expression_found):
do that..
else:
do this...
Is there an easy way to implement my two needs?
EDIT: Adding the prints to the terminal like os.system() does would be great, too.
Best wishes
Slev1n
I actually found a simple solution, the mistake is to pipe stderr instead of stdout. The first one being empty all/most of the time.
Here is a solution where the prints from the child process where displayed on the terminal in real time and where I was able to search for a keyword within the stdout pipe. I was also able to terminate the child process without errors. I could also add a timeout as well for terminating the child process. The codes was also validated on raspberry pi 4B with python 2.7.
Here the main process:
import subprocess, sys
import time
cmd = "contprint.py"
p = subprocess.Popen( cmd , shell=True,
stdout=subprocess.PIPE,
universal_newlines=True)
startTime = time.time()
maxTime = 8
while True:
currentTime = time.time()
if (currentTime - startTime) > maxTime:
p.terminate()
break
else:
output = p.stdout.readline()
print(output)
keyWord = "EOF"
if keyWord in output:
print("Keyword {} has be found".format(keyWord))
p.terminate()
break
if len(output) == 0:
print("Output is empty")
p.terminate()
break
if p.poll() is not None:
print("p.poll is not none")
p.terminate()
break
And here the child process:
import time, sys
count = 0
while(1):
count += 1
print(count)
try:
sys.stdout.flush()
except:
print("Escape Error!")
time.sleep(0.5)
if(count == 10):
print("EOF")
if(count == 20):
pass`enter code here`
Any comments are welcome.
I have an issue with Rsyslog's 'omprog' module when trying to get it to interact with my python (2.7) code. Rsyslog is supposed to send desired messages to python's stdin, yet it does not receive anything. I wonder if anyone else has had better success with this output module?
Rsyslog.conf
module(load="omprog")
template(name="sshmsg" type="string" string="%msg%")
if ($programname == "myprogram") then {
action(type="omprog"
binary="/usr/sshtrack.py"
template="sshmsg")
}
If I replace the binary with a test shell script containing a line below, it works
test.sh
!#/bin/sh
cat /dev/stdin >> /var/log/ssh2.log
I also tried reading stdin in the shell script into a variable using
var="$(</dev/stdin)"
and
var="$(cat /dev/stdin)"
Neither of the above resulted var containing anything
Finally, when trying to read stdin from python script, I get nothing. Sometimes, it says resource unavailable (errno 11) error message.
sshtrack.py
#!/usr/bin/python
import sys
f = open("/var/log/ssh2.log", "a", 0)
while True:
f.write("Starting\n")
for line in sys.stdin:
f.flush()
msg = line.strip()
if not msg:
break
f.write(msg)
f.write("\n")
f.close()
The issue seems similar to can not read correctly from STDIN except adding a non-block flag did nothing.
I notice that your template sshmsg doesn't end with a newline. Try changing it to string="%msg%\n". Though it won't matter to rsyslog, Python will not be able to give you the data until it sees a newline.
Then it should work, but you probably not see any output from your python as it is buffered. Try adding an f.flush() after the last write in the loop, or opening the file unbuffered.
omprog will keep the pipe open, sending multiple lines until your program exits.
Note, not all shells might understand $() syntax.
In case of your shell script you can use read to read into a variable.
#!/bin/bash
# This will read until \n
read log
echo $log
The python source code (tested with python 3.8.2) can be adjusted to:
#!/usr/bin/env python3
import sys
# Changed from unbuffered to buffered as unbuffered is only possible in binary mode Ref (1):
f = open("/var/log/ssh2.log", "a", 1)
while True:
f.write("Starting\n")
for line in sys.stdin:
f.flush()
msg = line.strip()
if not msg:
break
f.write(msg)
f.write("\n")
f.close()
In case you want to have the output of you executed script (debugging) you can adjust the settings in Rsyslog.conf with the output option
module(load="omprog")
template(name="sshmsg" type="string" string="%msg%")
if ($programname == "myprogram") then {
action(type="omprog"
binary="/usr/sshtrack.py"
output="/var/log/sshtrack.log"
template="sshmsg")
}
Ref (1): https://stackoverflow.com/a/45263101/13108341
Ok so I have code that is supposed to run through a txt file and ping the Ip's if the ping is equal to 0 its does an 'nslookup' on it and then it's supposed to continue but after it does the first one in the terminal it's left on a > as if waiting for input. In other instances, my code runs through the txt file fine but once I added in the 'nslookup' it stops after the first one and waits for input.
Is there a way to make it continue to cycle through the txt file till it gets to the end?
Heres the code I'm using I know there are other ways to do a look up on an Ip address but I'm trying to use 'nslookup' in this case unless its impossible.
import os
with open('test.txt','r') as f:
for line in f:
response = os.system("ping -c 1 " + line)
if response == 0:
print os.system('nslookup')
else:
print(line, "is down!")
that's simply because you forgot to pass the argument to nslookup
When you don't pass any argument, the program starts in interactive mode with its own shell.
L:\so>nslookup
Default server : mydomain.server.com
Address: 128.1.34.82
>
But using os.system won't make you able to get the output of the command. For that you would need
output = subprocess.check_output(['nslookup',line.strip()])
print(output) # or do something else with it
instead of your os.system command
I want to run command line in python, and capture the output. I can use subprocess.check_output. But it will suppress the output, how can i do it without suppressing the console output?
How about this?
from subprocess import Popen, PIPE
proc = Popen(["/usr/bin/nc", "-l", "9999"], stdout=PIPE)
buffer = []
line = proc.stdout.readline()
while line:
buffer.append(line)
print "LINE", line.strip()
line = proc.stdout.readline()
print "buffer", ''.join(buffer)
Using another terminal send some text
nc localhost 9999
# type something. the text should appear from the python code
Break the nc, and you get the output in buffer as well
I have a list of commands saved in text file ('command.log') which I want to run against a unit connected to 'COM5' and save the response for each command in a text file ('output.log'). The script gets stuck on the first command and I could get it to run the remaining commands. Any help will be appreciated.
import serial
def cu():
ser = serial.Serial(
port='COM5',
timeout=None,
baudrate=115200,
parity='N',
stopbits=1,
bytesize=8
)
ser.flushInput()
ser.flushOutput()
## ser.write('/sl/app_config/status \r \n') #sample command
fw = open('output.log','w')
with open('data\command.log') as f:
for line in f:
ser.write(line + '\r\n')
out = ''
while out != '/>':
out += ser.readline()
fw.write(out)
print(out)
fw.close()
ser.close()
print "Finished ... "
cu()
The bytes problem
First of all, you're misusing the serial.readline function: it returns a bytes object, and you act like it was a str object, by doing out += ser.readline(): a TypeError will be raised. Instead, you must write out += str(ser.readline(), 'utf-8'), which first converts the bytes into a str.
How to check when the transmission is ended ?
Now, the problem lays in the out != '/>' condition: I think you want to test if the message sent by the device is finished, and this message ends with '/<'. But, in the while loop, you do out += [...], so in the end of the message, out is like '<here the message>/>', which is totally different from '/>'. However, you're lucky: there is the str.endswith function! So, you must replace while out != '\>' by while not out.endswith('\>'.
WWhatWhat'sWhat's theWhat's the f*** ?
Also, in your loop, you write the whole message, if it's not already ended, in each turn. This will give you, in output.log, something like <<me<mess<messag<messag<message>/>. Instead, I think you want to print only the received characters. This can be achieved using a temporary variable.
Another issue
And, you're using the serial.readline function: accordingly to the docstrings,
The line terminator is always b'\n'
It's not compatible with you're code: you want your out to finish with "\>", instead, you must use only serial.read, which returns all the received characters.
Haaaa... the end ! \o/
Finally, your while loop will look as follows:
# Check if the message is already finished
while not out.endswith('/>'):
# Save the last received characters
# A `bytes` object is encoded in 'utf-8'
received_chars = str(ser.read(), 'utf-8')
# Add them to `out`
out += received_chars
# Log them
fw.write(received_chars)
# Print them, without ending with a new line, more "user-friendly"
print(received_chars, end='')
# Finally, print a new line for clarity
print()