I want to run command line in python, and capture the output. I can use subprocess.check_output. But it will suppress the output, how can i do it without suppressing the console output?
How about this?
from subprocess import Popen, PIPE
proc = Popen(["/usr/bin/nc", "-l", "9999"], stdout=PIPE)
buffer = []
line = proc.stdout.readline()
while line:
buffer.append(line)
print "LINE", line.strip()
line = proc.stdout.readline()
print "buffer", ''.join(buffer)
Using another terminal send some text
nc localhost 9999
# type something. the text should appear from the python code
Break the nc, and you get the output in buffer as well
Related
I have an issue with Rsyslog's 'omprog' module when trying to get it to interact with my python (2.7) code. Rsyslog is supposed to send desired messages to python's stdin, yet it does not receive anything. I wonder if anyone else has had better success with this output module?
Rsyslog.conf
module(load="omprog")
template(name="sshmsg" type="string" string="%msg%")
if ($programname == "myprogram") then {
action(type="omprog"
binary="/usr/sshtrack.py"
template="sshmsg")
}
If I replace the binary with a test shell script containing a line below, it works
test.sh
!#/bin/sh
cat /dev/stdin >> /var/log/ssh2.log
I also tried reading stdin in the shell script into a variable using
var="$(</dev/stdin)"
and
var="$(cat /dev/stdin)"
Neither of the above resulted var containing anything
Finally, when trying to read stdin from python script, I get nothing. Sometimes, it says resource unavailable (errno 11) error message.
sshtrack.py
#!/usr/bin/python
import sys
f = open("/var/log/ssh2.log", "a", 0)
while True:
f.write("Starting\n")
for line in sys.stdin:
f.flush()
msg = line.strip()
if not msg:
break
f.write(msg)
f.write("\n")
f.close()
The issue seems similar to can not read correctly from STDIN except adding a non-block flag did nothing.
I notice that your template sshmsg doesn't end with a newline. Try changing it to string="%msg%\n". Though it won't matter to rsyslog, Python will not be able to give you the data until it sees a newline.
Then it should work, but you probably not see any output from your python as it is buffered. Try adding an f.flush() after the last write in the loop, or opening the file unbuffered.
omprog will keep the pipe open, sending multiple lines until your program exits.
Note, not all shells might understand $() syntax.
In case of your shell script you can use read to read into a variable.
#!/bin/bash
# This will read until \n
read log
echo $log
The python source code (tested with python 3.8.2) can be adjusted to:
#!/usr/bin/env python3
import sys
# Changed from unbuffered to buffered as unbuffered is only possible in binary mode Ref (1):
f = open("/var/log/ssh2.log", "a", 1)
while True:
f.write("Starting\n")
for line in sys.stdin:
f.flush()
msg = line.strip()
if not msg:
break
f.write(msg)
f.write("\n")
f.close()
In case you want to have the output of you executed script (debugging) you can adjust the settings in Rsyslog.conf with the output option
module(load="omprog")
template(name="sshmsg" type="string" string="%msg%")
if ($programname == "myprogram") then {
action(type="omprog"
binary="/usr/sshtrack.py"
output="/var/log/sshtrack.log"
template="sshmsg")
}
Ref (1): https://stackoverflow.com/a/45263101/13108341
I'm trying to write a python script which
1) Compiles a cpp file.
2) Reads a text file "Input.txt" which has to be fed to the cpp file.
3) Compare the output with "Output.txt" file and Print "Pass" if all test cases have passed successfully else print "Fail".
`
import subprocess
from subprocess import PIPE
from subprocess import Popen
subprocess.call(["g++", "eg.cpp"])
inputFile = open("input.txt",'r')
s = inputFile.readlines()
for i in s :
proc = Popen("./a.out", stdin=int(i), stdout=PIPE)
out = proc.communicate()
print(out)
`
For the above code, I'm getting an output like this,
(b'32769', None)
(b'32767', None)
(b'32768', None)
Traceback (most recent call last):
File "/home/zanark/PycharmProjects/TestCase/subprocessEg.py", line 23, in <module>
proc = Popen("./a.out", stdin=int(i), stdout=PIPE)
ValueError: invalid literal for int() with base 10: '\n'
PS :- eg.cpp contains code to increment the number from the "Input.txt" by 2.
pass the string to communicate instead, and open your file as binary (else python 3 won't like it / you'll have to encode your string as bytes):
with open("input.txt",'rb') as input_file:
for i in input_file:
print("Feeding {} to program".format(i.decode("ascii").strip())
proc = Popen("./a.out", stdin=PIPE, stdout=PIPE)
out,err = proc.communicate(input=i)
print(out)
also don't convert input of the file to integer. Leave it as string (I suspect you'll have to filter out blank lines, though)
I have a method in which I can connect to a host and ping it. I can see the ping results on the screen, but cannot write them into a file. The file is always empty. Can anyone help me in this regard?
def Check(self,fname):
client = SSHClient()
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
client.load_system_host_keys()
client.connect(self.ip,22,self.username,self.password)
ping_command = 'ping -n -c 5 '+str(self.ip)
stdin, stdout, stderr = client.exec_command(ping_command)
for line in stdout.readlines():
print line
try:
with open(fname,'ab') as text_file:
text_file.wirte(stdout)
text_file.close()
except IOError:
pass
client.close()
I also tried to store stdout into a variable and then write that variable into the file and didn't get any results. I also found some topics in the site about Paramiko output, and none of them helped.
How can I print out the output of an external command by characters or at least by lines?
This code prints it in one block after the command returns.
import subprocess as sub
output, errors = sub.Popen(command, stdout=sub.PIPE, stderr=sub.PIPE, shell=True).communicate()
print output + errors
You can access to the standart output stream as
p = sub.Popen("cmd", stdout=sub.PIPE, stderr=sub.PIPE)
print(p.stdout.readline()) # This read a line
You can perform any operation o the file streams.
When you use readline in a standart output of a proccess the main thread of the app wait for that proccess to write some thing on the ouput. When that process write to the output you program continue.
You must know that before read a line from the proceess you need to call flush() on the stream. Because the streams have a cache time before the real values are written to it.
You can see this post, this is a good explanation of how this work on python
http://www.cyberciti.biz/faq/python-execute-unix-linux-command-examples/
import subprocess
p = subprocess.Popen(cmd, shell=True, stderr=subprocess.PIPE)
while True:
out = p.stderr.read(1)
if out == '' and p.poll() != None:
break
if out != '':
sys.stdout.write(out)
sys.stdout.flush()
It prints out the output of the cmd character by character.
I have a python script that want to ping a few (quite a few!) hosts. I have set this up to read the contents of a hosts.txt file as the hosts to ping in the script. The odd thing is that I am recieving the following error, for the first few addresses (no matter what they are):
Ping request could not find host 66.211.181.182. Please check the name and try again.
I have included the address shown above at twice (within the file) and it attempts a ping. Any thoughts on what I am doing wrong - I am a python newbie, so be gentle.
Here is my script:
import subprocess
hosts_file = open("hosts.txt","r")
lines = hosts_file.readlines()
for line in lines:
ping = subprocess.Popen(
["ping", "-n", "1",line],
stdout = subprocess.PIPE,
stderr = subprocess.PIPE
)
out, error = ping.communicate()
print out
print error
hosts_file.close()
Here is my hosts.txt file:
66.211.181.182
178.236.5.39
173.194.67.94
66.211.181.182
And here are the results from the above test:
Ping request could not find host 66.211.181.182
. Please check the name and try again.
Ping request could not find host 178.236.5.39
. Please check the name and try again.
Ping request could not find host 173.194.67.94
. Please check the name and try again.
Pinging 66.211.181.182 with 32 bytes of data:
Request timed out.
Ping statistics for 66.211.181.182:
Packets: Sent = 1, Received = 0, Lost = 1 (100% loss)
Looks like the line variable contains a linebreak at the end (except for the last line of the file). From the Python tutorial:
f.readline() reads a single line from the file; a newline character (\n) is left at the end of the string, and is only omitted on the last line of the file if the file doesn’t end in a newline.
You need to strip the \n before calling Popen: How can I remove (chomp) a newline in Python?
Few comments:
Using readlines() is highly not recommended as it will load the entire file into memory.
I suggest using Generator in order to perform rstrip on each line and then pinging the server.
No need to use file.close - you can use with statement that does it for you
Your code should look like this:
import subprocess
def PingHostName(hostname):
ping=subprocess.Popen(["ping","-n","1",hostname],stdout=subprocess.PIPE
,stderr=subprocess.PIPE)
out,err=ping.communicate();
print out
if err is not None: print err
with open('C:\\myfile.txt') as f:
striped_lines=(line.rstrip() for line in f)
for x in striped_lines: PingHostName(x)