I created two simple scripts:
script.py:
import time
import sys
import signal
try:
print('i am running')
time.sleep(10)
print('i am done')
except KeyboardInterrupt:
print("you don't like me??")
and test.py:
import subprocess
import signal
from threading import Thread
import time
import os
p = subprocess.Popen('python script.py', shell=True)
t = Thread(target=p.wait)
t.start()
print('sleeping')
time.sleep(2)
print('interrupt')
p.send_signal(signal.SIGINT)
#p.send_signal(signal.SIGTERM)
t.join()
print('process finished')
If I run test.py (on ubuntu) the expected result would be:
sleeping
i am running
interrupt
you don't like me??
process finished
instead the SIGINT seems to be ignored:
sleeping
i am running
interrupt
i am done
process finished
SIGTERM terminates the process as anticipated. However no KeyboardInterrupt is raised.
Even if I add the following lines to script.py
def signal_handler(signal, frame):
print('You pressed Ctrl+C!')
signal.signal(signal.SIGINT, signal_handler)
no SIGINT seems to be received.
However, when I press C+CTRL myself a SIGINT is received. But that's not an option for me since the SIGINT must be time triggered.
Does anybody have a clue why this happens?
Cheers,
Thomas
(I've removed the use of threading in my examples because it doesn't add anything to the example other than more lines of code)
This is to do with how signals are handled in process groups, you might find this other SO answer answer helpful.
import subprocess
import signal
import time
import os
p = subprocess.Popen('python script.py', shell=True, preexec_fn=os.setsid)
print('sleeping')
time.sleep(2)
os.killpg(os.getpgid(p.pid), signal.SIGINT)
print('interrupt')
p.wait()
print('process finished')
This yields the expected result:
andy#batman[14:58:04]:~/so$ python test.py
sleeping
i am running
interrupt
you don't like me??
process finished
Signals are handled by process groups, so sending one from a process within the process group doesn't work like you think.
Interestingly, if you don't use shell=True (which you shouldn't use if you can avoid it), it works just fine.
import subprocess
import signal
import time
import os
p = subprocess.Popen(['python', 'script.py'])
print('sleeping')
time.sleep(2)
p.send_signal(signal.SIGINT)
print('interrupt')
p.wait()
print('process finished')
So if I'm honest this answer is a little bit crap because I can show you two things which ostensibly work, but not really explain why.
Related
Correct code as expected:
from multiprocessing import Pool
import signal
import time
import os
def consumer(i):
while True:
# print os.getpid()
pass
def handler(signum, frame):
print 'Here you go'
signal.signal(signal.SIGTERM, handler)
p = Pool(5)
p.map_async(consumer, [1 for i in range(5)])
while True:
pass
p.terminate()
# p.close()
p.join()
==================================================
I have found the problem, when I use map function, the main func is blocked, and signal handler will be called only when map function is funished.
So, use "map_async" function is much more better to fix this problem.
Here is what I found:
A long-running calculation implemented purely in C (such as regular expression matching on a large body of text) may run uninterrupted for an arbitrary amount of time, regardless of any signals received. The Python signal handlers will be called when the calculation finishes.
==================================================
I wrote a program like the following, and I expect to exit/(like the program print string) in the program when I type "kill pid" in the terminal, but it not work. Is there any other strategy that block the SIGTERM get in the main func.
from multiprocessing import Pool
import signal
import time
import os
def consumer(i):
while True:
# print os.getpid()
pass
def handler(signum, frame):
print 'Here you go'
signal.signal(signal.SIGTERM, handler)
p = Pool(5)
p.map(consumer, [1 for i in range(5)])
p.terminate()
# p.close()
p.join()
You need to use the map_async method as the map one blocks until results are not ready. Therefore, in your example the call to terminate is never reached.
I posted a question about how to catch a "sudo shutdown -r 2" event in Python. I was sent to this thread: Run code in python script on shutdown signal .
I'm running a Raspberry Pi v2 with Jessy.
I have read about
signal
and have tried to follow the ideas in the above thread, but so far I have not been successful. Here is my code:
import time
import signal
import sys
def CloseAll(Code, Frame):
f = open('/mnt/usbdrive/output/TestSignal.txt','a')
f.write('Signal Code:' + Code)
f.write('Signal Frame:' + Frame)
f.write('\r\n')
f.close()
sys.exit(0)
signal.signal(signal.SIGTERM,CloseAll)
print('Program is running')
try:
while True:
#get readings from sensors every 15 seconds
time.sleep(15)
f = open('/mnt/usbdrive/output/TestSignal.txt','a')
f.write('Hello ')
f.write('\r\n')
f.close()
except KeyboardInterrupt:
f = open('/mnt/usbdrive/output/TestSignal.txt','a')
f.write('Done')
f.write('\r\n')
f.close()
The program runs in a "screen" session/window and reacts as expected to a CNTL-C. However, when I exit the screen session, leaving the program running, and enter "sudo shutdown -r 2", the Pi reboots as expected after 2 minutes, but the TestSignal.txt file does not show that the signal.SIGTERM event was processed.
What am I doing wrong? Or better yet, how can I trap the shutdown event, usually initiated by a cron job, and close my Python program running in a screen session gracefully?
When you do not try to await such an event, but in a parallel session send SIGTERMto that process (e.g. by calling kill -15 $PID on the process id $PID of the python script running) , you should see an instructive error message ;-)
Also the comment about the mount point should be of interest after you repaired the python errors (TypeError: cannot concatenate 'str' and 'int' objects).
Try something like:
import time
import signal
import sys
LOG_PATH = '/mnt/usbdrive/output/TestSignal.txt'
def CloseAll(Code, Frame):
f = open(LOG_PATH, 'a')
f.write('Signal Code:' + str(Code) + ' ')
f.write('Signal Frame:' + str(Frame))
f.write('\r\n')
f.close()
sys.exit(0)
signal.signal(signal.SIGTERM, CloseAll)
print('Program is running')
try:
while True:
# get readings from sensors every 15 seconds
time.sleep(15)
f = open(LOG_PATH, 'a')
f.write('Hello ')
f.write('\r\n')
f.close()
except KeyboardInterrupt:
f = open(LOG_PATH, 'a')
f.write('Done')
f.write('\r\n')
f.close()
as a starting point. If this works somehow on your system why not rewrite some portions like:
# ... 8< - - -
def close_all(signum, frame):
with open(LOG_PATH, 'a') as f:
f.write('Signal Code:%d Signal Frame:%s\r\n' % (signum, frame))
sys.exit(0)
signal.signal(signal.SIGTERM, close_all)
# 8< - - - ...
Edit: To further isolate the error and adapt more to production like mode, one might rewrite the code like this (given that syslog is running on the machine, which it should, but I never worked on devices of that kind):
#! /usr/bin/env python
import datetime as dt
import time
import signal
import sys
import syslog
LOG_PATH = 'foobarbaz.log' # '/mnt/usbdrive/output/TestSignal.txt'
def close_all(signum, frame):
"""Log to system log. Do not spend too much time after receipt of TERM."""
syslog.syslog(syslog.LOG_CRIT, 'Signal Number:%d {%s}' % (signum, frame))
sys.exit(0)
# register handler for SIGTERM(15) signal
signal.signal(signal.SIGTERM, close_all)
def get_sensor_readings_every(seconds):
"""Mock for sensor readings every seconds seconds."""
time.sleep(seconds)
return dt.datetime.now()
def main():
"""Main loop - maybe check usage patterns for file resources."""
syslog.syslog(syslog.LOG_USER, 'Program %s is running' % (__file__,))
try:
with open(LOG_PATH, 'a') as f:
while True:
f.write('Hello at %s\r\n' % (
get_sensor_readings_every(15),))
except KeyboardInterrupt:
with open(LOG_PATH, 'a') as f:
f.write('Done at %s\r\n' % (dt.datetime.now(),))
if __name__ == '__main__':
sys.exit(main())
Points to note:
the log file for the actual measurements is separate from the logging channel for operational alerts
the log file handle is safeguarded in context managing blocks and in usual operation is just kept open
for alerting the syslog channel is used.
as a sample for the message routing the syslog.LOG_USER on my system (OS X) gives me in all terminals a message, whilst the syslog.LOG_ERR priority message in signal handler only targets the system log.
should be more to the point during shutdown hassle (not opening a file, etc.)
The last point (5.) is important in case all processes receive a SIGTERM during shutdown, i.e. all want to do something (slowing things down), maybe screenalso does not accept any buffered input anymore (or does not flush), note stdout is block buffered not line buffered.
The decoupling of the output channels, should also ease the eventual disappearance of the mount point of the measurement log file.
import threading
import sys
from time import time
def stop():
print "stop"
sys.exit()
t = threading.Timer(10, stop)
volts = 22
if volts > 20:
t.start()
print "Start"
After 10 seconds it prints Stop, which is fine, but it ignores the sys.exit(). I need to do a sys.exit when the timer expires.
When you run
sys.exit()
inside of a thread, it raises the SystemExit exception. When you call thread.exit(), it raises the same exception, so you are only exiting your thread, not your program.
However in the case of the code you provided, exiting your thread will cause your program to finish as well.
I am writing some code that makes use of the multiprocessing module. However, since I am a newbie, what often happens is that some error pops up, putting a halt to the main application.
However, that applications' children still remain running, and I get a long, long list of running pythonw processes in my task manager list.
After an error occurs, what can I do to make sure all the child processes are killed as well?
There are two pieces to this puzzle.
How can I detect and kill all the child processes?
How can I make a best effort to ensure my code from part 1 is run whenever one process dies?
For part 1, you can use multiprocessing.active_children() to get a list of all the active children and kill them with Process.terminate(). Note the use of Process.terminate() comes with the usual warnings.
from multiprocessing import Process
import multiprocessing
def f(name):
print 'hello', name
while True: pass
if __name__ == '__main__':
for i in xrange(5):
p = Process(target=f, args=('bob',))
p.start()
# At user input, terminate all processes.
raw_input("Press Enter to terminate: ")
for p in multiprocessing.active_children():
p.terminate()
One solution to part 2 is to use sys.excepthook, as described in this answer. Here is a combined example.
from multiprocessing import Process
import multiprocessing
import sys
from time import sleep
def f(name):
print 'hello', name
while True: pass
def myexcepthook(exctype, value, traceback):
for p in multiprocessing.active_children():
p.terminate()
if __name__ == '__main__':
for i in xrange(5):
p = Process(target=f, args=('bob',))
p.start()
sys.excepthook = myexcepthook
# Sleep for a bit and then force an exception by doing something stupid.
sleep(1)
1 / 0
I am looking something like this.
If i have a method in a class, now that i accept 2 command line arguments (start, stop), so when i run python script.py start, it will start automatically, now after sometime if user enters script.py stop it should end gracefully, can it be done.
This is what i have tried to do
from multiprocessing import Process, Queue
from time import sleep
import sys
def start(name):
print('hello', name)
print "Sleeping for 60 sec"
# sleep (60)
if __name__ == '__main__':
p = Process(target=start, args=('bob',))
p.start()
Thanks