Suppress Python printout - c++

I'm embedding Python in a C++ application (using the Python C API) and I want Python exceptions that are thrown to be handled by an exception handler mechanism that is already setup in C++. This exception handler mechanism will print out the exception itself, so I don't want the embedded Python interpreter to print it to screen. Can I setup the embedded Python interpreter to suppress console output in some way?

You can plug a Stub-Streamhandler to both the standard out and standard error channels in python.
Here a sample (inclusive revoicing both channels):
import sys
import cStringIO
print("Silencing stdout and stderr")
_stdout = sys.stdout
_stderr = sys.stderr
sys.stdout = cStringIO.StringIO()
sys.stderr = cStringIO.StringIO()
print("This should not be printed out")
sys.stdout = _stdout
sys.stderr = _stderr
print("Revoiced stdout and stderr")
Running this sample should results in following output:
Silencing stdout and stderr
Revoiced stdout and stderr
[UPDATED]
And here the memory-saving variant with sending to devnull:
import os, sys
with open(os.devnull, 'w') as devnull:
sys.stdout = devnull
sys.stderr = devnull
# ... here your code

How did you execute the code? If an exception is not catched and printed by python directly (someone has installed an handler in python directly) it will be set to the exception handling and then the exception can be examined using C (see section Exception Handling in the reference manual).
If you call the python script from C-code, the calling function will return NULL or -1 (exception has occured) and now you can use the above stated API to read the exception (for e.g. PyErr_Occurred(),... ).
But if the things are done by the python code itself (if someone installed a hook or something inside its code and prints it directly), you have no change to catch things easily. Maybe you can add a hook by yourself, calling your code. This can be done via sys.excepthook(type, value, traceback) and one solution would be to find the position and remove it from the code or have a look at other methods to circumvent it.
This site is very informative about it: Python API/include files
Edit:
For redirecting stdout or stderr you can use the C-Api function:
FILE* sto = fopen("out.txt", "w+");
FILE* ste = fopen("outErr.txt", "w+");
PySys_SetObject("stdout", PyFile_FromFile(sto, "out.txt","wb", fclose));
PySys_SetObject("stderr", PyFile_FromFile(ste, "outErr.txt","wb", fclose));
before calling any other python code. PyFile_FromFile maybe also replaced by something else, I think, but I have not used this so far.

Related

Python subprocess module giving an OSError while running UNIX commands

Here's the context:
I am using python 2.7.5. And I would like to run UNIX commands as well as maven commands in a python script.
I was successful to do so, using os.system("cmd"), but I need to work on the result of the given command. After reading the doc and some threads in here, I decided to use the subprocess module to redirect the output to the stdout using PIPE. Unexpectedly, I am getting an OSError as shown in the attached image. Your help will be much appreciated.
In addition to the given sample in the attached image, I have tried:
p = os.popen("java -version")
result = subprocess.check_output(p, shell=True)
subprocess.call("ls /usr", shell=True)
p.s. Using shell=True is strongly discouraged (doc), since it can be dangerous when coupled with unsanitized input.
Also, I took a look at the given script in the error message /usr/lib64/python2.7/subprocess.py, line 711 adn 1327 but didn't learn more than what is mentionned in the error message: raise child_exception
Subprocess Terminal Output
You aren't using subprocess.check_output correctly. You're trying to pass a pipe file object (the return value of os.popen) to check_output but it's expecting a command argument or argument vector.
Also, the subprocess.call function won't capture the executed command's output, so you would only use that if you want the output of ls /usr (or whatever) to be seen by the user running the script interactively. (Which is pretty much the same as os.system.)
Try this instead (showing with and without the shell):
import subprocess
out1a = subprocess.check_output(['java', '-version'], stderr=subprocess.STDOUT)
print(out1a)
out1b = subprocess.check_output('java -version', stderr=subprocess.STDOUT, shell=True)
print(out1b)
out2a = subprocess.check_output(['ls', '/usr'])
print(out2a)
out2b = subprocess.check_output('ls /usr', shell=True)
print(out2b)
# Cannot capture output this way, but it will be visible to user
subprocess.call('ls /usr', shell=True)
Note that in the case of the java -version command, the version info gets printed to the command's standard error output so you must redirect that in order to capture it as the returned value of check_output (hence the stderr=subprocess.STDOUT).

How to close file descriptors in python?

I have the following code in python:
import os
class suppress_stdout_stderr(object):
'''
A context manager for doing a "deep suppression" of stdout and stderr in
Python, i.e. will suppress all print, even if the print originates in a
compiled C/Fortran sub-function.
This will not suppress raised exceptions, since exceptions are printed
to stderr just before a script exits, and after the context manager has
exited (at least, I think that is why it lets exceptions through).
'''
def __init__(self):
# Open a pair of null files
self.null_fds = [os.open(os.devnull,os.O_RDWR) for x in range(2)]
# Save the actual stdout (1) and stderr (2) file descriptors.
self.save_fds = (os.dup(1), os.dup(2))
def __enter__(self):
# Assign the null pointers to stdout and stderr.
os.dup2(self.null_fds[0],1)
os.dup2(self.null_fds[1],2)
def __exit__(self, *_):
# Re-assign the real stdout/stderr back to (1) and (2)
os.dup2(self.save_fds[0],1)
os.dup2(self.save_fds[1],2)
# Close the null files
os.close(self.null_fds[0])
os.close(self.null_fds[1])
for i in range(10**6):
with suppress_stdout_stderr():
print 'plop'
if i % 50 == 0:
print i
it fails at 5100 on OSX with OSError: [Errno 24] Too many open files. I'm wondering why and if there is a solution to close the file descriptor. I'm looking for a solution for a context manager which closes stdout and stderr.
I executed your code on a Linux machine and got the same error but at a different number of iterations.
I added the following two lines in the __exit__(self, *_) function of your class:
os.close(self.save_fds[0])
os.close(self.save_fds[1])
With this change I do not get an error and the script returns successfully. I assume that the duplicated file descriptors stored in self.save_fds are kept open if you don't close them with os.close(fds) and so you get the too many files open error.
Anyway my console printed "plop", but maybe this depends on my platform.
Let me know if it works :)

Print statement causing 500 response

def gettable(request):
reqdata = request.POST
data = reqdata['dataabc']
# print data
return HttpResponse("OK")
This works, but as soon as I uncomment print data, I see a 500 response in my dev console.
Why could this be happening? I just need to print a couple of things to the console to test.
python 3
In python 3, print is a function.
Using print as a statement will fail and raise an exception which will terminate your view function prematurely and cause an error 500.
logging module
print is bad practice in libraries and in server-side / background tasks code. please use the logging module instead. django even has a section for how to configure and use logging properly.
mod_wsgi and print
https://code.google.com/p/modwsgi/wiki/DebuggingTechniques
Prior to mod_wsgi version 3.0, you would see this when using print with sys.stdout:
IOError: sys.stdout access restricted by mod_wsgi
and you would need to explicitly use a file, e.g.:
print >> sys.stderr, data # python 2
You can however disable the restriction by editing your configuration and using the WSGIRestrictStdout directive.
Completely agree with dnozay. Logging module is a way to go.
More about print statements with wsgi:
http://blog.dscpl.com.au/2009/04/wsgi-and-printing-to-standard-output.html?m=1

Controlling C++ Output from Python Script

I have a bit of an issue here. I have a Python script which calls binaries compiled from C++. The Python script has its own set of outputs (to standard out and error), which are easily disable-able. The C++ binaries have their own set of outputs (to standard out and error, among others) as well; the source can be altered, but I am not the original author. This is an issue because I do not want the C++ output in my final program, and I also don't want future users to need to edit the C++ source.
What I'd like to be able to do is have some Python method which will catch the C++ code's output that is sent to standard out or error. Is this possible? If so, could someone point me in the right direction?
Thank you!!
One way to do this is:
Duplicate in python the file descriptors for stdout and stderr using os.dup.
Redirect the original stdout and stderr using reopen (from C's stdio) to write to a file of your choice.
Note: reopen isn't available directly from python, but you should be able to call it as in the example below or using any other wrapper available.
After this is done:
Every write to cout and cerr in C++ will write to the output files.
Every print statement in python will write to the output files.
However, since the original descriptors are duplicated, you can still (see example below):
Print to the original stdout/stderr using sdout.write and stdout.err
Use logging methods after configuring properly the stream parameter
The following code uses instant library to test real C++ code that is wrapped into python using SWIG and that should be similar to the library that you have:
import sys, os
import logging
from instant import inline
print 'This is printed from python to stdout'
stdout = os.fdopen(os.dup(sys.stdout.fileno()), 'w')
stderr = os.fdopen(os.dup(sys.stderr.fileno()), 'w')
logging.basicConfig(stream=stderr, level=logging.DEBUG)
redirect = inline("""
void redirect(void) {
freopen("my_stdout.txt", "w", stdout);
freopen("my_stderr.txt", "w", stderr);
}
""")
redirect()
cout = inline("""
void cout(void) {
std::cout << "This is written from C++ to my_stdout.txt" << std::endl;
std::cerr << "This is written from C++ to my_stderr.txt" << std::endl;
}
""")
cout()
print 'This is written from python to my_stdout.txt'
stdout.write('This is printed from python to stdout\n')
stderr.write('This is printed from python to stderr\n')
logging.info('This is printed to stderr from python using logging')
The output for this example is:
$ python test.py
This is printed from python to stdout
This is printed from python to stdout
This is printed from python to stderr
INFO:root:This is printed to stderr from python using logging
$ cat my_stdout.txt
This is written from C++ to my_stdout.txt
This is written from python to my_stdout.txt
$ cat my_stderr.txt
This is written from C++ to my_stderr.txt
Note: First time the code is executed, you might get gcc compilation messages (I've removed them to make the example clearer).
Are you using subprocess to compile the C++? If so, you can set where stderr and stdout go:
nowhere = StringIO()
subprocess.call("exit 1", shell=True, stdout=nowhere, stderr=nowhere)

how to pass a value to c++ from python and back?

i would like to pass values from python to a c++ program for an encryption from inside a python program and then return the value from there to the python program . how to do it?
If you want to use some existing Unix-style command line utility that reads from stdin and writes to stdout, you can use subprocess.Popen by using Popen.communicate():
import subprocess
p = subprocess.Popen(["/your/app"], stdin=subprocess.PIPE, stdout=subprocess.PIPE)
output = p.communicate(input)[0]
As said msw in the other post, the proper solution is using PyObject.
If you want to have a two-way communication between C++ & Python, Boost Python would be interesting for you. Take a look at website Boost Python,
This post would also be interesting:
How to expose a C++ class to Python without building a module