I have a bit of an issue here. I have a Python script which calls binaries compiled from C++. The Python script has its own set of outputs (to standard out and error), which are easily disable-able. The C++ binaries have their own set of outputs (to standard out and error, among others) as well; the source can be altered, but I am not the original author. This is an issue because I do not want the C++ output in my final program, and I also don't want future users to need to edit the C++ source.
What I'd like to be able to do is have some Python method which will catch the C++ code's output that is sent to standard out or error. Is this possible? If so, could someone point me in the right direction?
Thank you!!
One way to do this is:
Duplicate in python the file descriptors for stdout and stderr using os.dup.
Redirect the original stdout and stderr using reopen (from C's stdio) to write to a file of your choice.
Note: reopen isn't available directly from python, but you should be able to call it as in the example below or using any other wrapper available.
After this is done:
Every write to cout and cerr in C++ will write to the output files.
Every print statement in python will write to the output files.
However, since the original descriptors are duplicated, you can still (see example below):
Print to the original stdout/stderr using sdout.write and stdout.err
Use logging methods after configuring properly the stream parameter
The following code uses instant library to test real C++ code that is wrapped into python using SWIG and that should be similar to the library that you have:
import sys, os
import logging
from instant import inline
print 'This is printed from python to stdout'
stdout = os.fdopen(os.dup(sys.stdout.fileno()), 'w')
stderr = os.fdopen(os.dup(sys.stderr.fileno()), 'w')
logging.basicConfig(stream=stderr, level=logging.DEBUG)
redirect = inline("""
void redirect(void) {
freopen("my_stdout.txt", "w", stdout);
freopen("my_stderr.txt", "w", stderr);
}
""")
redirect()
cout = inline("""
void cout(void) {
std::cout << "This is written from C++ to my_stdout.txt" << std::endl;
std::cerr << "This is written from C++ to my_stderr.txt" << std::endl;
}
""")
cout()
print 'This is written from python to my_stdout.txt'
stdout.write('This is printed from python to stdout\n')
stderr.write('This is printed from python to stderr\n')
logging.info('This is printed to stderr from python using logging')
The output for this example is:
$ python test.py
This is printed from python to stdout
This is printed from python to stdout
This is printed from python to stderr
INFO:root:This is printed to stderr from python using logging
$ cat my_stdout.txt
This is written from C++ to my_stdout.txt
This is written from python to my_stdout.txt
$ cat my_stderr.txt
This is written from C++ to my_stderr.txt
Note: First time the code is executed, you might get gcc compilation messages (I've removed them to make the example clearer).
Are you using subprocess to compile the C++? If so, you can set where stderr and stdout go:
nowhere = StringIO()
subprocess.call("exit 1", shell=True, stdout=nowhere, stderr=nowhere)
Related
Is there any good way i can make some data created by my c++ program available to the shell after exiting the program?
I have a c++ program, inside which i have a string containing a path:
std::string path = "/home/some/path"
I want this path to be available after the c++ program exits main and i am returned to the shell, so that i can use that path (e.g. cd to that path).
I have considered/tried the following approaches:
I tried making an environment variable in c++ program using setenv(). However the environment variable only exists while in the c++ program, and it is apparently not possible to make those changes visible in the shell after exiting the program.
(considered) writing the path to a temporary file, so that a bash script could later access the details of the path from it. However i have read many suggestions to not do that due to security vulnerabilities.
I tried calling the bash script from within the c++ program, using system(). This does not work if i try to cd to that directory (exiting the program will keep me in the same directory as before).
I figure that if i am desperate, i could have my program cout the path, and use the solutions as described here:
$ ./program | tee output.txt
Then the path is stored inside the file. This works technically, but has the undesirable effect of creating a file and printing the path to the screen, and is basically creating a temporary file.
another option to, again, cout in my program, and use command substitution. running in the shell
$ var=$(./program)
storing the path in var. This didnt work because my program does many things including requiring user input before calling
std::cout<< path << std::endl;.
Particularly, i have observed this approach to not display a curses window, which is required for the program.
the only solution that has worked is piping the output to tee.
Environment variables are only an input, they cannot be used to return any information from a program.
You are already using std::cin and std::cout for user input, and std::cerr should be reserved for error messages. However, you can have the shell open more filedescriptors, and have your program write to those. However, doing this with pure C++ is not possible. But if you don't mind using POSIX C functions:
#include <cstdio>
int main() {
FILE *f = fdopen(3, "w");
fprintf(f, "some path\n");
}
And then use it like so:
./program 3> output.txt
This of course creates an undesirable file. I don't think there is any way to store the output from an extra filedescriptor directly to a variable in bash. However, you could create a temporary file inside /dev/shm, so it will never be written to disk, or create a FIFO object and redirect the output from the program to the FIFO, and then read it back. For some examples of how to do this, see this question.
You could write the output that you want the user to see to stderr instead of stdout. Only output what you need your shell script to see to stdout:
#include <iostream>
int main() {
std::clog << "Enter data: "; // clog prints to stderr like cerr
std::string line;
std::getline(std::cin, line);
std::cout << line << '\n';
}
Then this will work:
var=$(./program)
I am trying to get the output from subprocess.Popen assign it to a variable and then work with it in the rest of my program however it is executing code without ever assigning it to my variable
my current line of code is
result = subprocess.Popen('bzip2 --version', shell=True, stdout=subprocess.PIPE).communicate()[0]
currently to test it im just printing the length and the results which are currently empty
it does execute the code but it shows up in the terminal prior to my prints
I have tried the above-mentioned code using other commands and it works just as I would expect
any suggestions on how I can go about doing this?
Seems bzip2 writes to stderr instead of to stdout.
result = subprocess.Popen('bzip2 --version', shell=True, stderr=subprocess.PIPE).communicate()[1]
I am running the following code in Python 2.7:
values = os.system("bazel build tensorflow/examples/image_retraining:"
"label_image && bazel-bin/tensorflow/examples/image_retraining/label_image "
"--graph=/tmp/output_graph.pb --labels=/tmp/output_labels.txt "
"--output_layer=final_result:0 --image=$HOME/Desktop/Image-3/image1.png")
print values
But for the values variable I am returned a 0. I believe this means that I am not getting any errors. How do I store the output into a variable or into a .txt file?
You can just redirect the output of the system call appending > output.txt to your command.
The output of the command will be in file output.txt in the directory where you invoke the command (likely the very same one you invoke your python script in).
Since I can't readily reproduce your command, I used a simple example - try to switch to Pyopen in the subprocess module:
from subprocess import Popen
proc = Popen(['ls', '-t'], stdout = open('/path/redir.txt', 'w'))
Here you run the command in square brackets and redirect the output from stdout i.e. the terminal to a file redir.txt.
I'm embedding Python in a C++ application (using the Python C API) and I want Python exceptions that are thrown to be handled by an exception handler mechanism that is already setup in C++. This exception handler mechanism will print out the exception itself, so I don't want the embedded Python interpreter to print it to screen. Can I setup the embedded Python interpreter to suppress console output in some way?
You can plug a Stub-Streamhandler to both the standard out and standard error channels in python.
Here a sample (inclusive revoicing both channels):
import sys
import cStringIO
print("Silencing stdout and stderr")
_stdout = sys.stdout
_stderr = sys.stderr
sys.stdout = cStringIO.StringIO()
sys.stderr = cStringIO.StringIO()
print("This should not be printed out")
sys.stdout = _stdout
sys.stderr = _stderr
print("Revoiced stdout and stderr")
Running this sample should results in following output:
Silencing stdout and stderr
Revoiced stdout and stderr
[UPDATED]
And here the memory-saving variant with sending to devnull:
import os, sys
with open(os.devnull, 'w') as devnull:
sys.stdout = devnull
sys.stderr = devnull
# ... here your code
How did you execute the code? If an exception is not catched and printed by python directly (someone has installed an handler in python directly) it will be set to the exception handling and then the exception can be examined using C (see section Exception Handling in the reference manual).
If you call the python script from C-code, the calling function will return NULL or -1 (exception has occured) and now you can use the above stated API to read the exception (for e.g. PyErr_Occurred(),... ).
But if the things are done by the python code itself (if someone installed a hook or something inside its code and prints it directly), you have no change to catch things easily. Maybe you can add a hook by yourself, calling your code. This can be done via sys.excepthook(type, value, traceback) and one solution would be to find the position and remove it from the code or have a look at other methods to circumvent it.
This site is very informative about it: Python API/include files
Edit:
For redirecting stdout or stderr you can use the C-Api function:
FILE* sto = fopen("out.txt", "w+");
FILE* ste = fopen("outErr.txt", "w+");
PySys_SetObject("stdout", PyFile_FromFile(sto, "out.txt","wb", fclose));
PySys_SetObject("stderr", PyFile_FromFile(ste, "outErr.txt","wb", fclose));
before calling any other python code. PyFile_FromFile maybe also replaced by something else, I think, but I have not used this so far.
i would like to pass values from python to a c++ program for an encryption from inside a python program and then return the value from there to the python program . how to do it?
If you want to use some existing Unix-style command line utility that reads from stdin and writes to stdout, you can use subprocess.Popen by using Popen.communicate():
import subprocess
p = subprocess.Popen(["/your/app"], stdin=subprocess.PIPE, stdout=subprocess.PIPE)
output = p.communicate(input)[0]
As said msw in the other post, the proper solution is using PyObject.
If you want to have a two-way communication between C++ & Python, Boost Python would be interesting for you. Take a look at website Boost Python,
This post would also be interesting:
How to expose a C++ class to Python without building a module