I want to embed python in my C++ application. I'm using Boost library - great tool. But i have one problem.
If python function throws an exception, i want to catch it and print error in my application or get some detailed information like line number in python script that caused error.
How can i do it? I can't find any functions to get detailed exception information in Python API or Boost.
try {
module=import("MyModule"); //this line will throw excetion if MyModule contains an error
} catch ( error_already_set const & ) {
//Here i can said that i have error, but i cant determine what caused an error
std::cout << "error!" << std::endl;
}
PyErr_Print() just prints error text to stderr and clears error so it can't be solution
Well, I found out how to do it.
Without boost (only error message, because code to extract info from traceback is too heavy to post it here):
PyObject *ptype, *pvalue, *ptraceback;
PyErr_Fetch(&ptype, &pvalue, &ptraceback);
//pvalue contains error message
//ptraceback contains stack snapshot and many other information
//(see python traceback structure)
//Get error message
char *pStrErrorMessage = PyString_AsString(pvalue);
And BOOST version
try{
//some code that throws an error
}catch(error_already_set &){
PyObject *ptype, *pvalue, *ptraceback;
PyErr_Fetch(&ptype, &pvalue, &ptraceback);
handle<> hType(ptype);
object extype(hType);
handle<> hTraceback(ptraceback);
object traceback(hTraceback);
//Extract error message
string strErrorMessage = extract<string>(pvalue);
//Extract line number (top entry of call stack)
// if you want to extract another levels of call stack
// also process traceback.attr("tb_next") recurently
long lineno = extract<long> (traceback.attr("tb_lineno"));
string filename = extract<string>(traceback.attr("tb_frame").attr("f_code").attr("co_filename"));
string funcname = extract<string>(traceback.attr("tb_frame").attr("f_code").attr("co_name"));
... //cleanup here
This is the most robust method I've been able to come up so far:
try {
...
}
catch (bp::error_already_set) {
if (PyErr_Occurred()) {
msg = handle_pyerror();
}
py_exception = true;
bp::handle_exception();
PyErr_Clear();
}
if (py_exception)
....
// decode a Python exception into a string
std::string handle_pyerror()
{
using namespace boost::python;
using namespace boost;
PyObject *exc,*val,*tb;
object formatted_list, formatted;
PyErr_Fetch(&exc,&val,&tb);
handle<> hexc(exc),hval(allow_null(val)),htb(allow_null(tb));
object traceback(import("traceback"));
if (!tb) {
object format_exception_only(traceback.attr("format_exception_only"));
formatted_list = format_exception_only(hexc,hval);
} else {
object format_exception(traceback.attr("format_exception"));
formatted_list = format_exception(hexc,hval,htb);
}
formatted = str("\n").join(formatted_list);
return extract<std::string>(formatted);
}
In the Python C API, PyObject_Str returns a new reference to a Python string object with the string form of the Python object you're passing as the argument -- just like str(o) in Python code. Note that the exception object does not have "information like line number" -- that's in the traceback object (you can use PyErr_Fetch to get both the exception object and the traceback object). Don't know what (if anything) Boost provides to make these specific C API functions easier to use, but, worst case, you could always resort to these functions as they are offered in the C API itself.
This thread has been very useful for me, but I had problems with the Python C API when I tried to extract the error message itself with no traceback. I found plenty of ways to do that in Python, but I couldn't find any way to do this in C++. I finally came up with the following version, which uses the C API as little as possible and instead relies much more on boost python.
PyErr_Print();
using namespace boost::python;
exec("import traceback, sys", mainNamespace_);
auto pyErr = eval("str(sys.last_value)", mainNamespace_);
auto pyStackTrace = eval("'\\n'.join(traceback.format_exception(sys.last_type, sys.last_value, sys.last_traceback))", mainNamespace_);
stackTraceString_ = extract<std::string>(pyStackTrace);
errorSummary_ = extract<std::string>(pyErr);
The reason this works is because PyErr_Print() also sets the value for sys.last_value, sys.last_type, and sys.last_traceback. Those are set to the same values as sys.exc_info would give, so this is functionally similar to the following python code:
import traceback
import sys
try:
raise RuntimeError("This is a test")
except:
err_type = sys.exc_info()[0]
value = sys.exc_info()[1]
tb = sys.exc_info()[2]
stack_trace = "\n".join(traceback.format_exception(err_type, value, tb))
error_summary = str(value)
print(stack_trace)
print(error_summary)
I hope someone finds this useful!
Here's some code based on some of the other answers and comments, nicely formatted with modern C++ and comments. Minimally tested but it seems to work.
#include <string>
#include <boost/python.hpp>
#include <Python.h>
// Return the current Python error and backtrace as a string, or throw
// an exception if there was none.
std::string python_error_string() {
using namespace boost::python;
PyObject* ptype = nullptr;
PyObject* pvalue = nullptr;
PyObject* ptraceback = nullptr;
// Fetch the exception information. If there was no error ptype will be set
// to null. The other two values might set to null anyway.
PyErr_Fetch(&ptype, &pvalue, &ptraceback);
if (ptype == nullptr) {
throw std::runtime_error("A Python error was detected but when we called "
"PyErr_Fetch() it returned null indicating that "
"there was no error.");
}
// Sometimes pvalue is not an instance of ptype. This converts it. It's
// done lazily for performance reasons.
PyErr_NormalizeException(&ptype, &pvalue, &ptraceback);
if (ptraceback != nullptr) {
PyException_SetTraceback(pvalue, ptraceback);
}
// Get Boost handles to the Python objects so we get an easier API.
handle<> htype(ptype);
handle<> hvalue(allow_null(pvalue));
handle<> htraceback(allow_null(ptraceback));
// Import the `traceback` module and use it to format the exception.
object traceback = import("traceback");
object format_exception = traceback.attr("format_exception");
object formatted_list = format_exception(htype, hvalue, htraceback);
object formatted = str("\n").join(formatted_list);
return extract<std::string>(formatted);
}
Btw I was curious why everyone is using handle<> instead of handle. Apparently it disables template argument deduction. Not sure why you'd want that here but it isn't the same anyway, and the Boost docs say to use handle<> too so I guess there is a good reason.
Related
I'm writing a program with C++ and boost::python, and it seems very strange to me that boost::python::exec returns anything at all. For example, in the docs here, it says:
Effects
Execute Python source code from code in the context specified by the dictionaries globals and locals.
Returns
An instance of object which holds the result of executing the code.
And yet the documentation for python 3's exec function says that:
The return value is None.
So what's the point of returning anything if the function always returns none? Why not just make it a void function, or even better yet, have it return a python error if something goes wrong? Or maybe, I'm just misunderstanding the documentation, and there is something useful in there after all. Which is why I'm asking this question.
As I was trying to figure this out, I tried this sample program:
#include <boost\python.hpp>
#include <iostream>
int main()
{
using namespace boost::python;
Py_Initialize();
object main_module = import("__main__");
object main_namespace = main_module.attr("__dict__");
while (true)
{
try
{
std::cout << ">>> ";
std::string comm;
std::getline(std::cin, comm);
if (comm == "exit")
break;
object bar = exec(comm.c_str(), main_namespace);
if (bar.is_none())
std::cout << "None\n";
}
catch (error_already_set const &)
{
PyErr_Print();
}
}
}
And it seems like exec never returned an object that isn't None.
So is there ever, under any circumstance a reason to keep around the return value of a boost::python::exec call, or should I just always throw it away?
The concept of a void function in Python is one with no return value. If you attempt to assign the result of a void function, the result will always be None.
Boost::Python seems to follow this in their implementation of py::exec, although this is no surprise as even the CPython PyRun_String function that py::exec calls returns a PyObject that is always None.
So to answer your question, yes you can ignore the return value.
I am a beginner in python and have an intermediate level knowledge about C++.
I am trying to embed python code in c++. However, I get a build error and with my current level of knowledge I am unable to troubleshoot it.Kindly help me in this regard.
Following is the code.
#include <iostream>
using namespace std;
#include <Python.h>
int main()
{
cout<<"Calling Python to find the sum of 2 and 2";
// Initialize the Python interpreter.
Py_Initialize();
// Create some Python objects that will later be assigned values.
PyObject *pName,*pModule, *pDict, *pFunc, *pArgs, *pValue;
// Convert the file name to a Python string.
pName = PyString_FromString("Sample.py"); // Import the file as a Python module.
pModule = PyImport_Import(pName);
// Create a dictionary for the contents of the module.
pDict = PyModule_GetDict(pModule);
// Get the add method from the dictionary.
pFunc = PyDict_GetItemString(pDict, "add");
// Create a Python tuple to hold the arguments to the method.
pArgs = PyTuple_New(2);
// Convert 2 to a Python integer.
pValue = PyInt_FromLong(2);
// Set the Python int as the first and second arguments to the method.
PyTuple_SetItem(pArgs, 0, pValue);
PyTuple_SetItem(pArgs, 1, pValue);
// Call the function with the arguments.
PyObject* pResult = PyObject_CallObject(pFunc, pArgs);
// Print a message if calling the method failed.
if(pResult == NULL)
cout<<"Calling the add method failed.\n";
// Convert the result to a long from a Python object.
long result = PyInt_AsLong(pResult);
// Destroy the Python interpreter.
Py_Finalize(); // Print the result.
cout<<"The result is"<<result;
cout<<"check";
return 0;
}
I get the following error:
Unhandled exception at 0x00000000 in pytest.exe: 0xC0000005: Access violation.
and the build breaks at the line pModule = PyImport_Import(pName);
the file Sample.py has the contents:
# Returns the sum of two numbers.
def add(a, b):
return a+b
I am using python 2.7 and VS2010.I have created a win32 console project for this and I am building in release mode.I have copied the file Sample.py to the project folder.
I cannot figure out what's causing the build to crash.Kindly help.
Firstly, indent your code, MSVC even has an option to do it automatically! After all, you want people here to read it, so go and clean that up. Then, don't declare variables without initializing them in C++. This makes it clear from where on you can use them. Lastly, whenever you call a function, check its results for errors. By default, just throw std::runtime_error("foo() failed"); on errors. More elaborate, you could try to retrieve and add error information from Python.
Now, your immediate error is the use of a null pointer, which you would have avoided if you had checked returnvalues. After writing the code to correctly detect that error, the next thing I'd look at is the missing initialization of the Python interpreter. You already have a comment in place, but comments don't count. I guess that if you had implemented proper error handling, Python would also have told you about the missing initialization.
I'm trying to add embedded python to a OpenGL/SDL application.
So far everything works fine, like entering a string via SDL keyboard events and executing it with the embedded python interpreter.
I'm now into adding functions to call C/C++ functions like
void set_iterations(int c);
is invoked on the python interpreter with
>>> test.iterations(23)
The parsing of the integer parameter works like charm
static PyObject* test_iterations(PyObject *self, PyObject *args) {
int iterations;
PyArg_ParseTuple(args,"i",&iterations);
set_iterations(iterations);
return Py_None;
}
But when I try this: >>> test.addignore('something')
static PyObject* test_addignore(PyObject *self, PyObject *args) {
char* ignorestring;
//PyArg_ParseTuple(args,"s",ignorestring);
PyArg_ParseTuple(args,"s",&ignorestring);
add_global_ignore(ignorestring); // C function
return Py_None;
}
Python gives me this error:
Traceback (most recent call last):
File "<string>", line 1, in <module>
UnicodeDecodeError: 'utf8' codec can't decode bytes in position 0-3: invalid data
The string should be UTF-8, as SDL is set to grab UNICODE from the keyboard, and everything else works perfectly.
Does anyone have an idea on what I might be doing wrong here?
I also inspected the args object passed to the function with
std::string args_str;
PyObject* repr = PyObject_Repr(args);
if (repr != NULL) {
args_str = PyBytes_AsString(PyUnicode_AsEncodedString(repr, "utf-8", "Error"));
Py_DECREF(repr);
}
std::cout << args_str << "\n";
And it gives me this: ('somestring',)
Solution:
The error pointed out by rodrigo, was originally causing be believe that my debug code that should print the resulting string as PyObject was wrong. But the problem was that I was passing the parser the wrong pointer, leading to said undefined behaviour in the memory and therefore leading me to believe the parser was the problem.
The occurring last parser error was then the debug output itself, which was pointing to the wrong memory address.
Thanks rodrigo, since your answer lead to solving the problem: accepted. Thank you for your help and patience.
Try:
static PyObject* test_addignore(PyObject *self, PyObject *args) {
char* ignorestring;
if (!PyArg_ParseTuple(args,"s", &ignorestring)) //Note the &
return NULL; //Never ignore errors
add_global_ignore(ignorestring);
Py_RETURN_NONE; //Always use helper macros
}
I've got this code snnipet (the whole program compiles and links correctly):
...
try
{
boost::python::exec_file(
"myscript.py", // this file contains a syntax error
my_main_namespace,
my_local_namespace
);
return true;
}
catch(const boost::python::error_already_set &)
{
PyObject *ptype, *pvalue, *ptraceback;
PyErr_Fetch(&ptype, &pvalue, &ptraceback);
// the next line crashes on syntax error
std::string error = boost::python::extract<std::string>(pvalue);
...
}
The file that the program tries to execute has a syntax error, so an exception is thrown. When the program tries to get the error message crashes...
The code works well with run-time errors but somehow crashes with syntax errors.
How can i get the error string with this kind of errors?
Thanks in advance
From the documentation of PyErr_Fetch: "The value and traceback object may be NULL even when the type object is not". You should check whether pvalue is NULL or not before trying to extract the value.
std::string error;
if(pvalue != NULL) {
error = boost::python::extract<std::string>(pvalue);
}
If you want to check whether the exception is a SyntaxError you can compare ptype against the exception types listed here.
To answer anymore specifically I would need a backtrace from the point where it crashed.
Edit
pvalue is an exception object, not a str instance so you should use PyObject_Str to get the string representation of the exception.
You may need to call PyErr_NormalizeException first to turn pvalue into the correct exception type.
I have written some C++ code that generates a std::vector.
I also have a python script that manipulates some data that, for now, I am declaring like this (below).
import numpy
x = numpy.random.randn(1000)
y = numpy.random.randn(1000)
I can run the script fine. From my C++ code:
using namespace boost::python;
try{
Py_Initialize();
object main = import("__main__");
object global(main.attr("__dict__"));
object result = exec_file("scatterPlot.py", global, global);
Py_Finalize();
}
catch(error_already_set){
PyErr_Print();
}
return;
I have no idea how to get my C++ data to python. I've around quite a bit, but there doesn't seem to be anything definitive.
I have in my C++
BOOST_PYTHON_MODULE(vector_indexing_suite_ext){
boost::python::class_<std::vector<double> >("PyVec")
.def(boost::python::vector_indexing_suite<std::vector<double> >());
}
This seems to work, but as I understand, it only provides a class "PyVec" for my python script but not the data I need. Am I wrong?
I've also seen some other people use boost::shared_ptr in a python mailing list.
I also found this example but found it confusing.
I can think of a few approaches
Pass something to the boost::python::exec_file method
Using the boost_indexing_suite_ext
Uinsg boost::shared_ptr
Which approach is easiest to get going? No approach seems clear to me.
Here are some more links I've looked at:
from the boost website
from the python website
another mailing list thread
UPDATE:
This works for passing an int to my python code like below
int main(){
int five_squared=0;
int a =3;
try {
Py_Initialize();
object main_module = import("__main__");
object main_namespace = main_module.attr("__dict__");
main_namespace["var"]=a;
object ignored = exec("result = 5 ** var", main_namespace);
five_squared = extract<int>(main_namespace["result"]);
} catch( error_already_set ) {
PyErr_Print();
}
std::cout << five_squared << std::endl;
return 0;
}
But I want to pass a vector, when I try to do that in a similar fashion as above I get this error
TypeError: No to_python (by-value)
converter found for C++ type:
std::vector >
So, obviously I need to tell python how to deal with std::vector. I think this code could help with that.
BOOST_PYTHON_MODULE(vector_indexing_suite_ext){
boost::python::class_<std::vector<double> >("PyVec")
.def(boost::python::vector_indexing_suite<std::vector<double> >());
}
But since std::vector is pretty common, there must be a defined way to do this... right?
The following code works for me (Python 2.6, Boost 1.39). This is almost the same as your code, except without the BOOST_PYTHON_MODULE line itself (but with the class_ definition for the vector). BOOST_PYTHON_MODULE only needs to be used when creating extension modules.
#include <iostream>
#include <boost/python.hpp>
#include <boost/python/suite/indexing/vector_indexing_suite.hpp>
using namespace boost::python;
using namespace std;
int main()
{
vector<double> vec;
vec.push_back(1.2);
vec.push_back(3.4);
try {
Py_Initialize();
boost::python::class_<std::vector<double> >("PyVec")
.def(boost::python::vector_indexing_suite<std::vector<double> >());
object main_module = import("__main__");
object globals = main_module.attr("__dict__");
globals["var"]=vec;
object ignored = exec("result = sum(var)", globals, globals);
double result = extract<double>(globals["result"]);
std::cout << result << std::endl;
} catch( error_already_set ) {
PyErr_Print();
}
return 0;
}
I'm not sure if I understand correctly. After exporting your class "PyVec" which can hold std::vector<double>, you can export any c++ function taking vector as input or return type. So of course you can populate your vector within c++ and access this data in Python with the interfaced type "PyVec".