Boost Python - Unbound method call - c++

I'm trying to use Python embedded in C++ with Boost::python.
My embedded script are supposed to use decorator to register their methods like following:
class Test:
def __init__(self, object_id):
self.object_id = object_id
#decorator('my_id_1')
def func1(self):
print("{}.func1".format(self.object_id))
decorator is declared on the C++ side, defining the method the __init__ and __call__. Everything works has expected, until the call of the method, which lead to SIGSEGV or SIGARBT.
Here is an example of what I would like to do in Python:
#CPP side
item = {}
class decorator:
def __init__(self, _id):
self._id = _id
def __call__(self, func):
item[self._id] = func #saved as PyObject* in CPP
print func
return func
#Script side
class Test(CppBase):
def __init__(self, object_id):
CppBase.__init__(self)
self.object_id = object_id
#decorator('my_id_1')
def func1(self):
print("{}.func1".format(self.object_id))
#decorator('my_id_2')
def func2(self):
print("{}.func2".format(self.object_id))
#CPP side
obj1 = Test("obj1")
item['my_id_1'](obj1) #The crash append here
To do the call, I'm using the following function: boost::python::call<void>(my_PyObject_func, boost::ref(my_obj_instance))
I won't put my all C++ code because I'm actually updating a working project made from the old Python C API, and the whole API is quite huge. However, if you think I forgot some significant part of it, just tell me, and I will post those parts. Furthermore, I removed a lot of simple check such as being sure that the global Python var contain my object, no python error happened or the hash contain the requested id, to make the code lighter.
Here are my C++ Object definition
class CppBase: public boost::python::object {
public:
CppBase();
void open(std::string, boost::python::tuple arg);
void setRoutes(const std::hash<std::string, const Route*>&);
inline std::hash<std::string, const Route*>*const routes() const { return route; }
private:
std::string name;
QHash<std::string, const Decorator*> *route;
};
class Decorator {
public:
Decorator(std::string identifier);
PyObject* call(PyObject* func);
void invoke(boost::python::object&, boost::python::tuple) const;
static void commit(CppBase&);
private:
PyObject* method;
std::string identifier;
static std::hash<std::string, const Decorator*> routes;
};
Here is how I register my Python module
BOOST_PYTHON_MODULE(app)
{
boost::python::class_<CppBase, boost::noncopyable>("CppApp") //I tried to remove 'noncopyable', nothing change
;
boost::python::class_<Decorator, boost::noncopyable>("decorator", boost::python::init<std::string>())
.def("__repr__", &Decorator::repr)
.def("__call__", &Decorator::call)
;
}
Here is the implementation of CppBase::open that I think is the only one important to show in my class definition.
...
void CppBase::open(std::string id, boost::python::tuple arg /* unused so far */){
boost::python::call<void>(route->value(id), boost::ref(*this))
}
...
Here is the Python script sample, running with this example:
class MyScriptSubClass(CppApp):
def __init__(self, object_id):
CppBase.__init__(self)
self.object_id = object_id
#decorator('my_id_1')
def func1(self):
print("{}.func1".format(self.object_id))
Here is how I try to make everything work
//... Creating Python context, Executing the Script file...
boost::python::object cls(main_module.attr("MyScriptSubClass")); //Getting the classDefinition
CppBase core = boost::python::extract<CppBase>(cls()); //Instanciating the object with the previous catched definition
Decorator::commit(core); //Save all the decorator intercepted until now into the object
core.open('my_id_1'); //Calling the function matching with this id
I hope I made everything clear.
In advance, thank you.

Related

How to use python classes in C++?

I wrote a wrapper that allows me to use python functions in C++, the api looks like this:
auto module = python.getModule("pycode"); // calls PyUnicode_DecodeFSDefault
CppPyFunction<int(int, int)> multiply(module, "multiply");
std::cout << "multiply(a, b): " << multiply(a, b); // calls PyTuple_New, PyTuple_SetItem, PyObject_CallObject
However, I dont really understand how to use python classes directly
I can use classes in a somewhat hacky way:
// python code
class Person:
def __init__(self, name):
self.name = name;
print("Person: __init__" + str(name))
def printName(self, string):
print(self.name)
def makePerson(name):
return Person(index);
def printName(person)
person.printName()
Then the C++ code would be
auto personModule = python.getModule("Person");
CppPyFunction<PyObject * (std::string)> makePerson(module, "makePerson");
CppPyFunction<void(PyObject*)> printName(module, "printName");
auto alice = makePerson("Alice");
auto bob = makePerson("Bob");
printName(alice);
printName(bob);
But this is abit of a hack because it is not natural to create the functions def makePerson(name): and def printName(person).
Question1: Is there a way to call the Person constructor and methods directly from C++ without having to create a "factory method" in the python code?
Question2: Is there a way to construct a template "class" representation in C++ with nice syntax, eg
typedef ?? CppPyClass<???> ??? Person;
Person a("Alice");
Person b("Bob);
a.printName();
b.printName();

Calling embedded function in class method using pybind11

I'm working on a c++ application that uses pybind11 to embed python and I've run into a bit of a problem when trying to call an embedded function from a class method.
to start with here are my bindings:
#ifdef _DEBUG
#undef _DEBUG
#include <python.h>
#define _DEBUG
#else
#include <python.h>
#endif
#include <embed.h>
namespace py = pybind11;
using namespace py::literals;
void DebugInfo(std::string string_)
{
String LogMessage_(string_.c_str());
LOGINFO(LogMessage_);
}
PYBIND11_EMBEDDED_MODULE(Test, m) {
m.def("DebugInfo", &DebugInfo, "Posts message to DEBUGINFO");
}
I could then have a .py file with:
import Test
test.DebugInfo("I'm a lumberjack and that's OK")
and it will print just fine to debug
The trouble starts when I try and call it from within a class method.
import Test
class PyTest(object):
def __init__(self):
test.DebugInfo("I'm a lumberjack and that's OK")
test = PyTest()
when this runs it throws an exception against cast.h specifically against line 1985 which is part of this function:
template <return_value_policy policy>
class unpacking_collector {
public:
template <typename... Ts>
explicit unpacking_collector(Ts &&...values) {
// Tuples aren't (easily) resizable so a list is needed for collection,
// but the actual function call strictly requires a tuple.
auto args_list = list();
int _[] = { 0, (process(args_list, std::forward<Ts>(values)), 0)... };
ignore_unused(_);
m_args = std::move(args_list);
}
const tuple &args() const & { return m_args; }
const dict &kwargs() const & { return m_kwargs; }
tuple args() && { return std::move(m_args); }
dict kwargs() && { return std::move(m_kwargs); }
/// Call a Python function and pass the collected arguments
object call(PyObject *ptr) const {
PyObject *result = PyObject_Call(ptr, m_args.ptr(), m_kwargs.ptr());
if (!result)
throw error_already_set(); //EXCEPTION THROWS HERE!
return reinterpret_steal<object>(result);
}
And because it's probably relevant here's how I'm calling the whole thing from my main application
//Start the Python Interpreter
py::scoped_interpreter guard{};
//Python variables
py::object thing_;
std::string test_py = Import_File("test.py");
auto locals = py::dict();
py::exec(test_py, py::globals(), locals);
thing_ = locals["test"].cast<py::object>();
thing_.attr("start")();
and the contents of test.py
import Test
class PyTest(object):
def __init__(self, message = "Test Object initialized"):
self.message = message
iterstr = str(self.iter)
message = self.message + iterstr
self.iter = 0
Test.DebugInfo(message)
def start(self):
self.message = "Starting Python Object"
self.iter = self.iter + 1
iterstr = str(self.iter)
message = self.message + iterstr
Test.DebugInfo(message)
def update(self):
self.message = "Python Object Update Cycle:"
self.iter = self.iter + 1
iterstr = str(self.iter)
message = self.message + iterstr
Test.DebugInfo(message)
test = PyTest()
I'm not sure if I've run into a limitation of pybind11, a bug in it, or if I've just screwed the whole thing up.
Any insight would be greatly appreciated.
This was also filed as an issue with pybind11 here: https://github.com/pybind/pybind11/issues/1452
I came across both this SO and the issue, but I figured this out. Copying it here for anyone who stumbles across this first in the future
Basically, you don't actually want a blank py::dict for locals; it will cause all sorts of problems. If you look embedding sample code from the docs or the tests, the locals value always copies the global scope.
See:
* https://pybind11.readthedocs.io/en/stable/advanced/embedding.html
* https://github.com/pybind/pybind11/blob/master/tests/test_embed/test_interpreter.cpp#L57
Your options are to copy the global scope, or, in this case, simply don't pass in a locals
py::scoped_interpreter guard{};
auto globals = py::globals();
py::exec(test_py, globals);
thing_ = globals["Object"].cast<py::object>();
thing_.attr("start")();
It looks like that in the case of top-level code (not inside any module), the globals variable holds the values at this scope.
So after some experimentation I discovered that the problem is caused by pybind not being able to detect an imported module outside of the scope of the function.
import foo
def bar():
foo.func()
will always cause an error. However,
def bar():
import foo
foo.func()
will function as intended.

Mock object used insided the to be tested function using spock

I can mock a function of a to be tested class in several ways. But how do I mock an object that is created inside of a to be tested method?
I have this to be tested class
#Grab('org.codehaus.groovy.modules.http-builder:http-builder:0.7')
import groovyx.net.http.HTTPBuilder
class totest {
def get() {
def http = new HTTPBuilder('http://www.google.com')
def html = http.get( path : '/search', query : [q:'Groovy'] )
return html
}
}
How do I mock http.get so I can test the get function:
class TestTest extends Specification {
def "dummy test"() {
given:
// mock httpbuilder.get to return "hello"
def to_test = new totest()
expect:
to_test.get() == "hello"
}
}
A better approach would be to pass the HTTPBuilder into your constructor and then the test code can pass test mocks instead.
But if you want to mock the class construction going on internal to your code, have a look at mocking constructors and classes using GroovySpy and GroovyMock on here: http://spockframework.org/spock/docs/1.0/interaction_based_testing.html
You would need to do something like the below code:
import spock.lang.Specification
import groovyx.net.http.HTTPBuilder
class totest {
def get() {
def http = new HTTPBuilder('http://www.google.com')
def html = http.get( path : '/search', query : [q:'Groovy'] )
return html
}
}
class TestTest extends Specification{
def "dummy test"() {
given:'A mock for HTTP Builder'
def mockHTTBuilder = Mock(HTTPBuilder)
and:'Spy on the constructor and return the mock object every time'
GroovySpy(HTTPBuilder, global: true)
new HTTPBuilder(_) >> mockHTTBuilder
and:'Create object under test'
def to_test = new totest()
when:'The object is used to get the HTTP result'
def result = to_test.get()
then:'The get method is called once on HTTP Builder'
1 * mockHTTBuilder.get(_) >> { "hello"}
then:'The object under test returns the expected value'
result == 'hello'
}
}
What are you testing here? Do you care how the method gets it's result? Surely you care more that it gets the right result? In that case, the method should be changed so the URL is configurable, then you can stand up a server that returns a known string, and check that string is returned

Is it possible to import a class defined in a .groovy script from a GroovyTestCase?

I have a groovy script defined, which includes an inner class
foo.groovy:
// does something
Bar {
int foobar() {
return 1
}
}
// Does something else
Why I have the class defined in the script is a longer story, but it's necessary, unless I want to make a big redesign of the project structure (And the customer doesn't want to pay for this time)
Anyway, I have a GroovyTestCase, where I want to call this class from.
I want to do something like this:
class Test extends GroovyTestCase {
void testSomething() {
Bar bar = new Bar()
assertTrue(1, bar.foobar())
}
}
Is it possible to reference a class defined in a groovy script? If so, how?
foo.groovy
println "foooo 01"
class Bar {
int foobar() {
return 1
}
}
println "foooo 02"
Test.groovy (you can make it as a class)
def shell = new GroovyShell()
//parse script
def script = shell.parse(new File("/11/foo.groovy"))
//get class loader used for script parsing
def cl = shell.getClassLoader()
//get list of parsed/loaded classes
println cl.getLoadedClasses() // [class Bar, class foo]
def bar = cl.loadClass('Bar').newInstance()
println bar.foobar() // 1
but beware! if your foo.groovy script loaded some other classes that you want to access in testcase, you have to do it through the same shell classloader, otherwise there could be strange errors like could not assign String to String...

Mock static method with GroovyMock or similar in Spock

First-timer here, apologies if I've missed anything.
I'm hoping to get around a call to a static method using Spock. Feedback would be great
With groovy mocks, I thought I'd be able to get past the static call but haven't found it.
For background, I'm in the process of retrofitting tests in legacy java. Refactoring is prohibited. I'm using spock-0.7 with groovy-1.8.
The call to the static method is chained with an instance call in this form:
public class ClassUnderTest{
public void methodUnderTest(Parameter param){
//everything else commented out
Thing someThing = ClassWithStatic.staticMethodThatReturnsAnInstance().instanceMethod(param);
}
}
staticMethod returns an instance of ClassWithStatic
instanceMethod returns the Thing needed in the rest of the method
If I directly exercise the global mock, it returns the mocked instance ok:
def exerciseTheStaticMock(){
given:
def globalMock = GroovyMock(ClassWithStatic,global: true)
def instanceMock = Mock(ClassWithStatic)
when:
println(ClassWithStatic.staticMethodThatReturnsAnInstance().instanceMethod(testParam))
then:
interaction{
1 * ClassWithStatic.staticMethodThatReturnsAnInstance() >> instanceMock
1 * instanceMock.instanceMethod(_) >> returnThing
}
}
But if I run the methodUnderTest from the ClassUnderTest:
def failingAttemptToGetPastStatic(){
given:
def globalMock = GroovyMock(ClassWithStatic,global: true)
def instanceMock = Mock(ClassWithStatic)
ClassUnderTest myClassUnderTest = new ClassUnderTest()
when:
myClassUnderTest.methodUnderTest(testParam)
then:
interaction{
1 * ClassWithStatic.staticMethodThatReturnsAnInstance() >> instanceMock
1 * instanceMock.instanceMethod(_) >> returnThing
}
}
It throws down a real instance of ClassWithStatic that goes on to fail in its instanceMethod.
Spock can only mock static methods implemented in Groovy. For mocking static methods implemented in Java, you'll need to use a tool like GroovyMock , PowerMock or JMockit.
PS: Given that these tools pull of some deep tricks in order to achieve their goals, I'd be interested to hear if and how well they work together with tests implemented in Groovy/Spock (rather than Java/JUnit).
Here is how I solved my similar issue (mocking a static method call which is being called from another static class) with Spock (v1.0) and PowerMock (v1.6.4)
import org.junit.Rule
import org.powermock.core.classloader.annotations.PowerMockIgnore
import org.powermock.core.classloader.annotations.PrepareForTest
import org.powermock.modules.junit4.rule.PowerMockRule
import spock.lang.Specification
import static org.powermock.api.mockito.PowerMockito.mockStatic
import static org.powermock.api.mockito.PowerMockito.when
#PrepareForTest([YourStaticClass.class])
#PowerMockIgnore(["javax.xml.*", "ch.qos.logback.*", "org.slf4j.*"])
class YourSpockSpec extends Specification {
#Rule
Powermocked powermocked = new Powermocked();
def "something something something something"() {
mockStatic(YourStaticClass.class)
when: 'something something'
def mocked = Mock(YourClass)
mocked.someMethod(_) >> "return me"
when(YourStaticClass.someStaticMethod(xyz)).thenReturn(mocked)
then: 'expect something'
YourStaticClass.someStaticMethod(xyz).someMethod(abc) == "return me"
}
}
The #PowerMockIgnore annotation is optional, only use it if there is some conflicts with existing libraries
A workaround would be to wrap the static method call into an instance method.
class BeingTested {
public void methodA() {
...
// was:
// OtherClass.staticMethod();
// replaced with:
wrapperMethod();
...
}
// add a wrapper method for testing purpose
void wrapperMethod() {
OtherClass.staticMethod();
}
}
Now you can use a Spy to mock out the static method.
class BeingTestedSpec extends Specification {
#Subject BeingTested object = new BeingTested()
def "test static method"() {
given: "a spy into the object"
def spyObject = Spy(object)
when: "methodA is called"
spyObject.methodA()
then: "the static method wrapper is called"
1 * spyObject.wrapperMethod() >> {}
}
}
You can also stub in canned response for the wrapper method if it's supposed to return a value. This solution uses only Spock built-in functions and works with both Java and Groovy classes without any dependencies on PowerMock or GroovyMock.
The way I've gotten around static methods in Groovy/Spock is by creating proxy classes that are substituted out in the actual code. These proxy classes simply return the static method that you need. You would just pass in the proxy classes to the constructor of the class you're testing.
Thus, when you write your tests, you'd reach out to the proxy class (that will then return the static method) and you should be able to test that way.
I have recently found 'spock.mockfree' package, it helps mocking final classes and static classes/methods.
It is quite simple as with this framework, in this case, you would need only to Spy() the class under test and #MockStatic the static method you need.
Example:
We used a static method returnA of StaticMethodClass class
public class StaticMethodClass {
public static String returnA() {
return "A";
}
}
here is the calling code
public class CallStaticMethodClass {
public String useStatic() {
return StaticMethodClass.returnA();
}
}
Now we need to test the useStatic method of CallStaticMethodClass class But spock itself does not support mock static methods, and we support
class CallStaticMethodClassTest extends Specification {
def 'call static method is mocked method'() {
given:
CallStaticMethodClass callStaticMethodClass = Spy()
println("useStatic")
expect:
callStaticMethodClass.useStatic() == 'M'
}
#MockStatic(StaticMethodClass)
public static String returnA() {
return "M";
}
}
We use the #MockStatic annotation to mark which class needs to be mocked
Directly implement the static method that requires mocking under it, the method signature remains the same, but the implementation is different.
Link to the framework:
https://github.com/sayweee/spock-mockfree/blob/498e09dc95f841c4061fa8224fcaccfc53904c67/README.md