I'm working on embedding Python in our test suite application. The purpose is to use Python to run several tests scripts to collect data and make a report of tests. Multiple test scripts for one test run can create global variables and functions that can be used in the next script.
The application also provides extension modules that are imported in the embedded interpreter, and are used to exchange some data with the application.
But the user can also make multiple test runs. I don't want to share those globals, imports and the exchanged data between multiple test runs. I have to be sure I restart in a genuine state to control the test environment and get the same results.
How should I reinitialise the interpreter?
I used Py_Initialize() and Py_Finalize(), but get an exception on the second run when initialising a second time the extension modules I provide to the interpreter.
And the documentation warns against using it more than once.
Using sub-interpreters seems to have the same caveats with extension modules initialization.
I suspect that I'm doing something wrong with the initialisation of my extension modules, but I fear that the same problem happens with 3rd party extension modules.
Maybe it's possible to get it to work by launching the interpreter in it's own process, so as to be sure that all the memory is released.
By the way, I'm using boost-python for it, that also warns AGAINST using Py_Finalize!
Any suggestion?
Thanks
Here is another way I found to achieve what I want, start with a clean slate in the interpreter.
I can control the global and local namespaces I use to execute the code:
// get the dictionary from the main module
// Get pointer to main module of python script
object main_module = import("__main__");
// Get dictionary of main module (contains all variables and stuff)
object main_namespace = main_module.attr("__dict__");
// define the dictionaries to use in the interpreter
dict global_namespace;
dict local_namespace;
// add the builtins
global_namespace["__builtins__"] = main_namespace["__builtins__"];
I can then use use the namespaces for execution of code contained in pyCode:
exec( pyCode, global_namespace, lobaca_namespace );
I can clean the namespaces when I want to run a new instance of my test, by cleaning the dictionaries:
// empty the interpreters namespaces
global_namespace.clear();
local_namespace.clear();
// Copy builtins to new global namespace
global_namespace["__builtins__"] = main_namespace["__builtins__"];
Depending at what level I want the execution, I can use global = local
How about using code.IteractiveInterpreter?
Something like this should do it:
#include <boost/python.hpp>
#include <string>
#include <stdexcept>
using namespace boost::python;
std::string GetPythonError()
{
PyObject *ptype = NULL, *pvalue = NULL, *ptraceback = NULL;
PyErr_Fetch(&ptype, &pvalue, &ptraceback);
std::string message("");
if(pvalue && PyString_Check(pvalue)) {
message = PyString_AsString(pvalue);
}
return message;
}
// Must be called after Py_Initialize()
void RunInterpreter(std::string codeToRun)
{
object pymodule = object(handle<>(borrowed(PyImport_AddModule("__main__"))));
object pynamespace = pymodule.attr("__dict__");
try {
// Initialize the embedded interpreter
object result = exec( "import code\n"
"__myInterpreter = code.InteractiveConsole() \n",
pynamespace);
// Run the code
str pyCode(codeToRun.c_str());
pynamespace["__myCommand"] = pyCode;
result = eval("__myInterpreter.push(__myCommand)", pynamespace);
} catch(error_already_set) {
throw std::runtime_error(GetPythonError().c_str());
}
}
I'd write another shell script executing the sequence of test scripts with new instances of python each time. Or write it in python like
# run your tests in the process first
# now run the user scripts, each in new process to have virgin env
for script in userScript:
subprocess.call(['python',script])
Related
I have a requirement where I need to embed Ruby code in C++ (I could achieve this) and extend some functionality. When I try to call (require) my extension module, the program fails.
I have pasted a jist of the code below:
main.cpp
#include <ruby.h>
static VALUE Sum(VALUE rb_self, VALUE rb_param1, VALUE rb_param2)
{
double val1 = NUM2DBL(rb_param1);
double val2 = NUM2DBL(rb_param2);
return rb_float_new(val1+val2);
}
void Init_myMod()
{
myMod = rb_define_module("myMod");
rb_define_module_function(myMod, "Sum", Sum, 2);
}
int main()
{
... Init ruby method...
rb_require("myRubyfile");
Init_myMod(); // This is where I need to push my ruby extension.
rb_funcall(rb_mKernel, rb_intern("helloworld"), 0 , NULL);
....
}
myRubyfile.rb
require "myMod"
def helloworld()
puts "Hello world"
#puts "Sum of.. " + myMod.sum(4, 5)
end
The problem is if I dont use require "myMod" the code works and prints "Hello World", but if I use require.., the code dumps.
This is my requirement where I need to embed the ruby script in C, and reuse some of the C methods in ruby (extension).
Can someone help me with this?
You're getting a segfault because the Ruby VM segfaults when there is an uncaught exception. You need to use something like rb_rescue or rb_protect to wrap the code which could raise an exception.
require "myMod" is raising an exception because require is only used to load files. You don't need to require anything because you're creating the module directly in the VM's memory. So... why the error? Well, if you were to write your module in Ruby, you'd get a syntax error:
module myMod
...
end
# SyntaxError: class/module name must be CONSTANT
But when you use rb_define_module, rather than raise an exception, Ruby does a tricky thing: it creates the module but doesn't expose it to the Ruby code. In order to expose the module to Ruby, just make the name valid:
// .cpp
rb_define_module("MyMod");
and
# .rb
puts "Sum of.. " + MyMod.Sum(4, 5).to_s
Something like this should work. Make a bat script to your ruby script file and change system to the appropriate system call. Not sure if system() works in Windows. I have run many bash scripts this way so I would assume you can do the same thing in Windows.
#include <stdio.h>
#include <stdlib.h>
int main()
{
puts("Starting now:");
system("/path_to_script/script_name");
return 0;
}
I'm using boost::python to do a hybrid C++/python application: the C++ app calls a collection of python scripts, which in turn use the C++ program's functions, classes, etc., exposed as python objects. (Python 2.x.)
BOOST_PYTHON_MODULE(MyModule) exposes the C++ to python as expected.
My initialization code:
Py_Initialize();
initMyModule(); // import MyModule
namespace bpl = boost::python;
Now I want my C++ code to get at MyModule, too. In python, you just write globals()['MyModule']. But this (and things like it) don't work in C++:
bpl::object globals = bpl::eval("globals()");
This fails at run-time with
File "<string>", line 1, in <module>; NameError: name 'globals' is not defined
As an aside, I see many examples of setting up __main__ like this:
bpl::object m = bpl::import("__main__");
bpl::dict g = m.attr("__dict__"); // like locals(), but not globals()
This doesn't fail, and gives locals, but according to the Py_Initialize docs, __main__ is already set up. And it doesn't let you see globals, where you'd find your imported module.
You don't need the explicit bpl::import("__main__");.
Here are the globals:
bpl::dict globals()
{
bpl::handle<> mainH(bpl::borrowed(PyImport_GetModuleDict()));
return bpl::extract<bpl::dict>(bpl::object(mainH));
}
Since everything is managed by smart pointers, returning and manipulating bpl::dict directly works fine.
bpl::object myMod = globals()["MyModule"];
globals()["myNewGlobal"] = 88;
I am trying to embed Lua into a game. What I want to do is to create a load function, which will load all files in a folder and then create objects based off those files which will be stored on the C++ side. However, if I use something like dofile, it pollutes everything with the variables that are in that file. How do I change this import to be local?
You can use loadfile to get the function based on the file content, then setfenv(fn,{}) to set the environment, and then call that function (probably wrapped into pcall):
local fn, err = loadfile("myfile")
if fn then
setfenv(fn,{})
local ok, err = pcall(fn)
if not ok then error(err) end
else
error(err)
end
You can also populate the table you pass to setfenv with the values you need the script to have access to or provide access to the global environment with something like:
local env = {}
setmetatable(env,{__index = _G})
setfenv(fn,env)
This is all for Lua 5.1.
I am embedding Lua in a C++ application.
I have some modules (for now, simple .lua scripts) that I want to load programmatically, as the engine is being started, so that when the engine starts, the module(s) is/are available to scripts without them having to include a require 'xxx' at the top of the script.
In order to do this, I need to be able to programmatically (i.e. C++ end), ask the engine to load the modules, as part of the initialisation (or shortly thereafter).
Anyone knows how I can do this?
Hmm, I just use the simple approach: My C++ code just calls Lua's require function to pre-load the Lua scripts I want preloaded!
// funky = require ("funky")
//
lua_getfield (L, LUA_GLOBALSINDEX, "require"); // function
lua_pushstring (L, "funky"); // arg 0: module name
err = lua_pcall (L, 1, 1, 0);
// store funky module table in global var
lua_setfield (L, LUA_GLOBALSINDEX, "funky");
// ... later maybe handle a non-zero value of "err"
// (I actually use a helper function instead of lua_pcall
// that throws a C++ exception in the case of an error)
If you've got multiple modules to load, of course, put it in a loop... :)
The easiest way is to add and edit a copy of linit.c to your project.
You can add a property to a class using a getter and a setter (in a simplistic case):
class<X>("X")
.add_property("foo", &X::get_foo, &X::set_foo);
So then you can use it from python like this:
>>> x = mymodule.X()
>>> x.foo = 'aaa'
>>> x.foo
'aaa'
But how to add a property to a module itself (not a class)?
There is
scope().attr("globalAttr") = ??? something ???
and
def("globalAttr", ??? something ???);
I can add global functions and objects of my class using the above two ways, but can't seem to add properties the same way as in classes.
__getattr__ and __setattr__ aren't called on modules, so you can't do this in ordinary Python without hacks (like storing a class in the module dictionary). Given that, it's very unlikely there's an elegant way to do it in Boost Python either.
boost.python/HowTo on Python Wiki has an example of exposing C++ object as a module attribute inside BOOST_PYTHON_MODULE:
namespace bp = boost::python;
BOOST_PYTHON_MODULE(example)
{
bp::scope().attr("my_attr") = bp::object(bp::ptr(&my_cpp_object));
}
To set the attribute outside of BOOST_PYTHON_MODULE use
bp::import("example").attr("my_attr") = bp::object(bp::ptr(&my_cpp_object));
Now you can do in python something like
from example import my_attr
Of course, you need to register class of my_cpp_object in advance (e.g. you can do this inside the same BOOST_PYTHON_MODULE call) and ensure C++ object lifetime exceeds that of the python module. You can use any bp::object instead of wrapping C++ one.
Note that BOOST_PYTHON_MODULE swallows exceptions, so if you make a mistake, you don't receive any error indication and BOOST_PYTHON_MODULE-generated function will just immediately return. To ease debugging of this case you can catch exceptions inside BOOST_PYTHON_MODULE or temporary add some logging statement as a last line of BOOST_PYTHON_MODULE to see that it is reached:
BOOST_PYTHON_MODULE(example)
{
bp::scope().attr("my_attr2") = new int(1); // this will compile
std::cout << "init finished\n"; // OOPS, this line will not be reached
}