From the docs :
cppyy is an automatic, run-time, Python-C++ bindings generator, for
calling C++ from Python and Python from C++.
(Emphasis mine)
I don't see any instructions for doing the same, however, so is it possible to call Python via C++ using cppyy?
As a qualifier, since I don't know from where you obtained cppyy, the main code that was at the time the reason for typing that sentence does not exist in cppyy master, but does exist in its historic home of PyROOT. This so-called "class generator" plugin allows Cling to "see" Python classes as C++ classes, for straightforward callbacks and even inheritance of C++ classes from Python ones. See this publication (page 3) for some examples: https://www.researchgate.net/publication/288021806_Python_in_the_Cling_World
This code was not ported over to cppyy standalone b/c the class generator relies on interactive use (specifically, dynamic scopes), so only works from Cling, not compiled code, and there is no way (yet) to drop into the Cling prompt from Python (vice versa works).
The reason why that sentence is still there even without the class generator, is that cppyy has since grown a multitude of other ways to call Python from C++ (these have been backported into PyROOT). Examples include C-style function pointers, C++ std::function<> objects, lambdas, and cross-language inheritance. Moreover, these can all be used by importing cppyy into embedded Python (and thus be used from compiled C++).
See e.g. these examples in the documentation: callbacks and cross-inheritance. Cross-inheritance is probably the easiest to use: just define an abstract interface, implement it in Python, pass the pointer to C++ and use it like you would with any pointer-to-interface in C++. For callbacks, declare an extern function pointer (or std::function object) in a header, pass that in a cppyy.include in embedded Python, assign a Python function to that pointer, then call it in C++ as desired.
Callbacks can be made quite sophisticated, assuming that the C++ side can handle it. For example, by providing annotations on the Python side, Python functions can instantiate C++ function pointer templates. Take the completely generic callback in C++ below, which accepts any arguments and producing any result:
>>> import cppyy
>>> cppyy.cppdef("""\
... template<typename R, typename... U, typename... A>
... R callT(R(*f)(U...), A&&... a) {
... return f(a...);
... }""")
True
>>> def f(a: 'int') -> 'double':
... return 3.1415*a
...
>>> cppyy.gbl.callT(f, 2)
6.283
>>> def f(a: 'int', b: 'int') -> 'int':
... return 3*a*b
...
>>> cppyy.gbl.callT(f, 6, 7)
126
>>>
The final way of calling from C++ into Python is indeed not documented b/c it is (still) only available for CPython/cppyy, not PyPy/_cppyy and the naming is implementation-specific as well: CPyCppyy/API.h.
This header is meant to be included in C++ code, allowing the boxing and unboxing of cppyy-bound instances from C++, custom converters and executors, memory management, and parameter packing for stub functions. There are also a couple of convenience functions for dealing with one-offs. For example:
import cppyy
def pyfunc():
return 42
cppyy.cppdef("""\
#include "CPyCppyy/API.h"
int cppfunc() {
return (int)CPyCppyy::Eval("pyfunc()");
}""")
print(cppyy.gbl.cppfunc())
(although the example here is run from Python for convenience, this can all be called from embedded Python in compiled C++ as well).
I do not know if it answer to your question (from your title I understand that you want to call python function from c++ side, but later it seems you ask specifically about cpppy) but you can do the binding with pybind11. This package is heavily used and is being used in a lot of case as an alternative to swig (to understand the differences have a look at this open thread in the TensorFlow community). These are the two most used packages for Python-C++ binding.
To see how to call a python function in C++ for Pybind11 have a look at this question, while to see how to call a python function in C++ for swig have a look at this question.
Related
Say I have a function
def pyfunc():
print("ayy lmao")
return 4
and I want to call it in c++
int j = (int)python.pyfunc();
how exactly would I do that?
You might want to have a look into this:https://docs.python.org/2/extending/extending.html
In order to call a Python function from C++, you have to embed Python
in your C++ application. To do this, you have to:
Load the Python DLL. How you do this is system dependent:
LoadLibrary under Windows, dlopen under Unix. If the Python DLL is
in the usual path you use for DLLs (%path% under Windows,
LD_LIBRARY_PATH under Unix), this will happen automatically if you try
calling any function in the Python C interface. Manual loading will
give you more control with regards to version, etc.
Once the library has been loaded, you have to call the function
Py_Initialize() to initialize it. You may want to call
Py_SetProgramName() or Py_SetPythonHome() first to establish the
environment.
Your function is in a module, so you'll have to load that:
PyImport_ImportModule. If the module isn't in the standard path,
you'll have to add its location to sys.path: use
PyImport_ImportModule to get the module "sys", then
PyObject_GetAttrString to get the attribute "path". The path
attribute is a list, so you can use any of the list functions to add
whatever is needed to it.
Your function is an attribute of the module, so you use
PyObject_GetAttrString on the module to get an instance of the
function. Once you've got that, you pack the arguments into a tuple or
a dictionary (for keyword arguments), and use PyObject_Call to call
it.
All of the functions, and everything that is necessary, is documented
(extremely well, in fact) in https://docs.python.org/2/c-api/. You'll
be particularly interested in the sections on "Embedding Python" and
"Importing Modules", along with the more general utilities ("Object
Protocol", etc.). You'll also need to understand the general principles
with regards to how the Python/C API works—things like reference
counting and borrowed vs. owned references; you'll probably want to read
all of the sections in the Introduction first.
And of course, despite the overall quality of the documentation, it's
not perfect. A couple of times, I've had to plunge into the Python
sources to figure out what was going on. (Typically, when I'm getting
an error back from Python, to find out what it's actually complaining
about.)
The Python function exec() is a very good tool for programming.
For example:
/******************************************************
* we have array names={"ali","hamed"}
* goal is declare string ali="ali" and hamed="hamed"
******************************************************/
Python code:
# used for command
# below line is a syntax python
# in python we dont declare variable
list=["ali","hamed"] #python syntax
#in python we dont wite {
#python work with tab insteed of {}
for i in list:
temp = i + ' = "' + i + '"' #temp is a string
exec(temp)
My question: Is there a similar function exists in C++? If not, how to implement it in C++?
Python's exec, and similar features in other languages (i.e. eval in JavaScript) only work because those are interpreted languages where variables are defined at run time.
You can't do what you're asking in directly C++ because it's a compiled language that requires that all variables and their names are known at compile time.
You can do it, but it's not trivial, and it will only work on
systems where a C++ compiler is installed. Basically, C++ is
compiled, so if you want to execute some string which is C++,
you'll have to compile it: write it to a file (probably with
additional boilerplate like some includes), invoke the compiler
(with system) to build a DLL, and then load the DLL.
In practice, even if you went to all that effort, you'd probably
find it less useful than you think. Because C++ requires static
declarations for just about everything, it's much harder to
write small snippets like that and have them usable. (You
would, for example, have to wrap it in a function, and generate
extern for all of the variables it uses.) C++ doesn't support
this sort of thing for software engineering reasons; the very
features which make languages like Python so flexible for small,
quickly written programs, cause severe maintenance problems when
used in large scale robust software. Different languages are
designed for different purposes. It's very easy to create
a mixed language system using Python for the top level glue
(where you would want a facility such as you describe), and
calling into C++ for the heavy work (where such a facility would
allow the user to trigger core dumps at will).
Alternatively, you can implement a small language in C++. For
keeping user defined variables, for example, use std::map,
rather than declaring the variables. (That is, after all, what
Python does under the hood.)
If you are planning to use strings as parameters,it is NOT a good idea!
you can simply use an id generator function and give an id to every string that you are gonna use ,or if you are using a class you can put that function in the constructor.
no need to that exec() at all!!
As I understand the relationship between C and C++, the latter is essentially an extension of the former and retains a certain degree of backwards compatibility. Is it safe to assume that the python C API can be called with C++ code?
More to the point, I notice that the official python documentation bundles C and C++ extensions together on the same page. Nowhere am I able to find a C++ API. This leads me to believe that the same API is safe to use in both languages.
Can someone confirm or deny this?
EDIT:
I think I made my question much more complicated than it needs to be. The question is this: what must I do in order to write a python module in C++? Do I just follow the same directions as listed here, substituting C code for C++? Is there a separate API?
I can confirm that the same Python C API is safe to be used in both languages, C and C++.
However, it is difficult to provide you with more detailed answer, unless you will ask more specific question. There are numerous caveats and issues you should be aware of. For example, your Python extensions are defined as C types struct, not as C++, so don't expect to have their constructor/destructor implicitly defined and called.
For example, taking the sample code from Defining New Types in the Python manual, it can be written in C++ way and you can even blend-in C++ types:
// noddy.cpp
namespace {
struct noddy_NoddyObject
{
PyObject_HEAD
// Type-specific fields go here.
std::shared_ptr<int> value; // WARNING
};
PyObject* Noddy_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
try {
Noddy *self = (Noddy *)type->tp_alloc(type, 0);
if (self) {
self->value = std::make_shared(7);
// or more complex operations that may throw
// or extract complex initialisation as Noddy_init function
return self;
}
}
catch (...) {
// do something, log, etc.
}
return 0;
}
PyTypeObject noddy_NoddyType =
{
PyObject_HEAD_INIT(NULL)
// ...
}
} // unnamed namespace
But, neither constructor nor destructor of the std::shared_ptr will be called.
So, remember to define dealloc function for your noddy_NoddyType where you will reset the value with nullptr. Why even bother with having value defined as shared_ptr, you may ask. It is useful if you use your Python extension in C++, with exceptions, to avoid type conversions and casts, to have more seamless integration inside definitions of your implementation, error handling based on exception may be easier then, etc.
And in spite of the fact that your objects of the noddy_NoddyType are managed by machinery implemented in pure C, thanks to dealloc function the value will be released according to well-known RAII rules.
Here you can find interesting example of nearly seamless integration of Python C API with the C++ language: How To catch Python stdout in c++ code
Python C API can be called within C++ code.
Python C++ extensions are written using the same C API as C extensions use, or using some 3rd party API, such as boost::python.
I have a task to interface with a dll from a third party company using C++.
The dll package comes with:
the dll itself
a sample Java implementation - consists of a java wrapper(library) generated using the SWIG tool and the the java source file
documentation that states all the public datatypes, enumeration and member functions.
My other colleague is using Java(based on the example in package) to interface with the dll while I'm asked to use C++. The Java example looks straight forward... just import the wrapper and instantiate any class described in the docs..
More info on the dll:
From the docs, it says the dll was programmed using C++
From a hexdump, it shows that it was compiled using VC90 (VS C++ 2008 right?) and something from Dinkumware.
From a depends.exe output, the functions seems to be wrapped under JNI. For example: _Java_mas_com_oa_rollings_as_apiJNI_Server_1disconnect#20
My dilemma:
The dll company is not changing anything in the dll and not providing any other info.
How do i use the member functions in the class from the dll?
I did some simple LoadLibrary() and GetProcAddress and manage to get the address of the public member functions.
But i dunno how to use the functions that has the datatype parameters defined in the dll. For example:
From the docs, the member function is defined as:
void Server::connect(const StringArray, const KeyValueMap) throw(std::invalid_argument,std::out_of_range)
typedef std::map Server::KeyValueMap
typedef std::vector Server::StringArray
how do i call that function in C++. The std::map and std::vector in my compiler (VS 2005) has different functions listing that the one in the dll. For example, from the depends.exe output:
std::map // KeyValueMap - del, empty, get, has_1key,set
std::vector // StringArray - add, capacity, clear, get, isEMPTY, reserve, set, size
Any advice/strategy on how i should solve this? Is it possible to simply instantiate the class like the Java example?
If you are trying to use VS 2005 to try and interface with a DLL that is built using VS2008, your attempts will be mostly doomed unless you can use a plain C interface. Given your description, this is not the case; The runtime libraries differ between VS2005 and VS2008 so there is little chance that the object layout has stayed the same between compilers. The 'something from Dinkumware' that you're referring to is most likely the C++ standard library as ISTR that Microsoft uses the Dinkumware one.
With your above example you're also missing several important pieces of information - the types you describe (Server::StringArray and Server::KeyValueMap) are standard library containers. OK fine, but standard library containers of what? These containers are templates and unless you know the exact types these templates have been instantiated with, you're a little stuck.
Is this DLL intended to be called from C++ at all? The fact that it export a JNI interface suggests that it might not be in the first place. Does it export any other public symbols apart from those that are of the format _Java_...?
Of course if there is no other way in and you must use C++ instead of Java, you might want to look into embedding a JVM into your C++ app and use that to call through to the C++ dll. It's not what I'd call an elegant solution but it might well work.
I don't quite understand the use of C++ standard library data types here. How can Java code provide a std::map argument? Are the arguments you pass in always just "opaque" values you would get as output from a previous call to the library? That's the only way you're going to be able to make it work from code under a different runtime.
Anyway...
When you make a JNI module, you run javah.exe and it generates a header file with declarations like:
JNIEXPORT void JNICALL Java_Native_HelloWorld(JNIEnv *, jobject);
Do you have any such header file for the module?
These symbols are exported as extern "C" if I recall correctly, so if you can get the correct signatures, you should have no issues with name mangling or incompatible memory allocators, etc..
The "#20" at the end of the method signature means that the function is declared "stdcall" and that 20 bytes are put on the stack when the function is called. All these methods should start with a JNIEnv* and a jobject, these will total 8 bytes I believe, on a 32-bit environment, so that leaves 12 bytes of parameters you will need to know in order to generate a correct function prototype.
Once you figure out what the parameters are, you can generate something like this:
typedef void (__stdcall *X)(JNIEnv *, jobject, jint i, jboolean b);
Then, you can cast the result of GetProcAddress to an X and call it from your C++ code.
X x = (X)GetProcAddress(module, "name");
if (x) x(params...);
Unfortunately, what you have doesn't quite look like what I have seen in the past. I am used to having to deal with Java data types from C/C++ code, but it looks like this module is dealing with C++ data types in Java code, so I don't know how relevant any of my experience is. Hopefully this is some help, at least.
I have a program in C++ that uses the cryptopp library to decrypt/encrypt messages.
It offers two interface methods encrypt & decrypt that receive a string and operate on it through cryptopp methods.
Is there some way to use both methods in Python without manually wrapping all the cryptopp & files included?
Example:
import cppEncryptDecrypt
string foo="testing"
result = encrypt(foo)
print "Encrypted string:",result
If you can make a DLL from that C++ code, exposing those two methods (ideally as "extern C", that makes all interfacing tasks so much simpler), ctypes can be the answer, not requiring any third party tool or extension. Otherwise, it's your choice between cython, good old SWIG, SIP, Boost, ... -- many, many such 3rd party tools will let your Python code call those two C++ entry points without any need for wrapping anything else but them.
As Alex suggested you can make a dll, export the function you want to access from python and use ctypes(http://docs.python.org/library/ctypes.html) module to access e.g.
>>> libc = cdll.LoadLibrary("libc.so.6")
>>> printf = libc.printf
>>> printf("Hello, %s\n", "World!")
Hello, World
or there is alternate simpler approach, which many people do not consider but is equally useful in many cases i.e. directly call the program from command line. You said you have already working program, so I assume it does both encrypt/decrypt from commandline? if yes why don't you just call the program from os.system, or subprocess module, instead of delving into code and changing it and maintaining it.
I would say go the second way unless it can't fulfill your requirements.