In comparing documentation for itertools between Python 2 and 3, I noticed ifilter, imap, izip are missing from Python 3. I suspect this is because many builtin keywords have been converted to generators and replaced former keywords, but it is unclear in this case.
Is it true ifilter, imap, izip are now equivalent to filter, map, zip in Python 3? If not, where can I find rationales for why certain methods were removed from current itertools?
Python 2.3 introduced the itertools module, which defined variants of the global zip(), map(), and filter() functions that returned iterators instead of lists. In Python 3, those global functions return iterators, so those functions in the itertools module have been eliminated.
Instead of itertools.izip(), just use the global zip() function.
Instead of itertools.imap(), just use map().
itertools.ifilter() becomes filter().
The itertools module still exists in Python 3, it just doesn’t have the functions that have migrated to the global namespace. The 2to3 script is smart enough to remove the specific imports that no longer exist, while leaving other imports intact.
Read more here
Related
From the docs :
cppyy is an automatic, run-time, Python-C++ bindings generator, for
calling C++ from Python and Python from C++.
(Emphasis mine)
I don't see any instructions for doing the same, however, so is it possible to call Python via C++ using cppyy?
As a qualifier, since I don't know from where you obtained cppyy, the main code that was at the time the reason for typing that sentence does not exist in cppyy master, but does exist in its historic home of PyROOT. This so-called "class generator" plugin allows Cling to "see" Python classes as C++ classes, for straightforward callbacks and even inheritance of C++ classes from Python ones. See this publication (page 3) for some examples: https://www.researchgate.net/publication/288021806_Python_in_the_Cling_World
This code was not ported over to cppyy standalone b/c the class generator relies on interactive use (specifically, dynamic scopes), so only works from Cling, not compiled code, and there is no way (yet) to drop into the Cling prompt from Python (vice versa works).
The reason why that sentence is still there even without the class generator, is that cppyy has since grown a multitude of other ways to call Python from C++ (these have been backported into PyROOT). Examples include C-style function pointers, C++ std::function<> objects, lambdas, and cross-language inheritance. Moreover, these can all be used by importing cppyy into embedded Python (and thus be used from compiled C++).
See e.g. these examples in the documentation: callbacks and cross-inheritance. Cross-inheritance is probably the easiest to use: just define an abstract interface, implement it in Python, pass the pointer to C++ and use it like you would with any pointer-to-interface in C++. For callbacks, declare an extern function pointer (or std::function object) in a header, pass that in a cppyy.include in embedded Python, assign a Python function to that pointer, then call it in C++ as desired.
Callbacks can be made quite sophisticated, assuming that the C++ side can handle it. For example, by providing annotations on the Python side, Python functions can instantiate C++ function pointer templates. Take the completely generic callback in C++ below, which accepts any arguments and producing any result:
>>> import cppyy
>>> cppyy.cppdef("""\
... template<typename R, typename... U, typename... A>
... R callT(R(*f)(U...), A&&... a) {
... return f(a...);
... }""")
True
>>> def f(a: 'int') -> 'double':
... return 3.1415*a
...
>>> cppyy.gbl.callT(f, 2)
6.283
>>> def f(a: 'int', b: 'int') -> 'int':
... return 3*a*b
...
>>> cppyy.gbl.callT(f, 6, 7)
126
>>>
The final way of calling from C++ into Python is indeed not documented b/c it is (still) only available for CPython/cppyy, not PyPy/_cppyy and the naming is implementation-specific as well: CPyCppyy/API.h.
This header is meant to be included in C++ code, allowing the boxing and unboxing of cppyy-bound instances from C++, custom converters and executors, memory management, and parameter packing for stub functions. There are also a couple of convenience functions for dealing with one-offs. For example:
import cppyy
def pyfunc():
return 42
cppyy.cppdef("""\
#include "CPyCppyy/API.h"
int cppfunc() {
return (int)CPyCppyy::Eval("pyfunc()");
}""")
print(cppyy.gbl.cppfunc())
(although the example here is run from Python for convenience, this can all be called from embedded Python in compiled C++ as well).
I do not know if it answer to your question (from your title I understand that you want to call python function from c++ side, but later it seems you ask specifically about cpppy) but you can do the binding with pybind11. This package is heavily used and is being used in a lot of case as an alternative to swig (to understand the differences have a look at this open thread in the TensorFlow community). These are the two most used packages for Python-C++ binding.
To see how to call a python function in C++ for Pybind11 have a look at this question, while to see how to call a python function in C++ for swig have a look at this question.
Say I have a function
def pyfunc():
print("ayy lmao")
return 4
and I want to call it in c++
int j = (int)python.pyfunc();
how exactly would I do that?
You might want to have a look into this:https://docs.python.org/2/extending/extending.html
In order to call a Python function from C++, you have to embed Python
in your C++ application. To do this, you have to:
Load the Python DLL. How you do this is system dependent:
LoadLibrary under Windows, dlopen under Unix. If the Python DLL is
in the usual path you use for DLLs (%path% under Windows,
LD_LIBRARY_PATH under Unix), this will happen automatically if you try
calling any function in the Python C interface. Manual loading will
give you more control with regards to version, etc.
Once the library has been loaded, you have to call the function
Py_Initialize() to initialize it. You may want to call
Py_SetProgramName() or Py_SetPythonHome() first to establish the
environment.
Your function is in a module, so you'll have to load that:
PyImport_ImportModule. If the module isn't in the standard path,
you'll have to add its location to sys.path: use
PyImport_ImportModule to get the module "sys", then
PyObject_GetAttrString to get the attribute "path". The path
attribute is a list, so you can use any of the list functions to add
whatever is needed to it.
Your function is an attribute of the module, so you use
PyObject_GetAttrString on the module to get an instance of the
function. Once you've got that, you pack the arguments into a tuple or
a dictionary (for keyword arguments), and use PyObject_Call to call
it.
All of the functions, and everything that is necessary, is documented
(extremely well, in fact) in https://docs.python.org/2/c-api/. You'll
be particularly interested in the sections on "Embedding Python" and
"Importing Modules", along with the more general utilities ("Object
Protocol", etc.). You'll also need to understand the general principles
with regards to how the Python/C API works—things like reference
counting and borrowed vs. owned references; you'll probably want to read
all of the sections in the Introduction first.
And of course, despite the overall quality of the documentation, it's
not perfect. A couple of times, I've had to plunge into the Python
sources to figure out what was going on. (Typically, when I'm getting
an error back from Python, to find out what it's actually complaining
about.)
Sometimes I want to include a module in some other subroutine but I only need several subroutines from that module. What is the difference between
use a_module, only: a_subroutine
or simply
use a_module
?
Here is a complete answer, some of which has already been discussed in the comments.
From Metcalf et al. (2011) p.146 (a leading Fortran reference textbook), use a_module provides (emphasis added):
access to all the public named data objects, derived types,
interface blocks, procedures, generic identifiers, and namelist groups
in the module named.
Conversely, use a_module, only an_entity provides:
access to an entity in a module only if the entity ... is specified.
i.e. use a_module is equivalent to the not-recommended (e.g. in [2]) python practice:
from a_module import *
while use a_module, only an_entity is equivalent to the preferred python practice:
from a_module import an_entity
Unfortunately, the recommended python practice
import module [as name]
or
import module.submodule [as name]
is not available in Fortran since Fortran imports all entities into a global namespace rather than accessing entities from modules via the module's namespace as done in python, e.g.:
import numpy as np
array = np.array([1, 2, 3])
As noted in the comments and elsewhere (e.g. [3]), explicit imports (use a_module, only an_entity) are preferred over implicit imports (use a_module) for code clarity and to avoid namespace pollution / name clashes ("explicit is better than implicit").
Metcalf et al. (2011) also note that should you require two entities with the same name from different modules, name clashes can be avoided by renaming one (or both) of the clashing entities locally (i.e. within your program / module only), e.g.
use stats_lib, only sprod => prod
use maths_lib, only prod
where prod from stats_lib is accessed locally using the name sprod, while prod from maths_lib is accessed locally using the name prod.
Incidentally, Metcalf et al. (2011) also note:
A name clash is permitted if there is no reference to the name in the
scoping unit.
i.e. you can successfully compile:
use stats_lib
use maths_lib
without problems provided neither module's prod (or any other clashing name) is used in your program / module. However, for the reasons above, such practice is not recommended.
[1] Metcalf, M, Reid, J & Cohen, M. (2011) "Modern Fortran Explained" (Oxford University Press)
[2] https://www.tutorialspoint.com/python/python_modules.htm
[3] http://www.fortran90.org/src/best-practices.html
I need to import one of the core modules (datetime) inside my C extension module since I want to return a datetime.date from some functions of my module.
It appears that Python C extension modules have no complement for the PyMODINIT_FUNC upon destruction.
Question: What can I do short of importing the required module time and time again in every call inside my C extension module and then dereferencing it at the end of the call(s) again?
Rationale: Basically I fear that this (importing it over and over again) creates an unnecessary overhead, because from my understanding of the documentation dereferencing it means that the garbage collector can come around and collect it, so next time PyImport_ImportModule would have to be called again.
Somewhat related questions:
Import and use standard Python module from inside Python C extension
Making a C extension to Python that requires another extension
I have a program in C++ that uses the cryptopp library to decrypt/encrypt messages.
It offers two interface methods encrypt & decrypt that receive a string and operate on it through cryptopp methods.
Is there some way to use both methods in Python without manually wrapping all the cryptopp & files included?
Example:
import cppEncryptDecrypt
string foo="testing"
result = encrypt(foo)
print "Encrypted string:",result
If you can make a DLL from that C++ code, exposing those two methods (ideally as "extern C", that makes all interfacing tasks so much simpler), ctypes can be the answer, not requiring any third party tool or extension. Otherwise, it's your choice between cython, good old SWIG, SIP, Boost, ... -- many, many such 3rd party tools will let your Python code call those two C++ entry points without any need for wrapping anything else but them.
As Alex suggested you can make a dll, export the function you want to access from python and use ctypes(http://docs.python.org/library/ctypes.html) module to access e.g.
>>> libc = cdll.LoadLibrary("libc.so.6")
>>> printf = libc.printf
>>> printf("Hello, %s\n", "World!")
Hello, World
or there is alternate simpler approach, which many people do not consider but is equally useful in many cases i.e. directly call the program from command line. You said you have already working program, so I assume it does both encrypt/decrypt from commandline? if yes why don't you just call the program from os.system, or subprocess module, instead of delving into code and changing it and maintaining it.
I would say go the second way unless it can't fulfill your requirements.