python executing existent (&big) c++ code - c++

I have a program in C++ that uses the cryptopp library to decrypt/encrypt messages.
It offers two interface methods encrypt & decrypt that receive a string and operate on it through cryptopp methods.
Is there some way to use both methods in Python without manually wrapping all the cryptopp & files included?
Example:
import cppEncryptDecrypt
string foo="testing"
result = encrypt(foo)
print "Encrypted string:",result

If you can make a DLL from that C++ code, exposing those two methods (ideally as "extern C", that makes all interfacing tasks so much simpler), ctypes can be the answer, not requiring any third party tool or extension. Otherwise, it's your choice between cython, good old SWIG, SIP, Boost, ... -- many, many such 3rd party tools will let your Python code call those two C++ entry points without any need for wrapping anything else but them.

As Alex suggested you can make a dll, export the function you want to access from python and use ctypes(http://docs.python.org/library/ctypes.html) module to access e.g.
>>> libc = cdll.LoadLibrary("libc.so.6")
>>> printf = libc.printf
>>> printf("Hello, %s\n", "World!")
Hello, World
or there is alternate simpler approach, which many people do not consider but is equally useful in many cases i.e. directly call the program from command line. You said you have already working program, so I assume it does both encrypt/decrypt from commandline? if yes why don't you just call the program from os.system, or subprocess module, instead of delving into code and changing it and maintaining it.
I would say go the second way unless it can't fulfill your requirements.

Related

Calling a python 2.7 function in c++ using the default api?

Say I have a function
def pyfunc():
print("ayy lmao")
return 4
and I want to call it in c++
int j = (int)python.pyfunc();
how exactly would I do that?
You might want to have a look into this:https://docs.python.org/2/extending/extending.html
In order to call a Python function from C++, you have to embed Python
in your C++ application. To do this, you have to:
Load the Python DLL. How you do this is system dependent:
LoadLibrary under Windows, dlopen under Unix. If the Python DLL is
in the usual path you use for DLLs (%path% under Windows,
LD_LIBRARY_PATH under Unix), this will happen automatically if you try
calling any function in the Python C interface. Manual loading will
give you more control with regards to version, etc.
Once the library has been loaded, you have to call the function
Py_Initialize() to initialize it. You may want to call
Py_SetProgramName() or Py_SetPythonHome() first to establish the
environment.
Your function is in a module, so you'll have to load that:
PyImport_ImportModule. If the module isn't in the standard path,
you'll have to add its location to sys.path: use
PyImport_ImportModule to get the module "sys", then
PyObject_GetAttrString to get the attribute "path". The path
attribute is a list, so you can use any of the list functions to add
whatever is needed to it.
Your function is an attribute of the module, so you use
PyObject_GetAttrString on the module to get an instance of the
function. Once you've got that, you pack the arguments into a tuple or
a dictionary (for keyword arguments), and use PyObject_Call to call
it.
All of the functions, and everything that is necessary, is documented
(extremely well, in fact) in https://docs.python.org/2/c-api/. You'll
be particularly interested in the sections on "Embedding Python" and
"Importing Modules", along with the more general utilities ("Object
Protocol", etc.). You'll also need to understand the general principles
with regards to how the Python/C API works—things like reference
counting and borrowed vs. owned references; you'll probably want to read
all of the sections in the Introduction first.
And of course, despite the overall quality of the documentation, it's
not perfect. A couple of times, I've had to plunge into the Python
sources to figure out what was going on. (Typically, when I'm getting
an error back from Python, to find out what it's actually complaining
about.)

How to find a pointer to a function by string

I have a list of functions in a text file that I'd like to expose to LLVM for its execution engine at run time, I'm wondering if its possible to find pointers to the functions at runtime rather than hard code in all the GlobalMappings by hand as I'd probably like to add in more later. For example:
// File: InternalFunctions.txt
PushScreen
PopScreen
TopScreen
// File: ExposeEngine.cpp
// Somehow figure out the address of the function specified in a string
void* addy = magicAddress("PushScreen");
jit->addGlobalMapping(llvmfunction, addy);
If this is possible I love to know how to do it, as I am trying to write my game engine by jit-ing c++. I was able to create some results earlier, but I had to hard-code in the mappings. I noticed that Gtk uses something along the lines of what I'm asking. When you use glade and provide a signal handler, the program you build in c will automatically find the function in your executable referenced by the string provided in the glade file. If getting results requires me to look into this Gtk thing I'd be more than happy to, but I don't know what feature or part of the api deals with that - I've already tried to look it up. I'd love to hear suggestions or advice.
Yes, you can do this. Look at the man pages for dlopen() and dlsym(): these functions are standard on *nix systems and let you look up symbols (functions or variables) by name. There is one significant issue, which is that C++ function names are usually "mangled" to encode type information. A typical way around this is to define a set of wrapper functions in an extern "C" {} block: these will be non-member, C-style functions which can then call into your C++ code. Their names will not be mangled, making them easy to look up using dlsym().
This is a pretty standard way that some plugin architectures work. Or at least used to work, before everyone started using interpreted languages!

how to share a lib between process and called script subprocess using SWIG?

I have a C++ program foobar which starts with main() and then the flow of control goes through a first part, then the second part of the program. If I change main to foobar_main, I can then compile the whole program and a SWIG Python wrapper to a shared library foobar.so, and import this to Python, call foobar_main from within Python and everything works fine.
The second part communicates with the first one by some respectable C++ constructs. Specifically: the first part creates some single objects of some classes, and the second part uses class static methods to get those objects.
Now I want to run only the first part from main() and the second part from Python. That is, I want to start the C++ program foobar and then after the first part is finished, run a Python script (programmatically from within C++) that continues with the second part.
To do this, I:
compile the second part and a SWIG wrapper to foobar2.so
replace the second part of C++ code with system("python foobar2.py")
compile the modified C++ program to foobar1.so and load to foobar
write the script foobar2.py which imports foobar1 and foobar2 and then equivalent to the second part
Then I attempt to run foobar. It does not work, because it appears, that the routines in the second part complain that certain steps which should have been done in the first part, are not done.
This is embarasing but obviously I have some deep flaws here in my understanding of how computers work :) Can somebody clue me in what I am missing, including possibly simplifying the above process?
I'm going to assume your C++ code looks like this:
void part1()
{}
void part2()
{}
int main()
{
part1();
part2();
}
And that you have a Python version of part2() that is implemented with some other wrapped C++ functions. If these assumptions are wrong let me know.
I think the easiest way to go is to wrap part1() along with the other wrapped part2-related functions, then have a Python script like this:
import foobar
foobar.part1()
py_part2()
This of course means that the program starts in Python. If you need to start a C++ program for some reason (i.e. you need main()) then in order to use py_part2() you'll have to embed the Python interpreter inside your C++ program. This is a much more difficult and involved process, this answer has good info about how to get started.
Since you're learning I'll explain why system("python foobar2.py") doesn't work. In this scheme you have your C++ program start another process (program), named python, and then wait for it to finish. These are two completely different programs that in your case don't talk to each other and don't share anything in common. Hence why it doesn't work.
In general, reconsider anything that involves system. Its primary use seems to be to point out beginner programmers.

How to implement Python function exec() in C++?

The Python function exec() is a very good tool for programming.
For example:
/******************************************************
* we have array names={"ali","hamed"}
* goal is declare string ali="ali" and hamed="hamed"
******************************************************/
Python code:
# used for command
# below line is a syntax python
# in python we dont declare variable
list=["ali","hamed"] #python syntax
#in python we dont wite {
#python work with tab insteed of {}
for i in list:
temp = i + ' = "' + i + '"' #temp is a string
exec(temp)
My question: Is there a similar function exists in C++? If not, how to implement it in C++?
Python's exec, and similar features in other languages (i.e. eval in JavaScript) only work because those are interpreted languages where variables are defined at run time.
You can't do what you're asking in directly C++ because it's a compiled language that requires that all variables and their names are known at compile time.
You can do it, but it's not trivial, and it will only work on
systems where a C++ compiler is installed. Basically, C++ is
compiled, so if you want to execute some string which is C++,
you'll have to compile it: write it to a file (probably with
additional boilerplate like some includes), invoke the compiler
(with system) to build a DLL, and then load the DLL.
In practice, even if you went to all that effort, you'd probably
find it less useful than you think. Because C++ requires static
declarations for just about everything, it's much harder to
write small snippets like that and have them usable. (You
would, for example, have to wrap it in a function, and generate
extern for all of the variables it uses.) C++ doesn't support
this sort of thing for software engineering reasons; the very
features which make languages like Python so flexible for small,
quickly written programs, cause severe maintenance problems when
used in large scale robust software. Different languages are
designed for different purposes. It's very easy to create
a mixed language system using Python for the top level glue
(where you would want a facility such as you describe), and
calling into C++ for the heavy work (where such a facility would
allow the user to trigger core dumps at will).
Alternatively, you can implement a small language in C++. For
keeping user defined variables, for example, use std::map,
rather than declaring the variables. (That is, after all, what
Python does under the hood.)
If you are planning to use strings as parameters,it is NOT a good idea!
you can simply use an id generator function and give an id to every string that you are gonna use ,or if you are using a class you can put that function in the constructor.
no need to that exec() at all!!

Why is the C++ syntax so complicated? [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 13 years ago.
I'm a novice at programming although I've been teaching myself Python for about a year and I studied C# some time ago.
This month I started C++ programming courses at my university and I just have to ask; "why is the C++ code so complicated?"
Writing "Hello world." in Python is as simple as "print 'Hello world.'" but in C++ it's:
# include <iostream>
using namespace std;
int main ()
{
cout << "Hello world.";
return 0;
}
I know there is probably a good reason for all of this but, why...
... do you have to include the <iostream> everytime? Do you ever not need it?
... same question for the standard library, when do you not need std::*?
... is the "main" part a function? Do you ever call the main function? Why is it an integer? Why does C++ need to have a main function but Python doesn't?
... do you need "std::cout << "? Isn't that needlessly long and complicated compared to Python?
... do you need to return 0 even when you are never going to use it?
This is probably because I'm learning such basic C++ but every program I've made so far looks like this, so I have to retype the same code over and over again. Isn't that redundant? Couldn't the compiler just input this code itself, since it's always the same (i.e. afaik you always include <iostream>, std, int main, return 0)
C++ is a more low-level language that executes without the context of an interpreter. As such, it has many different design choices than does Python, because C++ has no environment which it can rely on to manage information like types and memory. C++ can be used to write an operating system kernel where there is no code running on the machine except for the program itself, which means that the language (some library facilities are not available for so-called freestanding implementations) must be self-contained. This is why C++ has no equivalent to Python's eval, nor a means of determining members, etc. of a class, nor other features that require an execution environment (or a massive overhead in the program itself instead of such an environment)
For your individual questions:
do you have to include the <iostream> everytime? Do you ever not need it?
#include <iostream> is the directive that imports the <iostream> header into your program. <iostream> contains the standard input/output objects - in particular, cout. If you aren't using standard I/O objects (for instance, you use only file I/O, or your program uses a GUI library, or are writing an operating system kernel), you do not need <iostream>
same question for the standard library, when do you not need std::*?
std is the namespace containing all of the standard library. using namespace std; is sort of like from std import *, whereas a #include directive is (in this regard) more like a barebones import std statement. (in actual fact, the mechanism is rather different, because C++ does not use using namespace std; to automatically lookup objects in std; the using-directive only imports the names into the global namespace.)
I'll note here that using-directives (using namespace) are frequently frowned upon in C++ code, as they import a lot of names and can cause name clashes. using-declarations (using std::cout;) are preferred when possible, as is limiting the scope of a using-directive (for instance, to one function or to one source file). Don't ever put using namespace in a header without good reason.
is the "main" part a function? Do you ever call the main function? Why is it an integer? Why does C++ need to have a main function but Python doesn't?
main is the entry point to the program - where execution starts. In Python, the __main__ module serves the same purpose. C++ does not execute code outside a defined function like Python does, so its entry point is a function rather than a module.
do you need "std::cout << "? Isn't that needlessly long and complicated compared to Python?
std::cout is only needed if you don't import the cout name into the global namespace, either by a using-directive (using namespace std;) or by a using-declaration (using std::cout). In this regard, it is once again much like the distinction between Python's import std and from std import * or from std import cout.
The << is an overloaded operator for standard stream objects. cout << value calls cout's function to output value. Python needs no such extra code because print is built into the language; this does not make sense for C++, where there may not even be an operating system, much less an I/O library.
do you need to return 0 even when you are never going to use it?
No. main (and no other function) has an implicit return 0; at the end. The return value of main (or, if the exit function is called, the value passed to it) is passed back to the operating system as the exit code. 0 indicates the program successfully executed - that it encountered no errors, etc. If an error is encountered, a non-zero value should be returned (or passed to exit).
In response to your questions at the end of the post, it can be summed up with the philosophy of C++:
You don't pay for what you don't use.
You don't always need to use stdin or stdout (Windows/GUI apps?), nor will you always be using the STL, nor will everything you write necessarily use the standard main (winAPI) etc. As a previous poster said, C++ is lower level than Python. You will be exposed to more of the details, which offers you more control over what you're doing.
... do you have to include the
everytime? Do you ever not
need it?
You don't need it if you're not going to use iostreams in that module. In larger programs, few modules do any actual IO directly, and so few actually need to use iostreams.
Turning the question around: in python you need to import sys and/or os in most non-trivial programs. Why?
... same question for the standard
library, when do you not need std::*?
You can have the using line or you can use the std:: prefix. This is very similar to the choice python gives you of either saying "from sys import *" or "import sys" and then having to prefix things with "sys.". In python you have to say "sys.stdout". Is "std::cout" really any worse?
... is the "main" part a function? Do
you ever call the main function? Why
is it an integer? Why does C++ need to
have a main function but Python
doesn't?
Yes, main is a function. Typically you wouldn't call main yourself. The name "main" is reserved for the entry-point of your program. It returns an integer because the value returned is used as the status code of your program. In Python you can use sys.exit if you want to return a non-zero status code.
Python doesn't have the same convention because with Python you can have code in a module not in a function. This code is executed when you load the module. Interestingly, many people feel it is bad style to have code at the top-level of a module and will instead create a main function by doing something like this:
def main(argv):
# program goes here
return 0
if __name__ == '__main__':
sys.exit(main(sys.argv))
Also, in Python you tell the interpreter with module is the "main" module when you run it. eg: "python foo.py". In C, the "main" module is (effectively) the one with a function called main. (If there are multiple modules with a main function, it's a linker error.)
... do you need "std::cout << "? Isn't
that needlessly long and complicated
compared to Python?
The equivalent in Python is actually "sys.stdout.write(...)". Python's print statement is a special-case short-hand.
That said, many people do feel the iostreams convention of using bit-shifting operators for IO was a bad idea. Ironically, Python seems to have been "inspired" by this syntax. If you want to use print to write to somewhere other than stdout you can say:
print >>file, "Hello"
... do you need to return 0 even when
you are never going to use it?
You aren't going to use it, but your program will. As mentioned earlier, the value you return is the status code of your program.
Aside: I actually do feel that C++ is overcomplicated, but not because of any of the points you mention. All of the differences you mention go away (in the sense that you need just as much complexity in Python) once you start writing non-trivial programs that have multiple modules and do more than just writing to stdout.
You include <iostream> when you want to output things to the console. Since printing "Hello world" involves console output, you need iostream.
The main function is called by the operating system, basically. It gets called with the command-line arguments passed to the program. It returns an integer because the program must return an error code to the operating system (this is the standard way for determining if the last command was successful).
You can always use printf("hello world"); instead of std::cout << "hello world"; if you want to go C style. It's a bit quicker to write and lets you do formatted output.
You return 0 from main to indicate that the program executed successfully.
The compiler does not automatically include all the standard libraries and use namespace std because sometimes name collisions can result between your code and library code that you may not actually need at all. You don't always need all the libraries. Likewise, sometimes you are using a different main routine (Windows development comes to mind with its own, different WinMain starting function). The compiler also does not automatically return 0 because sometimes the program needs to indicate that it completed unsuccessfully.
There are good reasons for all these things. C++ is a very broad language it is used for everything from small embedded systems to giant applications built by 100s of programmers. The use case of a guy building a small program to run on a desktop is by no means the only one. So sometimes you are building library components. In that case no main(). Sometimes you are working on a tiny system with no standard library. In that case no std. Sometimes you want to build a Unix tool that works with other Unix text tools and signals its completion status with an int returned from main().
In other words the things you complain about are boilerplate to you. But they are vital details that vary to other users of the language.
This reminds me of The Evolution of a Programmer. Some of the languages and technologies demonstrated are a bit dated now, but you should get the general idea. :)
One of the reasons C++ is rather complicated is because it was designed to address problems that crop up in large programs. At the time C++ was created as AT&T, their biggest C program was about 10 million lines of code. At that scale, C doesn't function very well. C++ addresses many of the problems you get with that kind of program.
With that said, it's also possible to answer the original questions:
You would include <iostream> where it's needed. If you've got 10.000 C++ files, it's quite common that less than 1000, sometimes less than 100 will produce user-visible output.
A statement like print "Hello, world" assumes that there is a default output, but makes it hard to generalize. The cout << "Hello, world" form makes it explicit where the output goes, but the same form also allows cerr << "Goodbye, world" and MyTmpFile << "Starting phase #" << i
The standard library is in the std:: namespace. My 10.000 files will be in an additional 25 namespaces.
main is an oddity in many ways, being the startup function.
Baldur:
You don't always need <iostream>. The only things that you will always need are:
A main function (or a WinMain, if you're writing Win32 apps).
Variables, functions, operators, language constructs (if, while, etc.).
The ability to include functionality from libraries into your program.
Everything else is application-specific.
As other posters say, the return value of the main function is an error code1. If main returns 0, be happy: everything worked OK!
1This is useful when you write programs that "communicate" with other programs. The most simple way that a program can "tell" another whether it executed properly is using an error code.
As people have said, the simple answer is that they're different languages, with different goals. To answer your specific questions...
... do you have to include the <iostream> everytime? Do you ever not need it?
<iostream> is one of the header files for iostreams, the part of the C++ standard library responsible for input/output; in this instance, you need it to gain access to std::cout. If you're not doing I/O operations in a source file, you don't need to include it -- for example, most files containing class definitions probably won't need <iostream>.
... same question for the standard library, when do you not need std::*?
std is the name of namespace containing classes in the standard library; it's there to avoid name collisions. Python has packages and modules to do this.
You can use the using statement to bring items from another namespace into your current scope, see this FAQ entry for an example (and an explanation of why it's bad to blindly bring all of std into scope!).
... why is the "main" part a function? Do you ever call the main function? Why is it an integer? Why does C++ need to have a main function but Python doesn't?
Executable statements in C++ have to be contained within a function, and the main function is defined as where execution begins. In Python, executable statements can be placed at the top-level of a file, and execution is defined to .
You can call main() if you wish -- it's just a function, after all -- but there's not often a reason to do this. Behind the scenes, most implementations of C++ call main() for you once some startup housekeeping has been done by the runtime library.
The return value of main() is returned back to the operating system. This stems from C and UNIX, in which application programs are required to provide a 1-byte exit status code, and returning that value from main() is a clear way of expressing this.
... why do you need "std::cout << "? Isn't that needlessly long and complicated compared to Python?
This is just a design difference. iostreams is a fairly complex beast with lots of features, and one of the side-effects of this is that the syntax is a bit ugly for simple tasks at times.
... why do you need to return 0 even when you are never going to use it?
You do use it; this is the value returned to the operating system as the exit status of the program.
Python is high-level language. C++ is middle-level language.