I want to write a Boost-Python program to take a symbolic python function from user and evaluate its derivative in my program.
For example the User provide a python file (Function.py) which defines a function like
F = sin(x)*cos(x).
Then I want to have access to F'(x) (derivative of F(x)) using symbolic differentiation ability of Sympy. I don't want to use numerical differentiation.
Is there a way to make such a function F'(x) accessible in the C++ using Boost-Python.
Here is some code that should help you get started.
main.cpp:
#include <boost/python.hpp>
#include <iostream>
using namespace boost::python;
int main(void) {
Py_Initialize();
object main_module = import("__main__");
object main_namespace = main_module.attr("__dict__");
exec("from __future__ import division\n"
"from sympy import *\n"
"x = symbols('x')\n"
"f = symbols('f', cls=Function)\n"
"f = cos(x) * sin(x)\n"
"f1 = lambda u: diff(f).subs(x, u);\n",
main_namespace);
exec("result = f1(1.0)", main_namespace);
double res = extract<double>(main_namespace["result"]);
std::cout << "Out: " << res << std::endl;
return 0;
}
Compile command, replace with your path and compiler:
$ clang++ -I"/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/Current/Headers/" -L"/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/Current/lib/" -lpython2.7 main.cpp
It compiles but does not work for me right now. Hope it helped.
I am not a SymPy expert but maybe this can help you:
You can define a Python method like:
def f(x):
return sin(x)*cos(x)
You can create an evaluable function f1 as the derivative of f using:
from sympy import *
x = symbols('x')
f1 = lambdify(x, diff(f(x)))
This function f1 can be called from C++ using boost::python. You may create an object to function f1, call the function using the () operator and convert the result to double using extract<>.
Here is an example:
namespace py = boost::python;
Py_Initialize();
py::object main_module = py::import("__main__");
py::object main_dict = main_module.attr("__dict__");
py::exec(
"def f(x):\n"
" return sin(x)*cos(x)\n",
main_dict
);
py::exec(
"from sympy import *\n"
"x = symbols('x')\n"
"f1 = lambdify(x, diff(f(x)))\n",
main_dict
);
py::object f1 = main_dict["f1"];
std::cout << py::extract<double>(f1(0.0)) << std::endl;
std::cout << py::extract<double>(f1(1.0)) << std::endl;
return 0;
Related
I wanted to try out embedding Python into C++. I was able to get that to work but I wanted to start writing prints with variables which are in declared in c++. For example:
(C++)
int num = 43;
PyRun_SimpleString("print("+num+")");
char g;
std::cin>>g;
PyRun_SimpleString("print("+g+")");
I tried to figure out how to use other related functions, but I don't seen to find enough information.
To pass char,
Python script:
def test(person):
return "Hello " + person;
C++:
PyObject *pName, *pModule, *pFunc, *pArgs, *pValue;
pName = PyUnicode_FromString((char*)"script");
pModule = PyImport_Import(pName);
pFunc = PyObject_GetAttrString(pModule, (char*)"test");
pArgs = PyTuple_Pack(1, PyUnicode_FromString((char*)"User"));
pValue = PyObject_CallObject(pFunc, pArgs);
auto result = _PyUnicode_AsString(pValue);
std::cout << result << std::endl;
Output:
Hello User
To pass integer it's the same like above. Here you are passing over double 2.0.
pArgs = PyTuple_Pack(1,PyFloat_FromDouble(2.0));
pValue = PyObject_CallObject(pFunc, pArgs);
You can refer to all the apis here >> https://docs.python.org/3/c-api/
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 11 months ago.
Improve this question
How to make the c ++ application work with the browser. I mean a program that retrieves data from a given page (let's assume that the page displays a string) and then performs some reaction on the page. For example, the page displays a random string, and the program enters the length of the string into the form.
I am a novice programmer, so I care about information and advice on where to start. Thanks in advance for any help.
As I already promised to OP in comments, posting Partial answer, which doesn't answer all questions, but only provides handy tool to wrap (call) any Python code inside C++ program.
In my code snippet I don't even do anything with browsers, but instead show only example of computing Greatest Common Divisor using Python's standard function math.gcd().
I decided to introduce this Python-in-C++ bridge only because there exist many beautiful Python modules that work with browsers or with parsing/composing HTML, hence it is much easier to write such tools in Python instead of C++.
But expert without knowledge of default Python C API, it is not that easy to implement even simple use case - compile text of Python code, pass to it any arguments from C++, receive response arguments, return arguments back to C++. Only these simple actions need usage of a dozen of different Python C API functions. That's why I decided to show how to do it, as I know.
I implemented from scratch (specifically for OP's question) handy class PyRunner which does all the magic, usage of this class is simple:
PyRunner pyrun;
std::string code = R"(
def gcd(a, b):
import math
return math.gcd(a, b)
res = gcd(*arg)
print('GCD of', arg[0], 'and', arg[1], 'is', res, flush = True)
)";
std::cout << pyrun.Run(code, "(2 * 3 * 5, 2 * 3 * 7)") << std::endl;
std::cout << pyrun.Run(code, "(5 * 7 * 11, 5 * 7 * 13)") << std::endl;
Basically you just pass any Python code snippet to PyRunner::Run() method and also any argument (represented as Python object converted to string). Result of this call is also a returned Python object converted to string. You can also use JSON to pass any large argument as string and parse returned argument, as any JSON string is also a valid stringized Python object.
Of course you need a knowledge of Python to be able to write complex code snippets inside C++.
One drawback of my PyRunner class is that for some reason (that I didn't yet understand), you can't import Python module inside global scope, as you can see I did import math within function scope. But this is not a big deal, I think, and maybe some experts will clarify the reason.
To compile and run code you need to have pre-installed Python, and pass Python's include folder and library file as compiler arguments. For example in Windows CLang you do following:
clang.exe -std=c++20 -O3 -Id:/bin/Python39/include/ d:/bin/Python39/libs/python39.lib prog.cpp
and in Linux:
clang -std=c++20 -O3 -I/usr/include/ -lpython3.9 prog.cpp
To run the program either you should provide environment variables PYTHONHOME or PYTHONPATH or run program from Python folder (like d:/bin/Python39/) or do sys.path.append("d:/bin/Python39/") on first lines of Python code snippet embedded in C++. Without these paths Python can't find location of its standard library.
PyRunner class is thread-safe, but only single-threaded always. It means that two calls to .Run() inside two threads will be exclusively blocked by mutex. I use std::mutex instead of Python's GIL to protect from multi-threading, because it is quite alright (and faster), if you don't use Python C API in any other threads simultaneously. Also it is not allowed right now to have two instances of PyRunner objects as it does Py_Initialize() and Py_FinalizeEx() in constructor and destructor, which should be done globally only once. Hence PyRunner should be a singleton.
Below is full C++ code with implementation of PyRunner class and its usage (usage is inside main()). See console output after code below. Click Try it online! link to see compile/run of this code on free GodBolt online Linux servers.
Try it online!
#include <iostream>
#include <functional>
#include <string>
#include <string_view>
#include <stdexcept>
#include <memory>
#include <mutex>
#include <Python.h>
#define ASSERT_MSG(cond, msg) { if (!(cond)) throw std::runtime_error("Assertion (" #cond ") failed at line " + std::to_string(__LINE__) + "! Msg: '" + std::string(msg) + "'."); }
#define ASSERT(cond) ASSERT_MSG(cond, "")
#define PY_ASSERT_MSG(cond, msg) { if (!(cond) || PyErr_Occurred()) { PyErr_Print(); ASSERT_MSG(false && #cond, msg); } }
#define PY_ASSERT(cond) PY_ASSERT_MSG(cond, "")
#define LN { std::cout << "LN " << __LINE__ << std::endl << std::flush; }
class PyRunner {
private:
class PyObj {
public:
PyObj(PyObject * pobj, bool inc_ref = false) : p_(pobj) {
if (inc_ref)
Py_XINCREF(p_);
PY_ASSERT_MSG(p_, "NULL PyObject* passed!");
}
PyObject * Get() { return p_; }
~PyObj() {
Py_XDECREF(p_);
p_ = nullptr;
}
private:
PyObject * p_ = nullptr;
};
public:
PyRunner() {
Py_SetProgramName(L"prog.py");
Py_Initialize();
}
~PyRunner() {
codes_.clear();
Py_FinalizeEx();
}
std::string Run(std::string code, std::string const & arg = "None") {
std::unique_lock<std::mutex> lock(mutex_);
code = StrUnIndent(code);
if (!codes_.count(code))
codes_.insert(std::pair{code, std::make_shared<PyObj>(Py_CompileString(code.c_str(), "script.py", Py_file_input))});
PyObj & compiled = *codes_.at(code);
PyObj globals_arg_mod = PyModule_New("arg"), globals_arg = PyModule_GetDict(globals_arg_mod.Get()), locals_arg = PyDict_New(),
globals_mod = PyModule_New("__main__"), globals = PyModule_GetDict(globals_mod.Get()), locals = PyDict_New();
// py_arg = PyUnicode_FromString(arg.c_str()),
PyObj py_arg = PyRun_String(arg.c_str(), Py_eval_input, globals_arg.Get(), locals_arg.Get());
PY_ASSERT(PyDict_SetItemString(locals.Get(), "arg", py_arg.Get()) == 0);
#if 0
PyObj result = PyEval_EvalCode(compiled.Get(), globals.Get(), locals.Get());
#else
PyObj builtins(PyEval_GetBuiltins(), true), exec(PyDict_GetItemString(builtins.Get(), "exec"), true);
PyObj exec_args = PyTuple_Pack(3, compiled.Get(), globals.Get(), locals.Get());
PyObj result = PyObject_CallObject(exec.Get(), exec_args.Get());
#endif
PyObj res(PyDict_GetItemString(locals.Get(), "res"), true), res_str = PyObject_Str(res.Get());
char const * cres = nullptr;
PY_ASSERT(cres = PyUnicode_AsUTF8(res_str.Get()));
return cres;
}
private:
static std::string StrUnIndent(std::string_view const & s) {
auto lines = StrSplit(s, "\n");
size_t min_off = size_t(-1);
for (auto const & line: lines) {
if (StrTrim(line).empty())
continue;
min_off = std::min<size_t>(min_off, line.find_first_not_of("\t\n\v\f\r "));
}
ASSERT(min_off < 10000ULL);
std::string res;
for (auto const & line: lines)
res += line.substr(std::min<size_t>(min_off, line.size())) + "\n";
return res;
}
static std::string StrTrim(std::string s) {
s.erase(0, s.find_first_not_of("\t\n\v\f\r ")); // left trim
s.erase(s.find_last_not_of("\t\n\v\f\r ") + 1); // right trim
return s;
}
static std::vector<std::string> StrSplit(std::string_view const & s, std::string_view const & delim) {
std::vector<std::string> res;
size_t start = 0;
while (true) {
size_t pos = s.find(delim, start);
if (pos == std::string::npos)
pos = s.size();
res.emplace_back(s.substr(start, pos - start));
if (pos >= s.size())
break;
start = pos + delim.size();
}
return res;
}
private:
std::unordered_map<std::string, std::shared_ptr<PyObj>> codes_;
std::mutex mutex_;
};
int main() {
try {
PyRunner pyrun;
std::string code = R"(
def gcd(a, b):
import math
return math.gcd(a, b)
res = gcd(*arg)
print('GCD of', arg[0], 'and', arg[1], 'is', res, flush = True)
)";
std::cout << pyrun.Run(code, "(2 * 3 * 5, 2 * 3 * 7)") << std::endl;
std::cout << pyrun.Run(code, "(5 * 7 * 11, 5 * 7 * 13)") << std::endl;
return 0;
} catch (std::exception const & ex) {
std::cout << "Exception: " << ex.what() << std::endl;
return -1;
}
}
Console output:
GCD of 30 and 42 is 6
6
GCD of 385 and 455 is 35
35
I am currently embedding Python in C++ using boost-python and boost-numpy.
I have the following Python test script:
import numpy as np
import time
def test_qr(m,n):
print("create numpy array")
A = np.random.rand(m, n)
print("Matrix A is {}".format(A))
print("Lets QR factorize this thing! Mathematics is great !!")
ts = time.time()
Q, R = np.linalg.qr(A)
te = time.time()
print("It took {} seconds to factorize A".format(te - ts))
print("The Q matrix is {}".format(Q))
print("The R matrix is {}".format(R))
return Q,R
def sum(m,n):
return m+n
I am able to execute a part of the code in C++ like this:
namespace p = boost::python;
namespace np = boost::python::numpy;
int main() {
Py_Initialize(); //initialize python environment
np::initialize(); //initialize numpy environment
p::object main_module = p::import("__main__");
p::object main_namespace = main_module.attr("__dict__");
// execute code in the main_namespace
p::exec_file("/Users/Michael/CLionProjects/CythonTest/test_file.py",main_namespace); //loads python script
p::exec("m = 100\n"
"n = 100\n"
"Q,R = test_qr(m,n)", main_namespace);
np::ndarray Q_matrix = p::extract<np::ndarray>(main_namespace["Q"]); // extract results as numpy array types
np::ndarray R_matrix = p::extract<np::ndarray>(main_namespace["R"]);
std::cout<<"C++ Q Matrix: \n" << p::extract<char const *>(p::str(Q_matrix)) << std::endl; // extract every element as a
std::cout<<"C++ R Matrix: \n" << p::extract<char const *>(p::str(R_matrix)) << std::endl;
std::cout<<"code also works with numpy, ask for a raise" << std::endl;
p::object sum = main_namespace.attr("sum")(10,10);
int result = p::extract<int>(main_namespace.attr("sum")(10,10));
std::cout<<"sum result works " << result << std::endl;
return 0;}
Now I am trying to use the sum function in the Python script but I do not always want to write a string like:
p::exec("m = 100\n"
"n = 100\n"
"Q,R = test_qr(m,n)", main_namespace);}
How can this be done without using the exec function?
I have tried things like:
p::object sum = main_namespace.attr("sum")(10,10);
int result = p::extract<int>(main_namespace.attr("sum")(10,10));
std::cout<<"sum result works " << result << std::endl;
As mentioned in the documentation of boost.
I also tried using the call_method function, but it didn't work.
I get either boost::python::error_already_set exception which mean there is something wrong in Python, but I do not know what.
Or an exit code 11.
The issue is rather trivial. Let's look at the tutorial you mention:
object main_module = import("__main__");
object main_namespace = main_module.attr("__dict__");
object ignored = exec("result = 5 ** 2", main_namespace);
int five_squared = extract<int>(main_namespace["result"]);
Notice how they extract the result object in the last line: main_namespace["result"]
The main_namespace object is a Python dictionary, and rather than extracting it's attribute, you're just looking for a value stored with the particular key. Hence, indexing with [] is the way to go.
C++ code:
#define BOOST_ALL_NO_LIB
#include <boost/python.hpp>
#include <boost/python/numpy.hpp>
#include <iostream>
namespace bp = boost::python;
int main()
{
try {
Py_Initialize();
bp::object module = bp::import("__main__");
bp::object globals = module.attr("__dict__");
bp::exec_file("bpcall.py", globals);
bp::object sum_fn = globals["sum"];
int result = bp::extract<int>(sum_fn(1,2));
std::cout << "Result (C++) = " << result << "\n";
} catch (bp::error_already_set) {
PyErr_Print();
}
Py_Finalize();
}
Python script:
def sum(m,n):
return m+n
Output:
Result (C++) = 3
I have recently created a php extension with PHPCPP - C++ library for developing PHP extensions and was expecting a performance boost, however instead of seeing a boost i m only seeing degradation in performance. I believe i'm doing something incorrectly and maybe somebody may spot why is this happening?
Here is a C++ code:
#include <phpcpp.h>
#include <ctime>
#include <iostream>
#include <algorithm>
#include <string>
#include <regex>
#include <stdlib.h> /* atof,strtod */
#include <cstddef> // std::size_t
#include <sstream>
#include <vector>
#include <boost/algorithm/string.hpp> // include Boost, a C++ library
#include <boost/regex.hpp>
using namespace std;
Php::Value test_function(Php::Parameters ¶ms)
{
double a = params[0];
double b = params[1];
return (a + b) * (a - b) / 100;
}
/**
* Switch to C context, because the Zend engine expects get get_module()
* to have a C style function signature
*/
extern "C" {
/**
* Startup function that is automatically called by the Zend engine
* when PHP starts, and that should return the extension details
* #return void*
*/
PHPCPP_EXPORT void *get_module()
{
// the extension object
static Php::Extension extension("test_extension", "1.0");
// add functions so that they can be called from PHP scripts
extension.add("test_function", test_function, {
Php::ByVal("a", Php::Type::Float,true),
Php::ByVal("b", Php::Type::Float,true)
});
// return the extension details
return extension;
}
}
Here is the command line extension compilation:
[root#test test_extension]# make
g++ -Wall -c -O2 -std=c++11 -fpic -o main.o main.cpp
g++ -shared -o test_extension.so main.o -lphpcpp
[root#test test_extension]# make install
cp -f test_extension.so /usr/lib64/php/modules
cp -f test_extension.ini /etc/php.d
Here is my simple test:
<?php
//function written in php
function local_test_function($a,$b){
$res = ($a + $b) * ($a - $b) / 100;
return $res;
}
$output = '';
//local php function test
$start_time = microtime(true);
$output.= "<br> test_function local: " . local_test_function(123.4,12.5);
$end_time = microtime(true);
$duration = $end_time - $start_time;
$duration = number_format($duration,20);
$output.=", Duration: ".$duration.'<br>';
// function written in c++
$start_time = microtime(true);
$output.= "function written in c++: " . test_function(123.4,12.5);
$end_time = microtime(true);
$duration = $end_time - $start_time;
$duration = number_format($duration,20);
$output.=", Duration: ".$duration . '<br>';
?>
The results are:
test_function local result: 150.7131, Duration: 0.00000715255737304688
function written in c++ result: 150.7131, Duration: 0.00003004074096679688
the conclusion PHP function written in C++ is 4 times slower.
Can someone tell why? And is there a way to improve my code in C++?
UPDATE and ANSWER
as suggested i'm trying to test with a more complex function to see if overhead is causing C++ function to be slower
i have modified existing "test_function" + added another function "test_function_inner" and declared it.
double test_function_inner(double a, double b);
Php::Value test_function(Php::Parameters ¶ms)
{
double a = params[0];
double b = params[1];
double res = 0.0;
int i = 100;
while (i > 0) {
res = res + test_function_inner(a, b);
--i;
}
return res;
}
double test_function_inner(double a, double b) {
double res = 0.0;
while (b > a) {
res = res + (a + b) * (a - b) / 100;
b = b - 1;
}
return res;
}
Testing:
<?php
$output = '';
//local php function test
$start_time = microtime(true);
$output.= "<br> test_function local: " . local_test_function(123.4,1200000.5);
$end_time = microtime(true);
$duration = $end_time - $start_time;
$duration = number_format($duration,20);
$output.=", Duration: ".$duration.'<br>';
// function written in c++
$start_time = microtime(true);
$output.= "function written in c++: " . test_function(123.4,1200000.5);
$end_time = microtime(true);
$duration = $end_time - $start_time;
$duration = number_format($duration,20);
$output.=", Duration: ".$duration . '<br>';
?>
results:
test_function local result: -5.7600142172989E+17, Duration: 9.23940896987915039062
function written in c++ result: -5.7600142172989E+17, Duration: 0.72759604454040527344
CONCLUSION
it was overhead caused by initial function call as suggested , and small operations and calculations doesn't worth putting into separate functions in the extension as it wouldn't boost the application but instead would cause negative effect on performance.
I saved my training data (maybe float vectors) in some files, and tried to load it as a Tensor using Tensorflow C++ reader class.
Here is my code.
using namespace tensorflow;
using namespace tensorflow::ops;
using namespace tensorflow::sparse;
Scope root = Scope::NewRootScope();
auto indexReader = FixedLengthRecordReader(root, sizeof(uint32_t));
auto queue = FIFOQueue(root, {DataType::DT_STRING});
auto file = Input::Initializer(std::string("mydata.feat"));
std::cerr << file.tensor.DebugString() << std::endl;
auto enqueue = QueueEnqueue(root, queue, {file});
std::cerr << Input(QueueSize(root, queue).size).tensor().DebugString() << std::endl;
auto rawInputIndex = ReaderRead(root, indexReader, queue);
std::cerr << Input(rawInputIndex.key).tensor().DebugString() << std::endl;
auto decodedInputIndex = DecodeRaw(root, rawInputIndex.value, DataType::DT_UINT8);
std::cerr << Input(decodedInputIndex.output).tensor().DebugString() << std::endl;
It is compiled very well but the cerr shows always empty Tensor. (below is execution result of my program on shell)
Tensor<type: string shape: [] values: mydata.feat>
Tensor<type: float shape: [0] values: >
Tensor<type: float shape: [0] values: >
Tensor<type: float shape: [0] values: >
I don't know why it doesn't work.
Or, is there any C++ example code for class ReaderRead or class FIFOQueue? I cannot find it anywhere...
What you're doing is building a graph. To run this graph you need to create a Session and run it. See the label_image example on the tensorflow codebase for an example of how to do this.