julia Cxx package different behavior when in module - c++

I'm using Julia 1.1.1, and version 0.3.2 of Cxx. I'm finding that I'm getting an error when I call icxx"" from within a module I've defined, but not when I'm calling it in the REPL. In the REPL I have:
using Cxx
function xformVectors()
cxx"""
#include <iostream>
#include <vector>
class Summer {
public:
Summer() {}
std::vector<int> compute_sum(const std::vector<std::vector<int>> &input) {
std::vector<int> result(input.size(), 0);
for (std::size_t i = 0; i != input.size(); ++i) {
for (std::size_t j = 0; j != input[i].size(); ++j) {
result[i] += input[i][j];
}
}
return result;
}
};
"""
as = [rand(1:10,5), rand(1:10,6)]
x = convert(cxxt"std::vector< std::vector< int > >", as)
summer = #cxxnew Summer()
cxx_v = icxx"$summer->compute_sum($x);"
for v in cxx_v
println(collect(v))
end
end
If I then call xformVectors(), it writes out a couple of integers and exits (the expected behavior). If I use Pkg.generate() to wrap this code in a module, and then call xformVectors(), I get a fatal error:
ERROR: BoundsError: attempt to access 36-element Array{Tuple{AbstractString,Symbol,Int64,Int64,Bool},1} at index [37]
Stacktrace:
[1] getindex(::Array{Tuple{AbstractString,Symbol,Int64,Int64,Bool},1}, ::Int64) at ./array.jl:729
[2] #s37#70 at /home/jov9025/.julia/packages/Cxx/vxYtJ/src/cxxstr.jl:705 [inlined]
[3] #s37#70(::Any, ::Any, ::Any, ::Any) at ./none:0
[4] (::Core.GeneratedFunctionStub)(::Any, ::Vararg{Any,N} where N) at ./boot.jl:522
[5] xformVectors() at /home/jov9025/sandbox/julia/Vector2Vector.jl/src/Vector2Vector.jl:30
[6] top-level scope at none:0
If, in the REPL, I include the source file (include("src/Vector2Vector.jl") and then call Vector2Vector.xformVectors(), it runs without complaint.
Any idea what's going on?

The line in my code where the error originated was:
cxx_v = icxx"$summer->compute_sum($x);"
I still don't understand what the problem was with icxx"", but I don't actually need to use it in this sample (or in my real code). I can just use the #cxx macro, and voila, no crash:
cxx_v = #cxx compute_sum(x)
So, I may have been asking the wrong question (although I think it's an interesting one).
Revisiting this in January 2020: with version 0.3.3 of Cxx, the code executes fine with the original icxx"" line in the REPL.

Related

Simulating Leach has encounter a problem. Finished with error in omnet++

When running the simulation in omnet++ 5.7 the execution stops suddenly and closes.
This is the code that is being run in omnet
auto simulation = getSimulation();
for (i = 1; i <= simulation->getLastComponentId(); i++) {
int x, y, id;
//scan the simulation module vector
mod = (cModule*)simulation->getModule(i);
if (strcmp(mod->getName(), "node") == 0) {
id = ((Node*)mod)->myId;
x = ((Node*)mod)->xpos;
y = ((Node*)mod)->ypos;
nodePtr[id] = ((Node*)mod);
if (id != this->myId) {
cGate* g;
char gName1[32], gName2[32];
// make new gate here
if (this->hasGate(gName1)) {
this->gate(gName1)->disconnect();
this->deleteGate(gName1);
}
this->addGate(gName1, cGate::OUTPUT, false);
// make new gate at other side
if (mod->hasGate(gName2)) {
mod->gate(gName2)->disconnect();
mod->deleteGate(gName2);
}
mod->addGate(gName2, omnetpp::cGate::INPUT, false);
//CHANNEL
cIdealChannel* ch = NULL;
this->gate(gName1)->connectTo(mod->gate(gName2), ch);
g = this->gate(gName1);
g->setDisplayString(g->getDisplayString());
}
}
}
I assume that the last line g->setDisplayString(g->getDisplayString()); is probably where the code breaks. The code repeats in the for loop with i<= simulation->getLastComponentId(). I'm new to Omnet++. Any suggestion to fix this would be helpful.
Thanks.
Several things in your code may be source of crashing:
getModule(i) may return nullptr, see OMNeT++ Simulation API, so you should check in the code whether result is not nullptr.
gName1 and gName2 are not set!
Other issues:
instead of (Node*)mod use dynamic_cast<Node*)>(mod) and check whether results is not nullptr.
instead of strcmp(mod->getName(), "node") == 0 I advice using mod->isName("node") - see OMNeT++ Simulation API
if you want to obtain a module whose name is "node", you do not need to manually check the name of every module - there is a useful method getModuleByPath() see OMNeT++ Simulation Manual

Error while compiling with clang/clang++ [-Werror,-Wrange-loop-constrcut]

I'm trying to build a c++ project. While doing so with g++ the project compiles fine. However if I try to compile with clang I get the error:
ec_read_plan.h:135:19: error: loop variable 'op' creates a copy from type 'const std::pair<ChunkPartType, ReadPlan::ReadOperation>' [-Werror,-Wrange-loop-construct]
for (const auto op : read_operationss_{
ec_read_plan.h:135:8: note: use reference type 'const std:pair<ChunkPartType, ReadPlan::ReadOperation> &' to prevent copying
for (const auto op : read_operations) {
Code is below, I have put a comment next on the line that is giving the error:
protected:
void recoverParts(uint8_t *buffer,
const std::bitset<Goal::Slice::kMaxPartsCount> &available_parts) const {
typedef ReedSolomon<slice_traits::ec::kMaxDataCount, slice_traits::ec::kMaxParityCount> RS;
int k = slice_traits::ec::getNumberOfDataParts(slice_type);
int m = slice_traits::ec::getNumberOfParityParts(slice_type);
int max_parts = k + m;
RS::ConstFragmentMap data_parts{{0}};
RS::FragmentMap result_parts{{0}};
RS::ErasedMap erased;
RS rs(k, m);
int available_count = 0;
for (int i = 0; i < max_parts; ++i) {
if (!available_parts[i] || available_count >= k) {
erased.set(i);
} else {
available_count++;
}
}
for (const auto op : read_operations) { //ERROR appears to be here
data_parts[op.first.getSlicePart()] = buffer + op.second.buffer_offset;
}
for (int i = 0; i < (int)requested_parts.size(); ++i) {
if (!available_parts[requested_parts[i].part]) {
result_parts[requested_parts[i].part] = buffer + i * buffer_part_size;
}
}
rs.recover(data_parts, erased, result_parts, buffer_part_size);
}
Why am I getting such an error with Clang and how can I fix this? Thank you.
For anyone who may want to reproduce the error, the source code is here: https://github.com/lizardfs/lizardfs. Afterwards do:
export CC=/usr/bin/clang
export CC=/usr/bin/clang++
cd lizardfs
mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/opt/lizardfs
make
It turns out the error has to do with the clang and clang++ versions. As mentioned gcc had no problems compiling. By default my machine uses clang/clang++ version 10. I tried using version 11 but also had the same problem. I therefore installed version 9 of clang/clang++ and tried compiling again and it worked.
If you happen to have the same problems make sure to run the following commands or have them in the ~/.bashrc file and run source ~/.bashrc after installing version 9 of clang/clang++
export CC=/usr/bin/clang-9
export CC=/usr/bin/clang++-9

Program takes (a lot) longer than it should due to a function call that never runs

I'm currently running Bayesian Optimization, written in c++. I use a toolbox call Bayesopt from Ruben Martinez-Cantin (http://rmcantin.bitbucket.org/html/). I'm doing my thesis about Bayesian Optimization (https://en.wikipedia.org/wiki/Bayesian_optimization).
I had previously experimented with this toolbox and I have noticed this week that the code is running a lot slower than I remembered. It's worth mentioning that I did write some code that works with this toolbox.
I decided to try to understand why this was happening and I did witness that the code was running much slower than it should.
To try to understand if it was my code's fault or otherwise, I tried an example that doesn't use any of my code.
Consider the following example:
#include <iostream>
#include <bayesopt.hpp>
class ExampleMichalewicz: public bayesopt::ContinuousModel
{
public:
ExampleMichalewicz(bopt_params par);
double evaluateSample(const vectord& x);
bool checkReachability(const vectord &query) {return true;};
void printOptimal();
private:
double mExp;
};
ExampleMichalewicz::ExampleMichalewicz(bopt_params par):
ContinuousModel(10,par)
{
mExp = 10;
}
double ExampleMichalewicz::evaluateSample(const vectord& x)
{
size_t dim = x.size();
double sum = 0.0;
for(size_t i = 0; i<dim; ++i)
{
double frac = x(i)*x(i)*(i+1);
frac /= M_PI;
sum += std::sin(x(i)) * std::pow(std::sin(frac),2*mExp);
}
return -sum;
}
void ExampleMichalewicz::printOptimal()
{
std::cout << "Solutions: " << std::endl;
std::cout << "f(x)=-1.8013 (n=2)"<< std::endl;
std::cout << "f(x)=-4.687658 (n=5)"<< std::endl;
std::cout << "f(x)=-9.66015 (n=10);" << std::endl;
}
int main(int nargs, char *args[])
{
bopt_params par = initialize_parameters_to_default();
par.n_iterations = 20;
par.n_init_samples = 30;
par.random_seed = 0;
par.verbose_level = 1;
par.noise = 1e-10;
par.kernel.name = "kMaternARD5";
par.crit_name = "cBEI";
par.crit_params[0] = 1;
par.crit_params[1] = 0.1;
par.n_crit_params = 2;
par.epsilon = 0.0;
par.force_jump = 0.000;
par.verbose_level = 1;
par.n_iter_relearn = 1; // Number of samples before relearn kernel
par.init_method = 1; // Sampling method for initial set 1-LHS, 2-Sobol (if available),
par.l_type = L_MCMC; // Type of learning for the kernel params
ExampleMichalewicz michalewicz(par);
vectord result(10);
michalewicz.optimize(result);
std::cout << "Result: " << result << "->"
<< michalewicz.evaluateSample(result) << std::endl;
michalewicz.printOptimal();
return 0;
}
If I compile this example alone, the run time is about 23 seconds.
With this cmake file
PROJECT ( myDemo )
ADD_EXECUTABLE(myDemo ./main.cpp)
find_package( Boost REQUIRED )
if(Boost_FOUND)
include_directories(${Boost_INCLUDE_DIRS})
else(Boost_FOUND)
find_library(Boost boost PATHS /opt/local/lib)
include_directories(${Boost_LIBRARY_PATH})
endif()
include_directories(${PROJECT_SOURCE_DIR}/include)
include_directories("../bayesopt/include")
include_directories("../bayesopt/utils")
set(CMAKE_CXX_FLAGS " -Wall -std=c++11 -lpthread -Wno-unused-local-typedefs -DNDEBUG -DBOOST_UBLAS_NDEBUG")
target_link_libraries(myDemo libbayesopt.a libnlopt.a)
Now consider the same main example, but where I add three additional files to my cmake project (without including them in main.cpp). These three files are subpart of all my code.
PROJECT ( myDemo )
ADD_EXECUTABLE(myDemo ./iCubSimulator.cpp ./src/DatasetDist.cpp ./src/MeanModelDist.cpp ./src/TGPNode.cpp)
find_package( Boost REQUIRED )
if(Boost_FOUND)
include_directories(${Boost_INCLUDE_DIRS})
else(Boost_FOUND)
find_library(Boost boost PATHS /opt/local/lib)
include_directories(${Boost_LIBRARY_PATH})
endif()
include_directories(${PROJECT_SOURCE_DIR}/include)
include_directories("../bayesopt/include")
include_directories("../bayesopt/utils")
set(CMAKE_CXX_FLAGS " -Wall -std=c++11 -lpthread -Wno-unused-local-typedefs -DNDEBUG -DBOOST_UBLAS_NDEBUG")
target_link_libraries(myDemo libbayesopt.a libnlopt.a)
This time, the run time is about 3 minutes. This is critical in my work since if I increase par.n_iterations it tends to get much worse.
I further arrived at the conclusion that if I comment a line in TGPNode.cpp
utils::cholesky_decompose(K,L); (NOTICE THAT THIS LINE IS NEVER CALLED).
I get the 23 seconds. This function belongs to a file: ublas_cholesky.hpp, from the bayesopt toolbox.
It is also important to note that the same function is also called within the toolbox code. This line is not commented and it runs during michalewicz.optimize(result);.
Does anyone have any ideia why this is happening? It would be a great help if anyone has some insight about the subject.
Greatly appreciated.
Kindly, José Nogueira
It's not gonna return.
It's going to infinitely recurse (to a stack overflow).
Here's what the code reads like:
bopt_params initialize_parameters_to_default(void)
{
bayesopt::Parameters par;
return par.generate_bopt_params();
And generate_bopt_params:
bopt_params Parameters::generate_bopt_params(){
bopt_params c_params = initialize_parameters_to_default();
It looks like someone tried to remove code duplication without actually testing things. At all. You could reinstate the commented out body of the first function

how to deal with the PyObject* from C++ in Python

I create DLL wrote in C++ , the exporting function returns PyObject * .Then I use ctypes to import the DLL in Python . Now , how can I get the real PyObject ??
here's some part of c++ code:
PyObject* _stdcall getList(){
PyObject * PList = NULL;
PyObject * PItem = NULL;
PList = PyList_New(10);
vector <int> intVector;
int i;
for(int i=0;i<10;i++){
intVector.push_back(i);
}
for(vector<int>::const_iterator it=intVector.begin();it<intVector.end();it++){
PItem = Py_BuildValue("i", &it);
PyList_Append(PList, PItem);
}
return PList;
}
and some python code :
dll = ctypes.windll.LoadLibrary(DllPath)
PList = dll.getList()
*I wanna get the real python list containing 1,2,3,4...10 ? *
Am I clear ?? Thanks advance
You have a number of issues of your code, some modifications:
#include <Python.h>
#include <vector>
extern "C" PyObject* _stdcall getList(){
PyObject *PList = PyList_New(0);
std::vector <int> intVector;
std::vector<int>::const_iterator it;
for(int i = 0 ; i < 10 ; i++){
intVector.push_back(i);
}
for(it = intVector.begin(); it != intVector.end() ; it++ ){
PyList_Append(PList, Py_BuildValue("i", *it));
}
return PList;
}
compile it:
> g++ -Wall -shared lib.cpp -I \Python27\include -L \Python27\libs -lpython27 -o lib.dll -Wl,--add-stdcall-alias
now you can load it as any function and set the getList return type to py_object as:
import ctypes
lib = ctypes.WinDLL('lib.dll')
getList = lib.getList
getList.argtypes = None
getList.restype = ctypes.py_object
getList()
test it:
>>> import ctypes
>>>
>>> lib = ctypes.WinDLL('lib.dll')
>>>
>>> getList = lib.getList
>>> getList.argtypes = None
>>> getList.restype = ctypes.py_object
>>> getList()
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>>
>>>
With Visual Studio, and Python 64 bits:
1- Create an empty Win32 Project (DLL Type)
2- Right Click on your Solution Project -> Configuration Manager
3- Active Solution configuration (Release)
4- Active Solution Platform -> New, then on the bottom Dropdown list, select x64 -> OK
5- In the Source Files Folder, add an empty C++ file
6- Put your C++ code (One modification for getList to be recognized)
#include <Python.h>
#include <vector>
extern "C" __declspec(dllexport) PyObject* _stdcall getList();
PyObject* _stdcall getList(){
PyObject *PList = PyList_New(0);
std::vector <int> intVector;
std::vector<int>::const_iterator it;
for (int i = 0; i < 10; i++){
intVector.push_back(i);
}
for (it = intVector.begin(); it != intVector.end(); it++){
PyList_Append(PList, Py_BuildValue("i", *it));
}
return PList;
}
I'm not exactly clear what you are asking. But I suppose you mean to ask what you can do now with your DLL.
Well, in order to use it appropriately, you'll have to build a special DLL which directly can be imported as a module in Python. In order to determine what to do in order to use this, it is best you have a look for other modules, how they do it. E. g. MySQLdb could be a candidate.
In short, you have this "wrapper" DLL call your function.
But if I have a second look at your question now, I see that you are trying to load your DLL via ctypes. This is viable as well, maybe even better, and you'll have to use the ctypes.py_object data type.

How to return std::vector<std::vector<int>> from C++ to C++/CLI?

In my solution I have C++/CLI (vs2012) project that executes some method in C++ (vs2010) project.
Here is a signature of native code:
void Pcl::Downsample(std::vector<CloudPointNative>& points, std::vector<std::vector<int>>& clusters)
And here is how I execute it on C++/CLI side:
std::vector<std::vector<int>> clusters;
pcl->Downsample(points, clusters);
Then I try to iterate over clusters:
for (int clusterIndex = 0; clusterIndex < clusters.size(); clusterIndex++)
{
auto cluster = clusters[clusterIndex];
The size of clusters is 7 and each item in the vector contains vectors of int. I can see this in debugger on the native side. As soon as I get back to managed side (C++/cli project) I get problems.
It works correctly if clusterIndex == 0 and clusterIndex == 5. But throws AccessViolationException on any other values of clusterIndex.
auto cluster0 = clusters[0]; // works
auto cluster1 = clusters[1]; // AccessViolationException
auto cluster5 = clusters[5]; // works
How can this be?
Solved. I have changed signature to:
std::vector<std::vector<int>*>* Pcl::Downsample(std::vector<CloudPointNative>& points)
and also added Free method to delete the vectors
void Pcl::Free(std::vector<std::vector<int>*>* clusters)
{
for(int i = 0; i < clusters->size(); i++)
{
delete (*clusters)[i];
}
delete clusters;
}
Because objects created in external DLL should be also deleted in external DLL.