Writing / Accessing points in a PointCloud2 structure C++ / ROS / PCL Library - c++

I am currently building a perception pipeline and am having trouble reading the points of my point cloud structure. I am reading in a point cloud into a PointCloud2 Structure and want to be able to write out the points in series of the point cloud or at the very least access them so I find a way to write them to a file later. The basic code I am working with is here:
void cloud_cropbox_cb(const sensor_msgs::PointCloud2ConstPtr &cloud_msg)
{
// Container for original & filtered data
pcl::PCLPointCloud2 *cloud = new pcl::PCLPointCloud2;
pcl::PCLPointCloud2ConstPtr cloudPtr(cloud);
pcl::PCLPointCloud2 cloud_filtered;
// convert given message into PCL data type
pcl_conversions::toPCL(*cloud_msg, *cloud);
I want to be able to access the x,y,z elements of each point in the cloud and ideally just write them all out to a textfile. I have tried using pcl::io::savePCDFileASCII() but I don't think that is quite applicable here. An additional note is the base program is constantly running and never exits until manually closed I don't know if that will be relavent to writing out to a file. Any help on what exact functions to use / steps I need to take would be much appreciated.

If you literally want to write out the contents of the message to a file, you can just (in launch file or commandline) use ros topic echo [args] > my_file.txt.
For ways that are compatible in C++, I recommend reading from http://wiki.ros.org/pcl_ros, which describes that:
pcl_ros extends the ROS C++ client library to support message passing with PCL native data types. Simply add the following include to your ROS node source code:
#include <pcl_ros/point_cloud.h>
This header allows you to publish and subscribe pcl::PointCloud objects as ROS messages. These appear to ROS as sensor_msgs/PointCloud2 messages, offering seamless interoperability with non-PCL-using ROS nodes. For example, you may publish a pcl::PointCloud in one of your nodes and visualize it in rviz using a Point Cloud2 display. It works by hooking into the roscpp serialization infrastructure.
This means that you can internally use PCL all you want, and it's all a ROS PointCloud2 message. But you have to declare the PCL type / interface you want to use in the message / publisher / subscriber that you access the data with. So instead of using the sensor_msgs/PointCloud2 type everywhere, use the pcl::Pointcloud<pcl::Point*> type everywhere.
Publishing:
typedef pcl::PointCloud<pcl::PointXYZ> PointCloud;
ros::Publisher pub = nh.advertise<PointCloud> ("points2", 1);
PointCloud::Ptr msg (new PointCloud);
msg->points.push_back (pcl::PointXYZ(1.0, 2.0, 3.0));
pub.publish (msg);
Subscribing:
typedef pcl::PointCloud<pcl::PointXYZ> PointCloud;
ros::Subscriber sub = nh.subscribe<PointCloud>("points2", 1, callback);
void callback(const PointCloud::ConstPtr& msg){
printf ("Cloud: width = %d, height = %d\n", msg->width, msg->height);
BOOST_FOREACH (const pcl::PointXYZ& pt, msg->points)
printf ("\t(%f, %f, %f)\n", pt.x, pt.y, pt.z);
}
More examples (or links to node source code) in http://wiki.ros.org/pcl_ros, tutorials on other pipeline stuff in http://wiki.ros.org/pcl_ros/Tutorials.

Related

Converting Protobuf to ROS messages with similar functionality

I recently inherited a large codebase at work utilizing MOOS & Protobuf messages.
At the request of my project lead, I am porting it to use exclusively ROS where ROS messages are used instead of protobuf. The code base heavily relies on utilizing protobuf functionality such as enumerator min / max, extracting a string from the variable field, ->has_variable() function, ->isValid(), etc.
So far I have only been able to find very basic ROS message functionality from the wiki.
Are there any 'hacks' or the like to have this type of pliability?
Example: Protbufs support enumerators, but ROS messages don't, so I have:
uint8 TYPE_FAILED = 0
uint8 TYPE_OPERATIONAL = 1
uint8 TYPE_INITIALIZING = 2
uint8 health_state_type
My health_state_type is my 'enumerator' but I don't have a min or max unless I hardcode one, and I can't extract TYPE_FAILED as a string. I've been slowly finding workarounds for this by using
my_message::custom_msg health;
health.health_state_type = health.TYPE_FAILED
But I'm having to modify many different areas that use it as a string, not integer.
Yes there is a hack. But you need to input a some work into it.
For using the publisher/subscriber methods in ROS you need to define messages for all topics in .msg files.
From this file then a C++ class is automatically generated. But you don't want to touch that autogenerated file! What you could do instead is define your class and associate it with the autogenerated class.
Look here for an example how to do it. You could then expand your custom class with desired methods like isValid.
Another (perhaps simpler) way would be to declare a helper class that would do the desired work for each type in messages.
Or you could simply continue to use protobuf. It is also used at least in Gazebo if not also in ROS.
Sometime ago I wrote some auto generation scripts that consume Protobufs and produce ROS headers (not the msg files) to transmit Protobuf blobs over ROS comms. This would satisfy your need without having to duplicate a Protobuf definition with a supporting ROS msg definition. Code.

Connecting nodes of different GraphDef's

From Python, I have a frozen graph.pb that I'm currently using in a C++ environment. Now the data for the input tensor are currently preprocessed on the CPU, but I would like to do this step in another GraphDef to run it on the GPU, but I can't seem to find a way to connect nodes between two GraphDef's.
Lets assume my frozen graph have an input/placeholder named mid that I'd like to connect with the preprocessing steps below
tf::GraphDef create_graph_extension() {
tf::Scope root = tf::Scope::NewRootScope();
auto a = tf::ops::Const(root.WithOpName("in"), {(float) 23.0, (float) 31.0});
auto b = tf::ops::Identity(root.WithOpName("mid"), a);
tf::GraphDef graph;
TF_CHECK_OK(root.ToGraphDef(&graph));
return graph;
}
I usually use session->Extend() to run multiple graphs in the same session, but always making sure their node names are unique. With non-unique node names, that I hoped to connect, I get an error
Failed to install graph:
Invalid argument: GraphDef argument to Extend includes node 'mid', which
was created by a previous call to Create or Extend in this session.
P.s. It seems like it is possible in python at least (link)
You can achieve what you're looking for using the same idea that was suggested for Python - import one GraphDef into another and remap inputs.
In case you do use the C API (which has stability guarantees), you'd want to look at:
TF_GraphImportGraphDef (which is parallel to the tf.import_graph_def call in Python), and
TF_ImportGraphDefOptionsAddInputMapping which serves the same purpose as the input_map argument in Python.
These are implemented on top of the C++ ImportGraphDef function, which you might be able to use directly instead (though that doesn't seem to yet be part of the exported C++ API)
Hope that helps.

pcl::PCLPointCloud2 usage

I'm confused with when to use pcl::PointCloud2 vs pcl::PointCloudPointCloud
For example, using these definitions for pcl1_ptrA, pcl1_ptrB and pcl1_ptrC:
pcl::PointCloud<pcl::PointXYZRGB>::Ptr pcl1_ptrA(new pcl::PointCloud<pcl::PointXYZRGB>); //pointer for color version of pointcloud
pcl::PointCloud<pcl::PointXYZRGB>::Ptr pcl1_ptrB(new pcl::PointCloud<pcl::PointXYZRGB>); //ptr to hold filtered Kinect image
pcl::PointCloud<pcl::PointXYZRGB>::Ptr pcl1_ptrC(new pcl::PointCloud<pcl::PointXYZRGB>); //ptr to hold filtered Kinect image
I can invoke the following PCL functions:
pcl::VoxelGrid<pcl::PointXYZRGB> vox;
vox.setInputCloud(pcl1_ptrA);
vox.setLeafSize(0.02f, 0.02f, 0.02f);
vox.filter(*pcl1_ptrB);
cout<<"done voxel filtering"<<endl;
cout<<"num bytes in original cloud data = "<<pcl1_ptrA->points.size()<<endl;
cout<<"num bytes in filtered cloud data = "<<pcl1_ptrB->points.size()<<endl; // ->data.size()<<endl;
Eigen::Vector4f xyz_centroid;
pcl::compute3DCentroid (*pcl1_ptrB, xyz_centroid);
float curvature;
Eigen::Vector4f plane_parameters;
pcl::computePointNormal(*pcl1_ptrB, plane_parameters, curvature); //pcl fnc to compute plane fit to point cloud
Eigen::Affine3f A(Eigen::Affine3f::Identity());
pcl::transformPointCloud(*pcl1_ptrB, *pcl1_ptrC, A);
However, if I instead use pcl::PCLPointCloud2 objects, e.g.:
pcl::PCLPointCloud2::Ptr pcl2_ptrA (new pcl::PCLPointCloud2 ());
pcl::PCLPointCloud2::Ptr pcl2_ptrB (new pcl::PCLPointCloud2 ());
pcl::PCLPointCloud2::Ptr pcl2_ptrC (new pcl::PCLPointCloud2 ());
This function works:
pcl::VoxelGrid<pcl::PCLPointCloud2> vox;
vox.setInputCloud(pcl2_ptrA);
vox.setLeafSize(0.02f, 0.02f, 0.02f);
vox.filter(*pcl2_ptrB);
But these do not even compile:
//the next 3 functions do NOT compile:
Eigen::Vector4f xyz_centroid;
pcl::compute3DCentroid (*pcl2_ptrB, xyz_centroid);
float curvature;
Eigen::Vector4f plane_parameters;
pcl::computePointNormal(*pcl2_ptrB, plane_parameters, curvature);
Eigen::Affine3f A(Eigen::Affine3f::Identity());
pcl::transformPointCloud(*pcl2_ptrB, *pcl2_ptrC, A);
I am having trouble discovering which functions accept which objects. Ideally, wouldn't all PCL functions accept pcl::PCLPointCloud2 arguments?
pcl::PCLPointCloud2 is a ROS (Robot Operating System) message type replacing the old sensors_msgs::PointCloud2. Hence, it should only be used when interacting with ROS. (see an example here)
If needed, PCL provides two functions to convert from one type to the other:
void fromPCLPointCloud2 (const pcl::PCLPointCloud2& msg, cl::PointCloud<PointT>& cloud);
void toPCLPointCloud2 (const pcl::PointCloud<PointT>& cloud, pcl::PCLPointCloud2& msg);
Extra Information
fromPCLPointCloud2 and toPCLPointCloud2 are PCL library functions for conversions. ROS has wrappers for those functions in pcl_conversions/pcl_conversions.h that you should use instead. These will call the right combinations of functions to convert between the message and templated format.
As a follow-up for what Albert said, For ROS Hydro and later (if you are using ROS), PCL has done a work to remove all its dependencies from ROS. What this means is the pcl has created a set of classes which are almost identical to the corresponding ROS messages, in order to minimize API breakage for pcl users. PointCloud users using "now depracated" sensor_msgs::PointCloud2 are now asked to use pcl_conversions package, this package implements conversions from/to sensor_msgs::PointCloud2 to/from) pcl::PointCloud and should include :
#include <pcl_conversions/pcl_conversions.h>
and the ROS code should be modified as the following :
void foo(const sensor_msgs::PointCloud2 &pc)
{
pcl::PCLPointCloud2 pcl_pc;
pcl_conversions::toPCL(pc, pcl_pc);
pcl::SomePCLFunction(pcl_pc);
...
}
Also, pcl is no longer packaged by the ROS community as a catkin package, so any packages which directly depend on pcl should instead use the new rosdep rules libpcl-all and libpcl-all-dev and follow the PCL developer's guidelines for using PCL in your CMake. Generally this means that a package.xml will change like this:
find_package(PCL REQUIRED)
...
include_directories(${PCL_INCLUDE_DIRS})
...
target_link_libraries(<YOUR_TARGET> ${PCL_LIBRARIES})
More on the migration rules can be found here, here and here.

Extract Point Cloud from a pcl::people::PersonCluster<PointT>

I am doing a project for the university and I need to extract the point cloud of the people to work with it. I made an ROS node adapting the code of the tutorial Ground based rgdb people detection, and now I want to publish in a topic the point cloud of the first detected cluster.
But I am not able to extract that point cloud, the definition of the class is defined here person_cluster.h. And there is a public member:
typedef pcl::PointCloud<PointT> PointCloud;
So to convert it in a sensor_msgs::PointCloud2 I do:
pcl_conversions::fromPCL(clusters.at(0).PointCloud,person_msg);
where person_msg is the PointCLoud2 message, clusters is a vector of pcl::people::PersonCluster<PointT>, and I want to publish only the first point cloud because I assume that there is only one person in the scene.
The compiler gives me this error:
error: invalid use of ‘pcl::people::PersonCluster::PointCloud’
pcl_conversions::fromPCL(clusters.at(0).PointCloud,person_msg);
I don't have a lot of knowledge about C++ and I am not able to overcome this error. Googling that error it seems that it appears when you don't "define" well a class, but I doubt that in the pcl library they defined bad a class.
For those that are interested I resolved my problem.
In the forum of pcl I found a post where the same developer of the people detector I used gave the answer.
So basically:
// Get cloud after voxeling and ground plane removal:
PointCloudT::Ptr no_ground_cloud (new PointCloudT);
no_ground_cloud = people_detector.getNoGroundCloud();
// Show pointclouds of every person cluster:
PointCloudT::Ptr cluster_cloud (new PointCloudT);
for(std::vector<pcl::people::PersonCluster<PointT> >::iterator it = clusters.begin(); it != clusters.end(); ++it)
{
if (it->getPersonConfidence() > min_confidence)
{
// Create the pointcloud with points belonging to the current cluster:
cluster_cloud->clear();
pcl::PointIndices clusterIndices = it->getIndices(); // cluster indices
std::vector<int> indices = clusterIndices.indices;
for(unsigned int i = 0; i < indices.size(); i++) // fill cluster cloud
{
PointT* p = &no_ground_cloud->points[indices[i]];
cluster_cloud->push_back(*p);
}
// Visualization:
viewer.removeAllPointClouds();
viewer.removeAllShapes();
pcl::visualization::PointCloudColorHandlerRGBField<PointT> rgb(cluster_cloud);
viewer.addPointCloud<PointT> (cluster_cloud, rgb, "cluster_cloud");
viewer.setPointCloudRenderingProperties (pcl::visualization::PCL_VISUALIZER_POINT_SIZE, 2, "cluster_cloud");
viewer.spinOnce(500);
}
}
Actually I was not able to convert such type of point cloud in a sensor message PointCloud2, even trying to convert that pointcloud in a pcl::PointCloud<pcl::PointXYZ>.
A working solution was that to use as cluster_cloud a type of pcl::PointCloud<pcl::PointXYZ> and then using a publisher of type
pcl::PointCloud<pcl::PointXYZ>
Such:
ros::Publisher person_pub = nh.advertise<PointCloud>("personPointCloud", 1000);
Anyway it was no publish anything, the rviz didnt showed anything. But the viever was displaying the point cloud of the detected person. Since that pointcloud was not such I expected (if you move the arm the algorithm does no give you all the arm) it is not useful for my project so I drop it.
So still remains the problem to publish it in ros, but the problem to get the pointcloud is resolved.

How to exchange custom data between Ops in Nuke?

This questions is addressed to developers using C++ and the NDK of Nuke.
Context: Assume a custom Op which implements the interfaces of DD::Image::NoIop and
DD::Image::Executable. The node iterates of a range of frames extracting information at
each frame, which is stored in a custom data structure. An custom knob, which is a member
variable of the above Op (but invisible in the UI), handles the loading and saving
(serialization) of the data structure.
Now I want to exchange that data structure between Ops.
So far I have come up with the following ideas:
Expression linking
Knobs can share information (matrices, etc.) using expression linking.
Can this feature be exploited for custom data as well?
Serialization to image data
The custom data would be serialized and written into a (new) channel. A
node further down the processing tree could grab that and de-serialize
again. Of course, the channel must not be altered between serialization
and de-serialization or else ... this is a hack, I know, but, hey, any port
in a storm!
GeoOp + renderer
In cases where the custom data is purely point-based (which, unfortunately,
it isn't in my case), I could turn the above node into a 3D node and pass
point data to other 3D nodes. At some point a render node would be required
to come back to 2D.
I am going into the correct direction with this? If not, what is a sensible
approach to make this data structure available to other nodes, which rely on the
information contained in it?
This question has been answered on the Nuke-dev mailing list:
If you know the actual class of your Op's input, it's possible to cast the
input to that class type and access it directly. A simple example could be
this snippet below:
//! #file DownstreamOp.cpp
#include "UpstreamOp.h" // The Op that contains your custom data.
// ...
UpstreamOp * upstreamOp = dynamic_cast< UpstreamOp * >( input( 0 ) );
if ( upstreamOp )
{
YourCustomData * data = yourOp->getData();
// ...
}
// ...
UPDATE
Update with reference to a question that I received via email:
I am trying to do this exact same thing, pass custom data from one Iop
plugin to another.
But these two plugins are defined in different dso/dll files.
How did you get this to work ?
Short answer:
Compile your Ops into a single shared object.
Long answer:
Say
UpstreamOp.cpp
DownstreamOp.cpp
define the depending Ops.
In a first attempt I compiled the first plugin using only UpstreamOp.cpp,
as usual. For the second plugin I compiled both DownstreamOp.cpp and
UpstreamOp.cpp into that plugin.
Strangely enough that worked (on Linux; didn't test Windows).
However, by overriding
bool Op::test_input( int input, Op * op ) const;
things will break. Creating and saving a Comp using the above plugins still
works. But loading that same Comp again breaks the connection in the node graph
between UpstreamOp and DownstreamOp and it is no longer possible to connect
them again.
My hypothesis is this: since both plugins contain symbols for UpstreamOp it
depends on the load order of the plugins if a node uses instances of UpstreamOp
from the first or from the second plugin. So, if UpstreamOp from the first plugin
is used then any dynamic_cast in Op::test_input() will fail and the two Op cannot
be connected anymore.
It is still surprising that Nuke would even bother to start at all with the above
configuration, since it can be rather picky about symbols from plugins, e.g if they
are missing.
Anyway, to get around this problem I did the following:
compile both Ops into a single shared object, e.g. myplugins.so, and
add TCL script or Python script (init.py/menu.py)which instructs Nuke how to load
the Ops correctly.
An example for a TCL scripts can be found in the dev guide and the instructions
for your menu.py could be something like this
menu = nuke.menu( 'Nodes' ).addMenu( 'my-plugins' )
menu.addCommand('UpstreamOp', lambda: nuke.createNode('UpstreamOp'))
menu.addCommand('DownstreamOp', lambda: nuke.createNode('DownstreamOp'))
nuke.load('myplugins')
So far, this works reliably for us (on Linux & Windows, haven't tested Mac).