Why is the point-cloud-library's loadPCDFile so slow? - c++

Question

I am reading 2.2 million points from a PCD file, and loadPCDFile is using ca 13 sec both in Release as well as Debug mode. Given that visualization programs like CloudCompare can read the file in what seems like milliseconds, I expect that I am doing something harder than it needs to be.
What am I doing wrong?
The top of my PCD file:
# .PCD v0.7 - Point Cloud Data file format
VERSION 0.7
FIELDS rgb x y z _
SIZE 4 4 4 4 1
TYPE F F F F U
COUNT 1 1 1 1 4
WIDTH 2206753
HEIGHT 1
VIEWPOINT 0 0 0 1 0 0 0
POINTS 2206753
DATA binary
¥•ÃöèÝÃájfD ®§”ÃÍÌÝÃá:fD H”ø¾ÝÃH!fD .....
From my code, reading the file:
#include <iostream>
#include <vector>
#include <pcl/common/common.h>
#include <pcl/common/common_headers.h>
#include <pcl/common/angles.h>
#include <pcl/io/pcd_io.h>
#include <pcl/point_types.h>
#include <pcl/visualization/pcl_visualizer.h>
#include <pcl/console/parse.h>
#include <pcl/filters/extract_indices.h>
#include <pcl/features/normal_3d.h>
#include <boost/thread/thread.hpp>
int main() {
(...)
pcl::PointCloud<pcl::PointXYZRGB>::Ptr largeCloud(new pcl::PointCloud<pcl::PointXYZRGB>);
largeCloud->points.resize(3000000); //Tried to force resizing only once. Did not help much.
if (pcl::io::loadPCDFile<pcl::PointXYZRGB>("MY_POINTS.pcd", *largeCloud) == -1) {
PCL_ERROR("Couldn't read file MY_POINTS.pcd\n");
return(-1);
}
(...)
return 0;
}
(Using PCL 1.8 and Visual Studio 2015)

Answer 1

Summary of below...
PCL is slightly slower at loading cloud compare formatted PCD files. Looking at the headers, CC seems to add an extra variable to each point "_" that PCL doesn't like and has to format out. But this is only a difference of 30%-40% load time.
Based on the result that with the same size point cloud (3M), my computer took 13 seconds to load it from cloud compare when the program was compiled in Debug mode and only 0.25s to load the same cloud in Release mode. I think that you are running in debug mode. Depending on how you compiled/installed PCL, you may need to rebuild PCL to generate the appropriate Release build. My guess is that whatever you think you are doing to change from Debug to Release is not in fact engaging the PCL release library.
In PCL, across almost all functions, moving from Debug to Release will often give you one to two orders of magnitude faster processing (due to PCL's heavy usage of large array objects that have to be managed differently in Debug mode for visibility)
Testing PCL with cloud compare files
Here is the code that I ran to produce the following outputs:
std::cout << "Press enter to load cloud compare sample" << std::endl;
std::cin.get();
TimeStamp stopWatch = TimeStamp();
pcl::PointCloud<pcl::PointXYZRGB>::Ptr tempCloud2(new pcl::PointCloud<pcl::PointXYZRGB>);
pcl::io::loadPCDFile("C:/SO/testTorusColor.pcd", *tempCloud2);
stopWatch.fullStamp(true);
std::cout <<"Points loaded: "<< tempCloud2->points.size() << std::endl;
std::cout << "Sample point: " << tempCloud2->points.at(0) << std::endl;
std::cout << std::endl;
std::cout << "Press enter to save cloud in pcl format " << std::endl;
std::cin.get();
pcl::io::savePCDFileBinary("C:/SO/testTorusColorPCLFormatted.pcd", *tempCloud2);
std::cout << "Press enter to load formatted cloud" << std::endl;
std::cin.get();
stopWatch = TimeStamp();
pcl::PointCloud<pcl::PointXYZRGB>::Ptr tempCloud3(new pcl::PointCloud<pcl::PointXYZRGB>);
pcl::io::loadPCDFile("C:/SO/testTorusColorPCLFormatted.pcd", *tempCloud3);
stopWatch.fullStamp(true);
std::cout << "Points loaded: " << tempCloud3->points.size() << std::endl;
std::cout << "Sample point: " << tempCloud3->points.at(0) << std::endl;
std::cout << std::endl;
std::cin.get();
Cloud compare generated colored cloud (3M points with color):
Running in Debug, reproduced your approximate load time with a 3M pt cloud:
Running in Release:

Answer 2

I was running into exactly this situation.
It simply comes down to file storage style. Your file (taking that long to load) is almost certainly an ASCII style point cloud file. If you want to be able to load it much faster (x100) then convert it to binary format. For reference, I load a 1M pt cloud in about a quarter second (but that is system dependent)
pcl::PointCloud<pcl::PointXYZ>::Ptr tempCloud(new pcl::PointCloud<pcl::PointXYZ>);
The load call is the same:
pcl::io::loadPCDFile(fp, *tempCloud);
but in order to save as binary use this:
pcl::io::savePCDFileBinary(fp, *tempCloud);
Just in case it helps, here is a snippet of the code I use to load and save clouds (I structure them a bit, but it is likely based on an example, so I don't know how important that is but you may want to play with it if you switch to binary and are still seeing long load times).
//save pt cloud
std::string filePath = getUserInput("Enter file name here");
int fileType = stoi(getUserInput("0: binary, 1:ascii"));
if (filePath.size() == 0)
printf("failed file save!\n");
else
{
pcl::PointCloud<pcl::PointXYZ> tempCloud;
copyPointCloud(*currentWorkingCloud, tempCloud);
tempCloud.width = currentWorkingCloud->points.size();
tempCloud.height = 1;
tempCloud.is_dense = false;
filePath = "../PointCloudFiles/" + filePath;
std::cout << "Cloud saved to:_" << filePath << std::endl;
if (fileType == 0){pcl::io::savePCDFileBinary(filePath, tempCloud);}
else
{pcl::io::savePCDFileASCII(filePath, tempCloud);}
}
//load pt cloud
std::string filePath = getUserInput("Enter file name here");
if (filePath.size() == 0)
printf("failed user input!\n");
else
{
filePath = "../PointCloudFiles/" + filePath;
pcl::PointCloud<pcl::PointXYZ>::Ptr tempCloud(new pcl::PointCloud<pcl::PointXYZ>);
if (pcl::io::loadPCDFile(filePath, *tempCloud) == -1) //* load the file
{
printf("failed file load!\n");
}
else
{
copyPointCloud(*tempCloud, *currentWorkingCloud); std::cout << "Cloud loaded from:_" << filePath << std::endl;
}
}
List item

Answer 3

This looks correct, when comparing with a pcl example. I think the main work of loadPCDFile is done in the function pcl::PCDReader::read, which is located in the file pcd_io.cpp. When checking the code for binary data, as it is in your case, there are 3 nested for loops which check if the numerical data of each field is valid. The exact code comment is
// Once copied, we need to go over each field and check if it has NaN/Inf values and assign cloud
That could be time consuming. However, I am speculating.