Big point cloud file reading optimization windows - c++

I'm on windows, and I'll work only with windows.
I have a question about opening big files (PTX).
On each line, I will have the coordinates of a points X Y Z I {R G B} ({R, G, B} are not forced to be present).
Since my files are huge (sometimes > 100Go), I would like to read them fastly using memory map (I never did that before), or at least read chunck of memory instead of reading it line by line.
My question is : if I read chunck of memory using
ifstream bigFile("mybigfile.dat");
constexpr size_t bufferSize = 1024 * 1024;
unique_ptr<char[]> buffer(new char[bufferSize]);
while (bigFile)
{
bigFile.read(buffer.get(), bufferSize);
// process data in buffer
}
for example, is there a way to be sure that my buffer won't stop in the middle of a line?
For example, my files is
x1 y1 z1 i1 r1 g1 b1
x2 y2 z2 i2 r2 g2 b2
x3 y3 z3 i3 r3 g3 b3
x4 y4 z4 i4 r4 g4 b4
x5 y5 z5 i5 r5 g5 b5
and I want to create a std::vector<Point>. So I read a buffer size of this file, put it in the buffer, and then I take data from buffer to create my points. But how can I be sure that the buffer won't stop at r3?
If the buffer contains x1 y1 z1 i1 r1 g1 b1 x2 y2 z2 i2 r2 g2 b2 x3 y3 z3 i3 r3 I can't create a point using only x3, y3, z3, i3, r3. I would need g3 and b3 too.
Is there a way to take care of that? I hope that it is understandable, English isn't my native language and I'm not sure I explained it well...

Related

confidence interval and prediction interval in SAS

Given the model:
Y = b0 + b1 x1 + b2 x2 + b3 x3
what command in SAS will allow me to see the confidence interval for Y?
proc reg; model y=x1 x2 x3;
see online for output details.

Calculate intersection of two lines using integers only

I can quite easily calculate the point of intersection given two lines. If I start with two vertices:
(x1,y1)
(x2,y2)
I can calculate the slope by doing (y1-y2)/(x1-x2), and then calculating the intercept
y1 - slope * x1
Then do that again, so I have to sets of slope and intercept, then just do:
x = (intercept2 - intercept1) / (slope1 - slope2)
y = slope1 * x + intercept1
(disclaimer: this might not even work, but i've gotten something very close to it to work, and it illustrates my general technique)
BUT that only works with data types with decimals, or non integral. Say the vertices are:
(0,1)
(10,2)
To calculate the slope would result in (1-2)/(0-10), which is -1/-10 which is not 1/10, it is 0.
How can I get code that yields a valid result using only integers?
Edit: I can't use floats AT ALL!. No casting, no nothing. Also, values are capped at 65535. And everything is unsigned.
In high school when subtracting fractions, our teachers taught us to find a common denominator
So 1/4 - 1/6 = 3/12 - 2/12 = 1/12
So do the same with your slopes.
int slope1 = n1 / d1; // numerator / denominator
int slope2 = n2 / d2;
// All divisions below should have 0 for remainder
int g = gcd( d1, d2 ); // gcd( 4, 6 ) = 2
int d = d1 * d2 / g; // common denominator (12 above)
int n = (d/d1) * n1 - (d/d2) * n2; // (1 in 1/12 above)
// n1/d1 - n2/d2 == n/d
I hope I got that right.
Hm..
(0,1)
(10,2)
and (y1-y2)/(x1-x2). Well, this is the description of one line, not the intersection of two lines.
As far as I remember lines are described in the form of x * v with x an skalar and v be a vector. Then it's
x * (0,1) = v2 and
x * (10, 2) = v2.
therefore the lines only intersect if exactly one solution to both equitions exist, overlap when there are infinitive numbers of solutions and don't intersect when they are parallel.
http://www.gamedev.net/topic/647810-intersection-point-of-two-vectors/
explains the calcuclation based on the dot - product.
Input: line L passing thru (x1, y1) and (x2, y2), and line M passing thru (X1, Y1) and (X2, Y2)
Output: (x, y) of the intersecting point of two lines L and M
Tell Wolfram Alpha to solve y = (y1-y2)/(x1-x2)*(x-x1)+y1 and y = (Y1-Y2)/(X1-X2)*(x-X1)+Y1 for x, y to get this solution:
http://www.wolframalpha.com/share/clip?f=d41d8cd98f00b204e9800998ecf8427e3at5u9evl8
But I have no idea on how to write a program to implement the above solution for your calculator with only uint16_t ALU.
Thanks to Graham Toal's answer, below is a primitive Rust implementation of the linked C code in their answer, modified to return the point of intersection for the complete line, as opposed to the line segment. It doesn't use much Rust-specific magic so should be reasonably easy to port to other languages.
The function returns a Point where the Lines intersect, if at all, and a flag denoting whether the intersection point lies on both intersected lines (true) or not (false).
/// 2D integer point
struct Point {
/// The x coordinate.
pub x: i32,
/// The y coordinate.
pub y: i32,
}
/// Line primitive
struct Line {
/// Start point
pub start: Point,
/// End point
pub end: Point,
}
/// Check signs of two signed numbers
///
/// Fastest ASM output compared to other methods. See: https://godbolt.org/z/zVx9cD
fn same_signs(a: i32, b: i32) -> bool {
a ^ b >= 0
}
/// Integer-only line segment intersection
///
/// If the point lies on both line segments, the second tuple argument will return `true`.
///
/// Inspired from https://stackoverflow.com/a/61485959/383609, which links to
/// https://webdocs.cs.ualberta.ca/~graphics/books/GraphicsGems/gemsii/xlines.c
fn intersection(l1: &Line, l2: &Line) -> Option<(Point, bool)> {
let Point { x: x1, y: y1 } = l1.start;
let Point { x: x2, y: y2 } = l1.end;
let Point { x: x3, y: y3 } = l2.start;
let Point { x: x4, y: y4 } = l2.end;
// First line coefficients where "a1 x + b1 y + c1 = 0"
let a1 = y2 - y1;
let b1 = x1 - x2;
let c1 = x2 * y1 - x1 * y2;
// Second line coefficients
let a2 = y4 - y3;
let b2 = x3 - x4;
let c2 = x4 * y3 - x3 * y4;
let denom = a1 * b2 - a2 * b1;
// Lines are colinear
if denom == 0 {
return None;
}
// Compute sign values
let r3 = a1 * x3 + b1 * y3 + c1;
let r4 = a1 * x4 + b1 * y4 + c1;
// Sign values for second line
let r1 = a2 * x1 + b2 * y1 + c2;
let r2 = a2 * x2 + b2 * y2 + c2;
// Flag denoting whether intersection point is on passed line segments. If this is false,
// the intersection occurs somewhere along the two mathematical, infinite lines instead.
//
// Check signs of r3 and r4. If both point 3 and point 4 lie on same side of line 1, the
// line segments do not intersect.
//
// Check signs of r1 and r2. If both point 1 and point 2 lie on same side of second line
// segment, the line segments do not intersect.
let is_on_segments = (r3 != 0 && r4 != 0 && same_signs(r3, r4))
|| (r1 != 0 && r2 != 0 && same_signs(r1, r2));
// If we got here, line segments intersect. Compute intersection point using method similar
// to that described here: http://paulbourke.net/geometry/pointlineplane/#i2l
// The denom/2 is to get rounding instead of truncating. It is added or subtracted to the
// numerator, depending upon the sign of the numerator.
let offset = if denom < 0 { -denom / 2 } else { denom / 2 };
let num = b1 * c2 - b2 * c1;
let x = if num < 0 { num - offset } else { num + offset } / denom;
let num = a2 * c1 - a1 * c2;
let y = if num < 0 { num - offset } else { num + offset } / denom;
Some((Point::new(x, y), is_on_segments))
}

Draw points and lines in gnuplot

How is it possible with gnuplot to plot both isolated points and lines for the same input file?
I mean, once I have a file data.dat of this kind that defines two lines 1-2 and 3-4
x1 y1
x2 y2
x3 y3
x4 y4
I can plot the lines with
$> plot 'data.dat' w lp
but if I want to also add some isolated points to be displayed with gnuplot I would like to add to my data.dat file the following
x1 y1
x2 y2
x3 y3
x4 y4
x5 y5
x6 y6
x7 y7
obviously the points 5,6,7 are treated by gnuplot as points of a line. How can I draw 5,6,7 as isolated points?
Organize your data in blocks. There should be exactly one empty line between two blocks:
x1 y1
x2 y2
x3 y3
x4 y4
x5 y5
x6 y6
x7 y7
Use every to specify which blocks of the datafile should be plotted. The syntax of every is described here or in gnuplot (type help every)
In your case you can then do the following
plot "data.txt" every :::0::1 with lp, "" every :::2::2 with points
if it's acceptable to plot point symbols for points 1..4, too, then just add empty lines after points 5 and 6 and say
plot "1.dat" with lines, "" with points
if not, I'd suggest writing the isolated points to a separate file.

How to calculate the point of intersection between two lines

I am attempting to calculate the point of intersection between lines for a Optical Flow algorithm using a Hough Transform. However, I am not getting the points that I should be when I use my algorithm for calculating the intersections.
I save the Lines as an instance of a class that I created called ImageLine. Here is the code for my intersection method.
Point ImageLine::intersectionWith(ImageLine other)
{
float A2 = other.Y2() - other.Y1();
float B2 = other.X2() - other.X1();
float C2 = A2*other.X1() + B2*other.Y1();
float A1 = y2 - y1;
float B1 = x2 - x1;
float C1 = A1 * x1 + B1 * y1;
float det = A1*B2 - A2*B1;
if (det == 0)
{
return Point(-1,-1);
}
Point d = Point((B2 * C1 - B1 * C2) / det, -(A1 * C2 - A2 * C1) / det);
return d;
}
Is this method correct, or did I do something wrong? As far as I can tell, it should work, as it does for a single point that I hard-coded through, however, I have not been able to get a good intersection when using real data.
Considering the maths side: if we have two line equations:
y = m1 * x + c1
y = m2 * x + c2
The point of intersection: (X , Y), of two lines described by the following equations:
Y = m1 * X + c1
Y = m2 * X + c2
is the point which satisfies both equation, i.e.:
m1 * X + c1 = m2 * X + c2
(Y - c1) / m1 = (Y - c2) / m2
thus the point of intersection coordinates are:
intersectionX = (c2 - c1) / (m1 - m2)
intersectionY = (m1*c1 - c2*m2) / m1-m2 or intersectionY = m1 * intersectionX + c1
Note: c1, m1 and c2, m2 are calculated by getting any 2 points of a line and putting them in the line equations.
(det == 0) is unlikely to be true when you're using floating-point arithmetic, because it isn't precise.
Something like (fabs(det) < epsilon) is commonly used, for some suitable value of epsilon (say, 1e-6).
If that doesn't fix it, show some actual numbers, along with the expected result and the actual result.
For detailed formula, please go to this page.
But I love code so, here, check this code (I get it from github, so all credit goes to the author of that code):
///Calculate intersection of two lines.
///\return true if found, false if not found or error
bool LineLineIntersect(double x1, double y1, //Line 1 start
double x2, double y2, //Line 1 end
double x3, double y3, //Line 2 start
double x4, double y4, //Line 2 end
double &ixOut, double &iyOut) //Output
{
//http://mathworld.wolfram.com/Line-LineIntersection.html
double detL1 = Det(x1, y1, x2, y2);
double detL2 = Det(x3, y3, x4, y4);
double x1mx2 = x1 - x2;
double x3mx4 = x3 - x4;
double y1my2 = y1 - y2;
double y3my4 = y3 - y4;
double xnom = Det(detL1, x1mx2, detL2, x3mx4);
double ynom = Det(detL1, y1my2, detL2, y3my4);
double denom = Det(x1mx2, y1my2, x3mx4, y3my4);
if(denom == 0.0)//Lines don't seem to cross
{
ixOut = NAN;
iyOut = NAN;
return false;
}
ixOut = xnom / denom;
iyOut = ynom / denom;
if(!isfinite(ixOut) || !isfinite(iyOut)) //Probably a numerical issue
return false;
return true; //All OK
}
Assuming your formulas are correct, try declaring all your intermediate arguments as 'double'. Taking the difference of nearly parallel lines could result in your products being very close to each other, so 'float' may not preserve enough precision.

Drawing line perpendicular to a given line

I have start and end coordinate of a line. I want to drawn another line sticking at the end of this this such that they will be perpendicular to each other.
I am trying to do this using the normal geometry. Is there any high-level API there in MFC for the same.
Thanks
If (dx,dy) are the differences in the x and y coodinates of the given line, you can make another line perpendicular by contriving for the differences in its coordinates to be (-dy, dx). You can scale that by any factor (-c*dy, c*dx) to change its length.
You have an existing line (x1, y1) to (x2, y2). The perpendicular line is (a1, b1) to (a2, b2), and centered on (x2, y2).
xdif = x2 - x1
ydif = y2 - y1
a1 = x2 - ydif / 2
b1 = y2 + xdif / 2
a2 = x2 + ydif / 2
b2 = y2 - xdif / 2
I think that works... I tested it for a few lines.
So if you have a line going from (1,1) to (5,3), the perpendicular line would be (5 - 2/2, 3+4/2) to (5 + 2/2, 3 - 4/2) or (4,5) to (6, 1).
You could use SetWorldTransform function from Win32 GDI API.
Sample code is here.
Let me add some c++ code based on kbelder answer. It make one vertex by origin point (x1,y1) and another vertex (x2,y2)
float GetDistance(float x1, float y1, float x2, float y2)
{
float cx = x2 - x1;
float cy = y2 - y1;
float flen = sqrtf((float)(cx*cx + cy*cy));
return flen;
}
void GetAxePoint(double x1, double y1, double x2, double y2, double& x3, double& y3, double vec_len, bool second_is_y)
{
double xdif = x2 - x1;
double ydif = y2 - y1;
if(second_is_y)
{
x3 = x1 - ydif;
y3 = y1 + xdif;
}
else
{
x3 = x1 + ydif;
y3 = y1 - xdif;
}
double vec3_len = GetDistance(x3, y3, x1, y1);
x3 = (x3-x1)/vec3_len;
y3 = (y3-y1)/vec3_len;
x3 = x1 + x3*vec_len;
y3 = y1 + y3*vec_len;
}