I am looking to take two csvs (read in through pandas), and combine them into a single 3D DataFrame.
The formats are similar to this:
Table1:
key1 key2 key3 value
x1 y1 z1 1
x1 y2 z1 2
x1 y3 z1 3
x2 y1 z1 4
x2 y2 z1 5
x2 y3 z2 6
x3 y1 z2 7
x3 y2 z2 8
x3 y3 z2 9
Table2:
key2 key3 value x1 x2 x3
y1 z1 0
y2 z1 1
y3 z1 2
y1 z2 3
y2 z2 4
y3 z2 5
My goal is that in table2 the values under the 'x' headers should be a lookup of the value in table1 (using all 3 keys) multiplied by the value in table2.
IIUC:
d1.set_index(
['key2', 'key3', 'key1']
).value.unstack().rename_axis(None, 1).reset_index()
key2 key3 x1 x2 x3
0 y1 z1 1.0 4.0 NaN
1 y1 z2 NaN NaN 7.0
2 y2 z1 2.0 5.0 NaN
3 y2 z2 NaN NaN 8.0
4 y3 z1 3.0 NaN NaN
5 y3 z2 NaN 6.0 9.0
Related
I have following problem that I try to solve with LPSolve IDE:
min: x1;
r_1: 1.08 - k <= x1;
r_2: -1.08 + k <= x1;
c_1: y1 + y2 + y3 = k;
c_2: 2.29 a1 y1 + 2.28 a2 y1 + 2.27 a3 y1 = 1;
c_3: 1.88 b1 y2 + 1.89 b2 y2 + 1.9 b3 y2 = 1;
c_4: 8.98 c1 y3 + 8.99 c2 y3 + 9.0 c3 y3 = 1;
c_14: a1+a2+a3=1;
c_15: b1+b2+b3=1;
c_16: c1+c2+c3=1;
bin a1,a2,a3,b1,b2,b3,c1,c2,c3;
Not sure why I get output from LPSolve as INFEASIBLE when I can use following param values to solve this:
a1=0, a2=1, a3=0
b1=0, b2=1, b3=0
c1=0, c2=1, c3=0
0 + 2.28 0.438596491 + 0 = 1
0 + 1.89 0.529100529 + 0 = 1
0 + 8.99 0.111234705 + 0 = 1
0.438596491 + 0.529100529 + 0.111234705 = 1.0789 (this is k)
1.08 - 1.0789 == 0.0011 <= x1
-1.08 + 1.0789 == -0.0011 <= x1
x1 = 0.0011
Am I formulating the problem in a wrong way, or doing something else wrong? If I relax that =1 constraint to >=1 there are some results, but I need it to be 1 (as it is in my solution).
Lpsolve is for linear models only. You have products of variables in the model such as 2.29 a1 y1. Lpsolve can not solve such quadratic models.
Too bad you don't get a good error message. I guess they never expected this input.
It is noted that products of binary and continuous variables can be linearized resulting in so-called big-M constraints (see link).
This is really a duplicate of lpsolve - unfeasible solution, but I have example of 1. Embarrassingly, this was an earlier question from the same poster!
Objective function => x1 - 2x2
Subject to =>
x2 <= 5
x1 - x2 >= 2
x1 ,x2, x3 >= 0
Maximize?
convert to standard form :
Maximize -> -x1 + 2x2
Subject to ->
x2 <= 5
-x1 + x2 <= -2
convert to slack form :
Z = -x1 + 2x2
x3 = 5 - x2
x4 = -2 +x1 -x2
Basic solution (0,0,5,-2)
Can I found optimal solution in here? If not why?
I am using CGAL QP package to solve the following quadratic problem:
I am using the following MPS file to define the problem (first_qp.mps):
NAME first_qp
ROWS
E c0
COLUMNS
x0 c0 1
x1 c0 1
x2 c0 1
x3 c0 1
x4 c0 1
x5 c0 1
x6 c0 1
x7 c0 1
x8 c0 1
RHS
rhs c0 1
BOUNDS
UP BND x0 0.2
UP BND x1 0.2
UP BND x2 0.2
UP BND x3 0.2
UP BND x4 0.2
UP BND x5 0.2
UP BND x6 0.2
UP BND x7 0.2
UP BND x8 0.2
QUADOBJ
x0 x0 39.07
x1 x0 25.54
x2 x0 27.29
x3 x0 28.56
x4 x0 24.38
x5 x0 10.23
x6 x0 11.12
x7 x0 15.26
x8 x0 25.17
x1 x1 38.82
x2 x1 18.11
x3 x1 20.67
x4 x1 17.20
x5 x1 8.10
x6 x1 12.41
x7 x1 9.82
x8 x1 14.69
x2 x2 39.97
x3 x2 26.82
x4 x2 22.55
x5 x2 12.81
x6 x2 10.90
x7 x2 16.17
x8 x2 26.42
x3 x3 29.00
x4 x3 24.61
x5 x3 10.37
x6 x3 10.65
x7 x3 14.93
x8 x3 23.61
x4 x4 49.71
x5 x4 7.04
x6 x4 6.20
x7 x4 17.41
x8 x4 25.87
x5 x5 12.47
x6 x5 8.21
x7 x5 7.53
x8 x5 9.73
x6 x6 19.02
x7 x6 7.47
x8 x6 7.87
x7 x7 16.04
x8 x7 14.95
x8 x8 28.90
ENDATA
Note that I am using QUADOBJ to define the D matrix. In case of QUADOBJ, only the entries of 2D on or below the diagonal must be specified, entries above the diagonal are deduced from symmetry. I then feed this file to the solver (first_qp_from_mps.cpp):
// example: read quadratic program in MPS format from file
// the QP below is the first quadratic program example in the user manual
#include <iostream>
#include <fstream>
#include <CGAL/basic.h>
#include <CGAL/QP_models.h>
#include <CGAL/QP_functions.h>
// choose exact integral type
#ifdef CGAL_USE_GMP
#include <CGAL/Gmpz.h>
typedef CGAL::Gmpz ET;
#else
#include <CGAL/MP_Float.h>
typedef CGAL::MP_Float ET;
#endif
// program and solution types
typedef CGAL::Quadratic_program_from_mps<int> Program;
typedef CGAL::Quadratic_program_solution<ET> Solution;
int main() {
std::ifstream in ("first_qp.mps");
Program qp(in); // read program from file
assert (qp.is_valid()); // we should have a valid mps file
// solve the program, using ET as the exact type
Solution s = CGAL::solve_quadratic_program(qp, ET());
// output solution
std::cout << s;
return 0;
}
The project compiles and the executable file runs and returns the solution vector (0 1 0 0 0 0 0 0 0) and the value of the objective function is 0. I know this is not correct. The solution vector does not satisfy the upper bound constraint. The objective function evaluated at this solution vector cannot be equal to 0.
Am I making a mistake in specifying the MPS file for my quadratic programming problem, or is there something I need to adjust in the way the solver searches for a solution? Could my problem be related to the exact type that CGAL uses?
For instance, I have tried changing <int> to <double> in the following line
typedef CGAL::Quadratic_program_from_mps<int> Program;
The program compiled, but when I ran the executable the solver returned that no solution was feasible. But I know there is a feasible solution - I have found one using the solver in Excel.
You should indeed use instead of in the Program type. But on top of that, ET should be typedef'd as CGAL::Gmpzf (exact floating point type), and not CGAL::Gmpz (exact integral type).
I wrote a simple Fortran program for the three body problem using Euler-Richardson algorithm. For some reason the output files give nothing but NaN. Would the issue be solved by using subroutines or functions?
PROGRAM Threebody
IMPLICIT none
!************** Variable declarations ***************
!real(8), parameter :: G = 6.6738D-11
INTEGER :: IOSTAT, dt, t_step, io_error, i
DOUBLE PRECISION:: st, g, m0, m1, m2, force0x, force0y, force1x, force1y, force2x, force2y, n0, n1, n2, x0,v0,x1,v1,x2,v2,y0,w0,y1
DOUBLE PRECISION:: w1,y2,w2
m0 = 1.d0 ! the masses of the three bodies
m1 = 1.d0
m2 = 1.d0
g = 9.80d0 ! m/s^2 s due to gravity
t_step = 1
y0 = 0.d0
x0 = 0.d0
v0 = 0.d0
w0 = 0.d0
y1 = 0.d0
x1 = 1.d0
v1 = 2.d0
w1 = 1.118d0
y2 = 0.d0
x2 = -1.d0
v2 = -1.118d0
w2 = 0.d0
dt = 1 ! time step
OPEN(unit=5, file='out.txt', status='replace',action='write', IOSTAT=io_error)
OPEN(unit=6, file='out1.txt', status='replace',action='write', IOSTAT=io_error)
OPEN(unit=7, file='out2.txt', status='replace',action='write', IOSTAT=io_error)
! particle no.1
DO i=0,1000,t_step ! 1000 is the total time
st=i!/10.d0
n0 = sqrt(((x2 - x1)**2 + (y2 - y1)**2)**3)
force0x = (- m1*(( x0 - x1 ) / n2)) - (m2*(( x0 - x2 ) / n1))
x0 = x0 + (w0 + 0.5*force0x*st)*st
w0 = w0 + force0x*st
force0y = (- m1 * (( y0 - y1 ) / n2)) - (m2 * (( y0 - y2 ) / n1))
y0 = y0 + (v0 + 0.5*force0y*st)*st
v0 = v0 + force0y*st
WRITE(5,*) i*t_step, x0, y0, n0
END DO
! particle no.2
DO i=0,1000,dt
st=i
n1 = sqrt(((x0 - x2 )**2 + (y0 - y2)**2)**3)
force1x = (- m2 * (( x1 - x0 ) / (n0))) - m0 * (( x1 - x2 ) / n2)
x1 = x1 + (w1 + 0.5*force1x*st)*st
w1 = w1 + force1x*st
force1y = (- m2 * ( y1 - y0 ) / n0) - (m0 * (( y1 - y2 ) / n2))
y1 = y1 + (v1 + 0.5*force1y*st)*st
v1 = v1 + force1y*st
WRITE(6,*) i*t_step, x1, y1
END DO
! particle no.3
DO i=0,1000,dt
st=i
n2 = sqrt ( ( ( x1 - x0 )**2 + ( y1 - y0 )**2 )**3 )
force2x = (- m0 * (( x2 - x0 ) / n1)) - (m1 * (( x2 - x1 ) / n0))
x2 = x2 + (w2 + 0.5*force2x*st)*st
w2 = w2 + force2x*st
force2y = (- m0 * (( y2 - y0 ) / n1)) - (m1 * (( y2 - y1 ) / n0))
y2 = y2 + (v2 + 0.5*force2y*st)*st
v2 = v2 + force2y*st
WRITE(7,*) i*t_step, x2, y2, n2
END DO
CLOSE(unit=5)
CLOSE(unit=6)
CLOSE(unit=7)
END PROGRAM Threebody
Compiling with -Wall in gfortran gives you a hint:
f.f90:56:0: warning: ‘n1’ may be used uninitialized in this function [-Wmaybe-uninitialized]
force0x = (- m1*(( x0 - x1 ) / n2)) - (m2*(( x0 - x2 ) / n1))
n1 and n2 are not initialized so they may contain anything. In my case, 6.95e-310, causing force0x to give NaN.
To debug this, you can print the variables on screen to check when they are becoming NaN. For that, please don't use units below 10, because they may be reserved. I think that Unit 5 is standard output so by reusing it you cannot print anything on screen.
I'm trying to implement this: from https://docs.google.com/viewer?url=http://www.tinaja.com/glib/bezdist.pdf&pli=1
The following BASIC program uses the method of finding distance. The
program also searches for the minimum squared distance between points and
a curve.
REM BEZIER.BAS JIM 20DEC92 12:37
DATA 2,3,5,8,8,14,11,17,14,17,16,15,18,11,-1
DATA 2,10,5,12,8,11,11,8,14,6,17,5,19,10,-1
DATA 2,5,5,7,8,8,12,12,13,14,12,17,10,18,8,17,7,14,8,12,12,8,15,7,18,5,-1
OPEN "BEZIER.OUT" FOR OUTPUT AS #1
OPEN "BEZ.ps" FOR OUTPUT AS #2
CLS
psscale = 20
FOR example% = 1 TO 3
REDIM rawdata(32)
FOR I% = 0 TO 32
READ rawdata(I%)
IF rawdata(I%) < 0! THEN EXIT FOR
NEXT I%
n% = I% - 1
PRINT "Example "; example%; (n% + 1) \ 2; " points"
PRINT #1, ""
PRINT #1, "Example "; example%; (n% + 1) \ 2; " points"
PRINT #1, " #
x
y"
J% = 0
FOR I% = 0 TO n% STEP 2
J% = J% + 1
PRINT #1, USING "### ####.### ####.###"; J%; rawdata(I%); rawdata(I% + 1)
LPRINT USING "####.### ####.### 3 0 360 arc fill"; rawdata(I%) * psscale; rawdata(I% + 1) * psscale
PRINT #2, USING "####.### ####.### 3 0 360 arc fill"; rawdata(I%) * psscale; rawdata(I% + 1) * psscale
NEXT I%
x0 = rawdata(0)
y0 = rawdata(1)
x1 = rawdata(2)
y1 = rawdata(3)
x2 = rawdata(n% - 3)
y2 = rawdata(n% - 2)
x3 = rawdata(n% - 1)
y3 = rawdata(n%)
IF example% = 3 THEN
’special guess for loop
x1 = 8 * x1 - 7 * x0
y1 = 8 * y1 - 7 * y0
x2 = 8 * x2 - 7 * x3
y2 = 8 * y2 - 7 * y3
ELSE
x1 = 2 * x1 - x0
y1 = 2 * y1 - y0
x2 = 2 * x2 - x3
y2 = 2 * y2 - y3
END IF
GOSUB distance
LPRINT ".1 setlinewidth"
PRINT #2, ".1 setlinewidth"
GOSUB curveto
e1 = totalerror
FOR Retry% = 1 TO 6
PRINT
PRINT "Retry "; Retry%
PRINT #1, "Retry "; Retry%
PRINT #1, " x1
y1
x2
y2
error"
e3 = .5
x1a = x1
DO
x1 = x1 + (x1 - x0) * e3
GOSUB distance
e2 = totalerror
IF e2 = e1 THEN
EXIT DO
ELSEIF e2 > e1 THEN
x1 = x1a
e3 = -e3 / 3
IF ABS(e3) < .001 THEN EXIT DO
ELSE
e1 = e2
x1a = x1
END IF
LOOP
e3 = .5
y1a = y1
DO
y1 = y1 + (y1 - y0) * e3
GOSUB distance
e2 = totalerror
IF e2 = e1 THEN
EXIT DO
ELSEIF e2 > e1 THEN
y1 = y1a
e3 = -e3 / 3
IF ABS(e3) < .01 THEN EXIT DO
ELSE
e1 = e2
y1a = y1
END IF
LOOP
e3 = .5
x2a = x2
DO
x2 = x2 + (x2 - x3) * e3
GOSUB distance
e2 = totalerror
IF e2 = e1 THEN
EXIT DO
ELSEIF e2 > e1 THEN
x2 = x2a
e3 = -e3 / 3
IF ABS(e3) < .01 THEN EXIT DO
ELSE
e1 = e2
x2a = x2
END IF
LOOP
e3 = .5
y2a = y2
DO
y2 = y2 + (y2 - y3) * e3
GOSUB distance
e2 = totalerror
IF e2 = e1 THEN
EXIT DO
ELSEIF e2 > e1 THEN
y2 = y2a
e3 = -e3 / 3
IF ABS(e3) < .01 THEN EXIT DO
ELSE
e1 = e2
y2a = y2
END IF
LOOP
IF Retry% = 6 THEN
LPRINT "1 setlinewidth"
PRINT #2, "1 setlinewidth"
END IF
GOSUB curveto
NEXT Retry%
LPRINT "100 200 translate"
PRINT #2, "100 200 translate"
NEXT example%
LPRINT "showpage"
PRINT #2, "showpage"
CLOSE #1
CLOSE #2
END
’
Bezier:
x = a0 + u * (a1 + u * (a2 + u * a3))
y = b0 + u * (b1 + u * (b2 + u * b3))
dx4 = x - x4: dy4 = y - y4
dx = a1 + u * (2 * a2 + u * 3 * a3)
dy = b1 + u * (2 * b2 + u * 3 * b3)
z = dx * dx4 + dy * dy4
s = dx4 * dx4 + dy4 * dy4
RETURN
’
distance:
totalerror = 0!
a3 = (x3 - x0 + 3 * (x1 - x2)) / 8
b3 = (y3 - y0 + 3 * (y1 - y2)) / 8
a2 = (x3 + x0 - x1 - x2) * 3 / 8
b2 = (y3 + y0 - y1 - y2) * 3 / 8
a1 = (x3 - x0) / 2 - a3
b1 = (y3 - y0) / 2 - b3
a0 = (x3 + x0) / 2 - a2
b0 = (y3 + y0) / 2 - b2
FOR I% = 2 TO n% - 2 STEP 2
x4 = rawdata(I%)
y4 = rawdata(I% + 1)
stepsize = 2 / (n% + 1)
FOR u = -1! TO 1.01 STEP stepsize
GOSUB Bezier
IF s = 0! THEN u1 = u: z1 = z: s1 = s: EXIT FOR
IF u = -1! THEN u1 = u: z1 = z: s1 = s
IF s < s1 THEN u1 = u: z1 = z: s1 = s
NEXT u
IF s1 <> 0! THEN
u = u1 + stepsize
IF u > 1! THEN u = 1! - stepsize
DO
GOSUB Bezier
IF s = 0! THEN EXIT DO
IF z = 0! THEN EXIT DO
u2 = u
z2 = z
temp = z2 - z1
IF temp <> 0! THEN
u = (z2 * u1 - z1 * u2) / temp
ELSE
u = (u1 + u2) / 2!
END IF
IF u > 1! THEN
u = 1!
ELSEIF u < -1! THEN
u = -1!
END IF
IF ABS(u - u2) < .0001 THEN EXIT DO
u1 = u2
z1 = z2
LOOP
END IF
totalerror = totalerror + s
NEXT I%
PRINT totalerror;
PRINT #1, USING "####.### ####.### ####.### ####.### ######.###"; x1; y1; x2; y2; totalerror
RETURN
’
curveto:
LPRINT USING "####.### ####.### moveto"; x0 * psscale; y0 * psscale
PRINT #2, USING "####.### ####.### moveto"; x0 * psscale; y0 * psscale
F$ = "####.### ####.### ####.### ####.### ####.### ####.### curveto stroke"
LPRINT USING F$; x1 * psscale; y1 * psscale; x2 * psscale; y2 * psscale; x3 * psscale; y3 * psscale
PRINT #2, USING F$; x1 * psscale; y1 * psscale; x2 * psscale; y2 * psscale; x3 * psscale; y3 * psscale
RETURN
I want to implement it in c++ because I'm trying to get my algorithm to best fit beziers from points.
What would the above look like in pseudo-code or c / c++?
thanks
The best approach here is to split the code bit by bit and do minor refactorings until it's in a usable state. Data can be changed into global variables at first.
Then start taking small chunks of the code and turning them into functions. At first they'll just use a bunch of global data. As you rewrite the pieces into C++ things will become more clear.
Once you have most of the code built out functionally, then you can start refactoring the variables. The goal would be to remove all the global non-const data and have all the working data be locals. const values can remain namespace level initialized data.
Finally once you have it procedure-based, you can decide if it's worth the effort to encapsulate the work into objects and methods. Depending on how long the program needs to be maintained grouping the data and methods may be a good long-term step.