Stack Overflow error in GP - polynomials

My attepts to tackle computing the norm of an ideal in the 5th cyclotomic field, (denoted K5) polcyclo(5) the polynomial defining the field were successful in PARI/GP. I was also able to find its principal generators and compute more (the command on the third and fourth lines).
(09:24) gp > J = idealhnf(bnfinit(polcyclo(5)),11,x-3);
(09:24) gp > idealnorm(bnfinit(polcyclo(5)),J)
%16 = 11
(09:24) gp > Jp = bnfisprincipal(bnfinit(polcyclo(5)),J)
%17 = [[]~, [1, -1, -1, 0]~]
(09:24) gp > u = nfbasistoalg(bnfinit(polcyclo(5)),Jp[2])
%18 = Mod(-x^2 - x + 1, x^4 + x^3 + x^2 + x + 1)
(09:24) gp >
I would like to perform the same task at the same pace for K179, but this is not working. What is wrong with PARI/GP? Do I need another program to do this?
(09:25) gp > J = idealhnf(bnfinit(polcyclo(179)),359,x-2);
*** at top-level: J=idealhnf(bnfinit(polcyclo(179
*** ^--------------------
*** bnfinit: the PARI stack overflows !
current stack size: 16000000 (15.259 Mbytes)
[hint] you can increase GP stack with allocatemem()
(09:26) gp > idealnorm(bnfinit(polcyclo(179)),J)
*** at top-level: idealnorm(bnfinit(polcyclo(179
*** ^--------------------
*** bnfinit: the PARI stack overflows !
current stack size: 16000000 (15.259 Mbytes)
[hint] you can increase GP stack with allocatemem()
(09:27) gp > Jp = bnfisprincipal(bnfinit(polcyclo(179)),J)
*** at top-level: Jp=bnfisprincipal(bnfinit(polcyclo(179
*** ^--------------------
*** bnfinit: the PARI stack overflows !
current stack size: 16000000 (15.259 Mbytes)
[hint] you can increase GP stack with allocatemem()
(09:28) gp > u = nfbasistoalg(bnfinit(polcyclo(179)),Jp[2])
*** at top-level: u=nfbasistoalg(bnfinit(polcyclo(179
*** ^--------------------
*** bnfinit: the PARI stack overflows !
current stack size: 16000000 (15.259 Mbytes)
[hint] you can increase GP stack with allocatemem()
(09:29) gp >
I also increased the stack size to 1024000000 (the maximum) and it still gives the same overflow error.
Does anyone know what I should do to perform all four computations correctly?
Thanks in advance.

Related

Need help writing estimates statements in proc genmod

I'm using proc genmod to predict an outcome measured at 4 time points. The outcome is a total score on a mood inventory, which can range from 0 to 82. A lot of participants have a score of 0, so the negative binomial distribution in proc genmod seemed like a good fit for the data.
Now, I'm struggling with how to write/interpret the estimates statements. The primary predictors are TBI status at baseline (0=no/1=yes), and visit (0=baseline, 1=second visit, 2=third visit, 4=fourth visit), and an interaction of TBI status and visit.
How do I write my estimates, such that I'm getting out:
1. the average difference in mood inventory score for person with TBI versus a person without, at baseline.
and
2. the average difference in mood inventory change score for a person with TBI versus a person without, over the 4 study visits?
Below is what I have thus far, but I'm not sure how to interpret the output, also below, if indeed my code is correct.:
proc genmod data = analyze_long_3 ;
class id screen_tbi (param = ref ref = first) ;
model nsi_total = visit_cent screen_tbi screen_tbi*visit_cent /dist=negbin ;
output predicted = predstats;
repeated subject=id /type=cs;
estimate "tbi" intercept 1 visit_cent 0 0 0 0 screen_tbi 1 0 /exp;
estimate "no tbi" intercept 1 visit_cent 0 0 0 0 screen_tbi 0 1 /exp;
estimate 'longitudinal TBI' intercept 1
visit_cent -1 1 1 1
screen_tbi 1 0
screen_tbi*visit_cent 1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0 / exp;
estimate 'longitudinal no TBI ' intercept 1
visit_cent -1 1 1 1
screen_tbi 0 1
screen_tbi*visit_cent 0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1 / exp;
where sample = 1 ;
run;
The first research question is to have the average difference score, at baseline, for person with TBI versus a person without. It can be achieved by the following steps:
1) Get the estimated average log (score) when TBI = yes, and Visit = baseline;
2) Get the estimated average log (score) when TBI = no, and Visit =baseline;
3) 1) – 2) to have the difference in log(score) values
4) Exp[3)] to have the difference as percentage of change in scores
To simplify, let T=TBI levels, and V = Visit Levels. One thing to clarify, in your post, there are 4 visit points, the first as reference; therefore there should be 3 parameters for V, not four.
Taking the example of step 1), let’s try to write the ESTIMATE statement. It is a bit tricky. At first it sounds like this (T=0 and V =0 as reference):
ESTIMATE ‘Overall average’ intercept T 1 V 0 0 0;
But it is wrong. In the above statement, all arguments for V are set to 0. When all arguments are 0, it is the same as taking out V from the statement:
ESTIMATE ‘Overall average’ intercept T 1;
This is not the estimate of average for T=1 at baseline level. Rather, it produces an average for T=1, regardless of visit points, or, an average for all visit levels.
The problem is that the reference is set as V=0. In that case, SAS cannot tell the difference between estimates for the reference level, and the estimates for all levels. Indeed it always estimates the average for all levels. To solve it, the reference has to be set to -1, i.e., T=-1 and V=-1 as reference, such that the statement likes:
ESTIMATE ‘Average of T=1 V=baseline’ intercept T 1 V -1 -1 -1;
Now that SAS understands: fine! the job is to get the average at baseline level, not at all levels.
To make the reference value as -1 instead of 0, in the CLASS statement, the option should be specified as PARAM = EFFECT, not PARAM = REF. That brings another problem: once PARAM is not set as REF, SAS will ignore the user defined references. For example:
CLASS id T (ref=’…’) V (ref=’…’) / PARAM=EFFECT;
The (ref=’…’) is ignored when PARAM=EFFECT. How to let SAS make TBI=No and Visit=baseline as references? Well, SAS automatically takes the last level as the reference. For example, if the variable T is ordered ascendingly, the value -1 comes as the first level, while the value 1 comes as the last level; therefore 1 will be the reference. Conversely, if T is ordered in descending order, the value -1 comes at the end and will be used as the ref. This is achieved by the option ‘DESCENDING’ in the CLASS statement.
CLASS id T V / PARAM=EFFECT DESCENDING;
That way, the parameters are ordered as:
T 1 (TBI =1)
T -1 (ref level of TBI, i.e., TBI=no)
V 1 0 0 (for visit =4)
V 0 1 0 (visit = 3)
V 0 0 1 (visit =2)
V -1 -1 -1 (this is the ref level, visit=baseline)
The above information is reported in the ODS table ‘Class Level Information’. It is always good to check the very table each time after running PROC GENMOD. Note that the level (visit = 4) comes before the level (visit =3), visit =3 coming before visit=2.
Now, let’s talk a bit about the parameters and the model equation. As you might know, in SAS, the V for multi-levels is indeed broken down into dummy Vs. If baseline is set as ref level, the dummies will be like:
V4 = the fourth visit or baseline
V3= the third visit, or baseline
V2 = the second visit or baseline
Accordingly, the equation can be written as:
LOG(s) = b0 + b1*T + b2*V4 + b3*V3 + b4*V2
whereas:
s = the total score on a mood inventory
T = 1 for TBI status of yes, = -1 for TBI status of no
V4 = 1 for the fourth visit, = -1 for baseline
V3 = 1 for the third visit, =-1 for baseline
V2 = 1 for the second visit, = -1 for the baseline
b0 to b4 are beta estimates for the parameters
Of note, the order in the model is the same as the order defined in the statement CLASS, and the same as the order in the ODS table ‘Class Level Information’. The V4, V3, V2 have to appear in the model, all or none, i.e., if the VISIT term is to be included, V4 V3 V2 should be all introduced into the model equation. If the VISIT term is not included, none of V4, V3, and V2 should be in the equation.
With interaction terms, 3 more dummy terms must be created:
T_V4 = T*V4
T_V3 = T*V3
T_V2 = T*V2
Hence the equation with interaction terms:
Log(s) = b0 + b1*T + b2*V4 + b3*V3 + b4*V2 + b5*T_V4 + b6* T_V3 + b7* T_V2
The SAS statement of ‘ESTIMATE’ is correspondent to the model equation.
For example, to estimate an overall average for all parameters and all levels, the equation is:
[Log(S)] = b0 ;
whereas [LOG(S)] stands for the expected LOG(score). Accordingly, the statement is:
ESTIMATE ‘overall (all levels of T and V)’ INTERCEPT;
In the above statement, ‘INTERCEPT’ in the statement is correspondent to ‘b0’ in the equation
To estimate an average of log (score) for T =1, and for all levels of visit points, the equation is
[LOG(S)] = b0 + b1 * T = b0 + b1 * 1
And the statement is
ESTIMATE ‘T=Yes, V= all levels’ INTERCEPT T 1;
In the above case, ‘T 1’ in the statement is correspondent to the part “*1” in the equation (i.e., let T=1)
To estimate an average of log (score) for T =1, and for visit = baseline, the equation is:
[Log(s)] = b0 + b1*T + b2*V4 + b3*V3 + b4*V2
= b0 + b1*(1) + b2*(-1)+ b3*(-1) + b4*(-1)
The statement is:
ESTIMATE ‘T=Yes, V=Baseline’ INTERCEPT T 1 V -1 -1 -1;
‘V -1 -1 -1’ in the statement is correspondent to the values of V4, V3, and V2 in the equation. We’ve mentioned above that the dummies V4 V3 and V2 must be all introduced into the model. That is why for the V term, there are always three numbers, such as ‘V -1 -1 -1’, or ‘V 1 1 1’, etc. SAS will give warning in log if you make it like ‘V -1 -1 -1 -1’, because there are four '-1's, 1 more than required. In that case, the excessive '-1' will be ignored. On the contrary, ‘V 1 1’ is fine. It is the same as ‘V 1 1 0’. But what does 'V 1 1 0' means? To figure it out, you have to read Allison’s book (see reference).
For now, let’s carry on, and add the interaction terms. The equation:
[Log(s)] = b0 + b1*T + b2*V4 + b3*V3 + b4*V2 + b5*T_V4 + b6*T_V3 + b7*T_V2
As T_V4 = T*V4 = 1 * (-1) = -1, similarly T_V3 = -1, T_V2=-1, substitute into the equation:
[Log(s)] = b0 + b1*1 + b2*(-1)+ b3*(-1)+ b4*(-1)+ b5*(-1) + b6*(-1) + b7*(-1)
The statement is:
ESTIMATE ‘(1) T=Yes, V=Baseline, with interaction’ INTERCEPT T 1 V -1 -1 -1 T*V -1 -1 -1;
The ‘T*V -1 -1 -1’ are correspondent to the values of T_V4, T_V3 and T_V2 in the equation.
And that is the statement for step 1)!
Step 2 follows the same thoughts. To get the estimated average log (score) when TBI = no, and Visit =baseline.
T = -1, V4=-1, V3=-1, V2=-1.
T_V4 = T * V4 = (-1) * (-1) = 1
T_V3 = T * V3 = (-1) * (-1) = 1
T_V2 = T * V2 = (-1) * (-1) = 1
Substituting the values in the equation:
[Log(s)] = b0 + b1*1 + b2*(-1)+ b3*(-1)+ b4*(-1)+ b5*(1) + b6*(1) + b7*(1)
Note that the numbers: For T: 1; for V: -1 -1 -1; for interaction terms: 1 1 1
And the SAS statement:
ESTIMATE ‘(2) T=No, V=Baseline, with interaction’ INTERCEPT T 1 V -1 -1 -1 T*V 1 1 1;
The estimate results can be found in the ODS table ‘Contrast Estimate Results’.
For step 3), subtract the estimate (1) – (2), to have the difference of log(score); and for step(4), have the exponent of the diff in step 3).
For the second research question:
The average difference in mood inventory change score for a person with TBI versus a person without, over the 4 study visits.
Over the 4 study visits means for all visit levels. By now, you might have known that the statement is simpler:
ESTIMATE ‘(1) T=Yes, V=all levels’ INTERCEPT T 1;
ESTIMATE ‘(2) T=Yes, V=all levels’ INTERCEPT T -1;
Why there are no interaction terms? Because all visit levels are considered. And when all levels are considered, you do not have to put any visit-related terms into the statement.
Finally, the above approach requires some manual calculation. Indeed it is possible to make one single line of ESTIMATE statement that is equivalent to the aforementioned approach. However, the method we discussed above is way easier to understand. For more sophisticated methods, please read Allison’s book.
Reference:
1. Allison, Paul D. Logistic Regression Using SAS®: Theory and Application, Second Edition. Copyright © 2012, SAS Institute Inc.,Cary, North Carolina, USA.

dramatic error in lp_solve?

I've a simple problem that I passed to lp_solve via the IDE (5.5.2.0)
/* Objective function */
max: +r1 +r2;
/* Constraints */
R1: +r1 +r2 <= 4;
R2: +r1 -2 b1 = 0;
R3: +r2 -3 b2 = 0;
/* Variable bounds */
b1 <= 1;
b2 <= 1;
/* Integer definitions */
int b1,b2;
The obvious solution to this problem is 3. SCIP as well as CBC give 3 as answer but not lp_solve. Here I get 2. Is there a major bug in the solver?
Thank's in advance.
I had contact to the developer group that cares about lpsolve software. The error will be fixed in the next version of lpsolve.
When I tried it, I am getting 3 as the optimal value for the Obj function.
Model name: 'LPSolver' - run #1
Objective: Maximize(R0)
SUBMITTED
Model size: 3 constraints, 4 variables, 6 non-zeros.
Sets: 0 GUB, 0 SOS.
Using DUAL simplex for phase 1 and PRIMAL simplex for phase 2.
The primal and dual simplex pricing strategy set to 'Devex'.
Relaxed solution 4 after 4 iter is B&B base.
Feasible solution 2 after 6 iter, 3 nodes (gap 40.0%)
Optimal solution 2 after 7 iter, 4 nodes (gap 40.0%).
Excellent numeric accuracy ||*|| = 0
MEMO: lp_solve version 5.5.2.0 for 32 bit OS, with 64 bit REAL variables.
In the total iteration count 7, 1 (14.3%) were bound flips.
There were 2 refactorizations, 0 triggered by time and 0 by density.
... on average 3.0 major pivots per refactorization.
The largest [LUSOL v2.2.1.0] fact(B) had 8 NZ entries, 1.0x largest basis.
The maximum B&B level was 3, 0.8x MIP order, 3 at the optimal solution.
The constraint matrix inf-norm is 3, with a dynamic range of 3.
Time to load data was 0.001 seconds, presolve used 0.017 seconds,
... 0.007 seconds in simplex solver, in total 0.025 seconds.

How can I write an if condition for my variable in GLPK?

Here is my full problem:
Information:
*Max. total investment: $125
*Pay-off is the sum of the units bought x pay-off/unit
*Cost per investment: Buy-in cost + cost/unit x number of units if you buy at least one unit
*The cost is sum of the costs per investment
Constraints:
*You may not invest in both 2 and 5.
*You may invest in 1 only if you invest at least one of 2 and 3.
*You must invest at least two of 3,4,5.
*You may not invest more than max number of units.
Problem: Maximize profit : pay-off - cost
xi: # of units i ∈ {1,2,3,4,5}
yi=1 if xi>0 else yi=0
cost = sum{i in I} buyInCost_i * yi + cost-unit_i*xi
pay-off = sum{i in I} (pay-off/unit)_i*xi
profit = pay-off - cost
Maximize profit
Subject to
y2+y5 <= 1
y1<= y2+y3
y3+y4+y5 >= 2
x1<=5, x2<=4, x3<=5, x4<=7, x5<=3
cost<=125
Here is my question:
For example I have this binary variable y
yi=1 if xi>0 else yi=0 and i ∈ {1,2,3,4,5}
I declared i as a data set
set I;
data;
set I := 1 2 3 4 5;
I don't know how to add if else condition to y variable in glpk. Can you please help me out?
My modelling :
set I;
/*if x[i]>0 y[i]=1 else y[i]=0 ?????*/
var y{i in I}, binary;
param a{i in I};
/* buy-in cost of investment i */
param b{i in I};
/* cost per unit of investment i */
param c{i in I};
/* pay-off per unit of investment i */
param d{i in I};
/* max number of units of investment i */
var x{i in I} >=0;
/* Number of units that is bought of investment i */
var po := sum{i in I} c[i]*x[i];
var cost := sum{i in I} a[i]*y[i] + b[i]*x[i];
maximize profit: po-cost;
s.t. c1: y[2]+y[5]<=1;
s.t. c2: y[1]<y[2]+y[3];
s.t. c3: y[3]+y[4]+y[5]>=2;
s.t. c4: x[1]<=5
x[2]<=4
x[3]<=5
x[4]<=7
x[5]<=3;
s.t. c5: cost <=125;
s.t. c6{i in I}: M * y[i] > x[i]; // if condition of y[i]
set I := 1 2 3 4 5;
param a :=
1 25
2 35
3 28
4 20
5 40;
param b :=
1 5
2 7
3 6
4 4
5 8;
param c :=
1 15
2 25
3 17
4 13
5 18;
param d :=
1 5
2 4
3 5
4 7
5 3;
param M := 10000;
I am getting this syntax error:
problem.mod:21: syntax error in variable statement
Context: ...I } ; param d { i in I } ; var x { i in I } >= 0 ; var po :=
MathProg model processing error
You can't directly do that (there is no way to write 'directly' an if constraint in a LP).
However, there are workarounds for this.
For example, you can write:
M * yi > xi
where M is a large constant (greater than any value of xi).
This way:
if xi > 0, then the constraint is equivalent to yi > 0, that is yi == 1 since yi is binary (if M is large enough).
if xi == 0, then the constraint is always verified, and yi will be equal to 0 since your objective is increasing with yi and you are minimizing.
in both case, the constraint is equivalent to the if test.

core dump inside stored procedure

i am executing the stored procedure from c++ as below:
dbsqlexec(_bsmdb);
when i execute the stored procedure from command line it gives me proper output.
2> declare #apicounter int
exec CapApi_CountTgRxObjects #apicounter output, 'subnetwork="NETSim_BAG",bsc=3> "BAG01",site="RS8000#0",mo="RXOTG-0",%'
4> go
(return status = 0)
Return parameters:
-----------
9
(1 row affected)
but in c++ code its giving me core dump.
below is the stored procedure.
CREATE PROCEDURE CapApi_CountTgRxObjects (#count_out int output,#tgdn_in varchar
(200))
AS
BEGIN
declare #likeGE varchar(200),
#likeLT varchar(200),
#rc int
/*
** Build the simulated like strings
*/
exec #rc =
CapApi_likeFix #likeStr=#tgdn_in, #likeGE=#likeGE OUT, #likeLT=#likeLT OUT
if (#rc != 0)
begin
return 1
end
/*
** Execute the Worker procedure
*/
exec CapApi_CountTgRxObjectsW #count_out=#count_out OUT, #tgdn_GE=#like
GE, #tgdn_LT=#likeLT
/*
** Normal exit point
*/
return 0
END
(3 rows affected)
(return status = 0)
The core trace is below:
----------------- lwp# 1 / thread# 1 --------------------
fcb16576 t_splay (140e6958) + 1f
fcb16454 t_delete (140e6958) + 2a
fcb1618e realfree (140e6918) + 58
fcb16797 cleanfree (0) + 44
fcb15cb3 _malloc_unlocked (8, 83df9d0, 140e68c8, fc4df378, 802a988, fc44ee9b) + ad
fcb15bdc malloc (4) + 34
fc44ee9b comn_malloc (4) + 1b
fc438012 dbsvretval (83df9d0) + 3f6
fc4369c5 dbsqlok (83df9d0) + 115
fc436668 dbsqlexec (83df9d0, 0) + 30
fec1c19f __1cICACCC_tgWCheck_BTS_capabilities6FrnGcna_Mo_ri_v_ (802c24c, 821cc40) + 3ff
fec1d4a2 __1cICACCC_tgOPerform_checks6FrnGcna_Mo_ri_v_ (802c24c, 821cc40) + 272
fec16953 __1cICACCC_tgNDirect_checks6FrknGvector4CpnGcna_MO___rikikb_v_ (802c2b8, 821cc40, 2, 0, 0, 0) + 3c3
feaa760d __1cJCACCC_bscNDirect_checks6FpnTCACCC_consist_check_rknGvector4CpnGcna_MO___rikikbrfkf_v_ (821cc28, 802c4b0, 821cc40, 2, 0, 821cc54) + 64d
fe9b25ae __1cTCACCC_consist_checkJcheck_bsc6M_v_ (821cc28, 0) + 17e
fe9ac7de __1cTCACCC_consist_checkNcheck_perform6M_i_ (821cc28, 0) + 14e
fe9acd2b __1cTCACCC_consist_checkFcheck6M_i_ (821cc28, 0) + 9b
0805c656 main (2, 802d1ac, 802d1b8, 802d1a0) + 11a6
08051ebd _start (2, 802d95c, 802d97d, 0, 802d993, 802d9cb) + 7d
----------------- lwp# 2 / thread# 2 --------------------
fcb7ae55 ___nanosleep (78) + 15
fd52a4c2 run (821f668) + 1a2
fcb7875b _thr_setup (fc040200) + 4e
fcb78a60 _lwp_start (fc040200, 0, 0, fbffeff8, fcb78a60, fc040200)
Could anybody please help me finding the problem.....
It looks like your heap is corrupt (it's dying down inside mallocs free-store management code, not in anything DB-related).
Try using a memory-checker tool to identify when the heap gets corrupted (such as Purify if you have it, or IIRC Solaris has a drop-in replacement debugging malloc lib).
This is a C++ bug, unrelated to the stored proc.
Try using libumem, if it is a corruption is due to double delete, it can be easily found. Here is the link.
http://developers.sun.com/solaris/articles/libumem_library.html
You don't even need to recompile the code, just define the variables mentioned in the link in the startup script.

How does the capacity of std::vector grow automatically? What is the rate?

I had been going through the Book:
C++ Primer, Third Edition By Stanley B. Lippman, Josée Lajoie, found 1 mistake in the program given under the Article 6.3 How a vector Grows Itself, this program missed a "<" in the couts:
#include <vector>
#include <iostream>
using namespace std;
int main() {
vector<int> ivec;
cout < "ivec: size: " < ivec.size() < " capacity: " < ivec.capacity() < endl;
for (int ix = 0; ix < 24; ++ix) {
ivec.push_back(ix);
cout < "ivec: size: " < ivec.size()
< " capacity: " < ivec.capacity() < endl;
}
}
Later within that article:
"Under the Rogue Wave implementation, both the size and the capacity
of ivec after its definition are 0. On inserting the first element,
however, ivec's capacity is 256 and its size is 1."
But, on correcting and running the code i get the following output:
ivec: size: 0 capacity: 0
ivec[0]=0 ivec: size: 1 capacity: 1
ivec[1]=1 ivec: size: 2 capacity: 2
ivec[2]=2 ivec: size: 3 capacity: 4
ivec[3]=3 ivec: size: 4 capacity: 4
ivec[4]=4 ivec: size: 5 capacity: 8
ivec[5]=5 ivec: size: 6 capacity: 8
ivec[6]=6 ivec: size: 7 capacity: 8
ivec[7]=7 ivec: size: 8 capacity: 8
ivec[8]=8 ivec: size: 9 capacity: 16
ivec[9]=9 ivec: size: 10 capacity: 16
ivec[10]=10 ivec: size: 11 capacity: 16
ivec[11]=11 ivec: size: 12 capacity: 16
ivec[12]=12 ivec: size: 13 capacity: 16
ivec[13]=13 ivec: size: 14 capacity: 16
ivec[14]=14 ivec: size: 15 capacity: 16
ivec[15]=15 ivec: size: 16 capacity: 16
ivec[16]=16 ivec: size: 17 capacity: 32
ivec[17]=17 ivec: size: 18 capacity: 32
ivec[18]=18 ivec: size: 19 capacity: 32
ivec[19]=19 ivec: size: 20 capacity: 32
ivec[20]=20 ivec: size: 21 capacity: 32
ivec[21]=21 ivec: size: 22 capacity: 32
ivec[22]=22 ivec: size: 23 capacity: 32
ivec[23]=23 ivec: size: 24 capacity: 32
Is the capacity increasing with the formula 2^N where N is the initial capacity? Please explain.
The rate at which the capacity of a vector grows is required by the standard to be exponential (which, IMHO, is over-specification). The standard specifies this in order to meet the amortized constant time requirement for the push_back operation. What amortized constant time means and how exponential growth achieves this is interesting.
Every time a vector's capacity is grown the elements need to be copied. If you 'amortize' this cost out over the lifetime of the vector, it turns out that if you increase the capacity by an exponential factor you end up with an amortized constant cost.
This probably seems a bit odd, so let me explain to you how this works...
size: 1 capacity 1 - No elements have been copied, the cost per element for copies is 0.
size: 2 capacity 2 - When the vector's capacity was increased to 2, the first element had to be copied. Average copies per element is 0.5
size: 3 capacity 4 - When the vector's capacity was increased to 4, the first two elements had to be copied. Average copies per element is (2 + 1 + 0) / 3 = 1.
size: 4 capacity 4 - Average copies per element is (2 + 1 + 0 + 0) / 4 = 3 / 4 = 0.75.
size: 5 capacity 8 - Average copies per element is (3 + 2 + 1 + 1 + 0) / 5 = 7 / 5 = 1.4
...
size: 8 capacity 8 - Average copies per element is (3 + 2 + 1 + 1 + 0 + 0 + 0 + 0) / 8 = 7 / 8 = 0.875
size: 9 capacity 16 - Average copies per element is (4 + 3 + 2 + 2 + 1 + 1 + 1 + 1 + 0) / 9 = 15 / 9 = 1.67
...
size 16 capacity 16 - Average copies per element is 15 / 16 = 0.938
size 17 capacity 32 - Average copies per element is 31 / 17 = 1.82
As you can see, every time the capacity jumps, the number of copies goes up by the previous size of the array. But because the array has to double in size before the capacity jumps again, the number of copies per element always stays less than 2.
If you increased the capacity by 1.5 * N instead of by 2 * N, you would end up with a very similar effect, except the upper bound on the copies per element would be higher (I think it would be 3).
I suspect an implementation would choose 1.5 over 2 both to save a bit of space, but also because 1.5 is closer to the golden ratio. I have an intuition (that is currently not backed up by any hard data) that a growth rate in line with the golden ratio (because of its relationship to the fibonacci sequence) will prove to be the most efficient growth rate for real-world loads in terms of minimizing both extra space used and time.
To be able to provide amortized constant time insertions at the end of the std::vector, the implementation must grow the size of the vector (when needed) by a factor K>1 (*), such that when trying to append to a vector of size N that is full, the vector grows to be K*N.
Different implementations use different constants K that provide different benefits, in particular most implementations go for either K = 2 or K = 1.5. A higher K will make it faster as it will require less grows, but it will at the same time have a greater memory impact. As an example, in gcc K = 2, while in VS (Dinkumware) K = 1.5.
(*) If the vector grew by a constant quantity, then the complexity of push_back would become linear instead of amortized constant. For example, if the vector grew by 10 elements when needed, the cost of growing (copy of all element to the new memory address) would be O( N / 10 ) (every 10 elements, move everything) or O( N ).
Just to add some mathematic proof on the time complexity on vector::push_back, say the size of vector is n, what we care about here is the number of copies happened so far, say y, notice the copy happens every time you grow the vector.
Grow by factor of K
y = K^1 + K^2 + K^3 ... K^log(K, n)
K*y = + K^2 + K^3 ... K^log(K, n) + K*K^log(K, n)
K*y-y = K*K^log(K, n) - K
y = K(n-1)/(K-1) = (K/(K-1))(n-1)
T(n) = y/n = (K/(K-1)) * (n-1)/n < K/(K-1) = O(1)
K/(K-1) is a constant, and see the most common cases:
K=2, T(n) = 2/(2-1) = 2
K=1.5, T(n) = 1.5/(1.5-1) = 3
and actually there is a reason of choosing K as 1.5 or 2 in different implementations, see this graph: as T(n) reaching the minimum when K is around 2, there is not much benefit on using a larger K, at the cost of allocating more memory
Grow by constant quantity of C
y = C + 2*C + 3*C + 4*C + ... (n/C) * C
= C(1+2+3+...+n/C), say m = n/C
= C*(m*(m-1)/2)
= n(m-1)/2
T(n) = y/n = (n(m-1)/2)/n = (m-1)/2 = n/2C - 1/2 = O(n)
As we could see it is liner
The capacity of the vector is completely implementation-dependent, no one can tell how it's growing..
Are you using the "Rogue Wave" implementation?
How capacity grows is up to the implementation. Yours use 2^N.
Yes, the capacity doubles each time it is exceeded. This is implementation dependent.
before pushing back an element the vector check if the size is greater than it's capacity like bellow
i will explain it with reserve function :
void push_back(const value_type &val) //push_back actual prototype
{
if (size_type < 10)
reserve(size_type + 1);
else if (size_type > (_capacity / 4 * 3))
reserve(_capacity + (this->_capacity / 4));
//then the vector get filled with value
}
size_type : the vector size.
_capacity : the vector _capacity.