I wonder what's the algorithm of make_heap in in C++ such that the complexity is 3*N? Only way I can think of to make a heap by inserting elements have complexity of O(N Log N). Thanks a lot!
You represent the heap as an array. The two elements below the i'th element are at positions 2i+1 and 2i+2. If the array has n elements then, starting from the end, take each element, and let it "fall" to the right place in the heap. This is O(n) to run.
Why? Well for n/2 of the elements there are no children. For n/4 there is a subtree of height 1. For n/8 there is a subtree of height 2. For n/16 a subtree of height 3. And so on. So we get the series n/22 + 2n/23 + 3n/24 + ... = (n/2)(1 * (1/2 + 1/4 + 1/8 + . ...) + (1/2) * (1/2 + 1/4 + 1/8 + . ...) + (1/4) * (1/2 + 1/4 + 1/8 + . ...) + ...) = (n/2) * (1 * 1 + (1/2) * 1 + (1/4) * 1 + ...) = (n/2) * 2 = n. Or, formatted maybe more readably to see the geometric series that are being summed:
n/2^2 + 2n/2^3 + 3n/2^4 + ...
= (n/2^2 + n/2^3 + n/2^4 + ...)
+ (n/2^3 + n/2^4 + ...)
+ (n/2^4 + ...)
+ ...
= n/2^2 (1 + 1/2 + 1/2^4 + ...)
+ n/2^3 (1 + 1/2 + 1/2^3 + ...)
+ n/2^4 (1 + 1/2 + 1/2^3 + ...)
+ ...
= n/2^2 * 2
+ n/2^3 * 2
+ n/2^4 * 2
+ ...
= n/2 + n/2^2 + n/2^3 + ...
= n(1/2 + 1/4 + 1/8 + ...)
= n
And the trick we used repeatedly is that we can sum the geometric series with
1 + 1/2 + 1/4 + 1/8 + ...
= (1 + 1/2 + 1/4 + 1/8 + ...) (1 - 1/2)/(1 - 1/2)
= (1 * (1 - 1/2)
+ 1/2 * (1 - 1/2)
+ 1/4 * (1 - 1/2)
+ 1/8 * (1 - 1/2)
+ ...) / (1 - 1/2)
= (1 - 1/2
+ 1/2 - 1/4
+ 1/4 - 1/8
+ 1/8 - 1/16
+ ...) / (1 - 1/2)
= 1 / (1 - 1/2)
= 1 / (1/2)
= 2
So the total number of "see if I need to fall one more, and if so which way do I fall? comparisons comes to n. But you get round-off from discretization, so you always come out to less than n sets of swaps to figure out. Each of which requires at most 3 comparisons. (Compare root to each child to see if it needs to fall, then the children to each other if the root was larger than both children.)
Related
I was working on a simple problem and I came up with a recursive function in C++, below is my function.
void test(int arr[],int n,int x = 0){
cout<<arr[x];
for(int i = x+1;i < n;i++){
test(arr, n, i);
}
}
I wonder what will be the time complexity of the above function if anyone can calculate the time complexity for the above method it will be a great help in improving my function.
You can write its recurrent relation likes the following:
T(n) = T(n-1) + T(n-2) + ... + T(1) + 1
Indeed T'(x) is T(n - x) and T(1) = 1 (The last one in the realtion is is for cout). We can see:
T(2) = T(1) + 1 = 2
T(3) = T(2) + T(1) + 1 = 2 + 1 + 1 = 4
T(4) = 4 + 2 + 1 + 1 = 2^2 + 2^1 + 2^0 + 1 = 8
T(5) = 8 + 4 + 2 + 1 + 1 = 2^3 + 2^2 + 2^1 + 2^0 + 1 = 16
.
.
.
T(n) = 2^{n-2} + 2^{n-1} + ... + 2^0 + 1 = 2^{n-1}
Hence, T(n) = \Theta(2^n).
I am working with GP and minimum polynomials as follows running on ASUS x75:
(19:25) gp > elt=Mod(a*x^3+b*x^2+c*x+d,('x^5-1)/('x-1))
%122 = Mod(a*x^3 + b*x^2 + c*x + d, x^4 + x^3 + x^2 + x + 1)
(19:25) gp > (poly=minpoly(elt,x='x))
%123 = x^4 + (a + (b + (c - 4*d)))*x^3 + (a^2 + (-3*b + (2*c - 3*d))*a + (b^2 + (2*c - 3*d)*b + (c^2 - 3*d*c + 6*d^2)))*x^2 + (a^3 + (-2*b + (3*c - 2*d))*a^2 + (-2*b^2 + (c + 6*d)*b + (-2*c^2 - 4*d*c + 3*d^2))*a + (b^3 + (-2*c - 2*d)*b^2 + (3*c^2 - 4*d*c + 3*d^2)*b + (c^3 - 2*d*c^2 + 3*d^2*c - 4*d^3)))*x + (a^4 + (-b + (-c - d))*a^3 + (b^2 + (2*c + 2*d)*b + (c^2 - 3*d*c + d^2))*a^2 + (-b^3 + (-3*c + 2*d)*b^2 + (2*c^2 - d*c - 3*d^2)*b + (-c^3 + 2*d*c^2 + 2*d^2*c - d^3))*a + (b^4 + (-c - d)*b^3 + (c^2 + 2*d*c + d^2)*b^2 + (-c^3 - 3*d*c^2 + 2*d^2*c - d^3)*b + (c^4 - d*c^3 + d^2*c^2 - d^3*c + d^4)))
The first command came out successfully, while the second one below did finish successfully and gave an allocatemem() error. How is it possible to get the second command to work, without overheating the computer or program exhaustion? And the WHOLE output to command below this is needed. Thanks for the help.
(19:23) gp > elt=Mod(a*x^5+b*x^4+c*x^3+d*x^2+e*x+f,('x^7-1)/('x-1))
%120 = Mod(a*x^5 + b*x^4 + c*x^3 + d*x^2 + e*x + f, x^6 + x^5 + x^4 + x^3 + x^2 + x + 1)
(19:23) gp > (poly=minpoly(elt,x='x))
*** at top-level: poly=minpoly(elt,x='x)
*** ^-----------------
*** minpoly: the PARI stack overflows !
current stack size: 9000000 (8.583 Mbytes)
[hint] you can increase GP stack with allocatemem()
You can increase the PARI/GP's heap up to any limit you want at run-time following the example below (demonstrates how to set heap size to 120000000 bytes):
default(parisize, 120000000)
default(parisize, 10000000000) is more than 8 GB and in my case was
enough to make advanced calculations with matrices.
The following function produces the nth number in catalan numbers. What is the exact time complexity function of this function or how can I find it myself?
int catalan(int n)
{
if (n==0 || n==1)
return 1;
int sum = 0;
for(int i=1;i<n;i++)
sum += catalan(i)*catalan(n-i);
return sum;
}
Note: I know this is the worst possible way to compute a catalan number.
To evaluate the complexity, let us focus on the number of recursive calls performed, let C(n).
A call for n implies exactly 2(n-1) recursive calls, each of them adding their own costs, 2(C(1)+C(2)+...C(n-1)).
A call for n+1 implies exactly 2n recursive calls, each of them adding their own costs, 2(C(1)+C(2)+...C(n-1)+C(n)).
By difference, C(n+1)-C(n) = 2+2C(n), which can be written C(n) = 2+3C(n-1).
C(1) = 0
C(2) = 2+2C(1) = 2+3C(0) = 2
C(3) = 4+2(C(1)+C(2)) = 2+3C(2) = 8
C(3) = 6+2(C(1)+C(2)+C(3)) = 2+3C(3) = 26
C(4) = 8+2(C(1)+C(2)+C(3)+C(4)) = 2+3C(4) = 80
...
C(n) = 2n-2+2(C(1)+C(2)+...C(n-1)) = 2+3C(n-1)
To solve this recurrence easily, notice that
C(n)+1 = 3(C(n-1)+1) = 9(C(n-2)+1) = ...3^(n-2)(C(2)+1) = 3^(n-1)
Hence, for n>1 the exact formula is
C(n) = 3^(n-1)-1
The number of calls to Catalan(1) (constant time), is also C(n), and the numbers of adds or multiplies are C(n)/2 each.
It is easy to reduce the complexity from O(3^n) to O(2^n) by noting that all terms in the loop (except the middle one) are computed twice - but that still doesn't make it an acceptable implementation :)
Assume
any step other than for-loop is k;
summation and multiple in for-loop is c and
catalan(r) is T(r)
In the for-loop of catalan(n), catalan(i) performs n-1 times where value of i from 1 to n-1 and catalan(n-i) performs n-1 times where value of n-i from n-1 to 1. In short, catalan(i) and catalan(n-i) equals to two times all catalan(x) where value of x from 1 to n-1.
T(n) = 2(T(1) + T(2) + T(3) + ... + T(n-2) + T(n-1)) + k + (n-1)c
Similarly,
T(n-1) = 2(T(1) + T(2) + T(3) + ... + T(n-2)) + k + (n-2)c
Reorder T(n) as 2(T(1) + T(2) + T(3) + ... + T(n-2)) + 2T(n-1) + k + (n-2)c + c
T(n) = 2(T(1) + T(2) + T(3) + ... + T(n-2)) + k + (n-2)c + 2T(n-1) + c
T(n) = T(n-1) + 2T(n-1) + c
T(n) = 3T(n-1) + c
T(n) = (3^2)T(n-2) + 3c + c
T(n) = (3^3)T(n-3) + (3^2)c + 3c + c
and so on...
T(n) = (3^(n-1))T(n-(n-1)) + c(3^0 + 3^1 + 3^2 + ... + 3^(n-2))
T(n) = (3^(n-1))T(1) + ((3^(n-1)-1)/2)c
So, the time complexity is O(3 ^ N)
My process is quite similar to #hk6279's one but I believe easier to understand because I start from the code itself. I start defining the recurrence relation as it is in code and then substitute it.
I also remove all the + k + c variables from #hk6279's approach because it adds noise to the equation and at the end all those variables will be ruled out.
Recurrence relation
T(n) => 1 -> n = 1
T(i) * T(n-i) -> n > 1; for i in 1..n-1
Visualize when n > 1
T(n) = [T(1) + T(2) + T(3) + .... + T(n-2) + T(n-1)] + [T(n-1) + T(n-2) + .... + T(3) + T(2) + T(1)]
T(n) = [T(1) + T(2) + T(3) + .... + T(n-2) + T(n-1)] + [T(1) + T(2) + T(3) + .... + T(n-2) + T(n-1)]
T(n) = 2 * [T(1) + T(2) + T(3) + .... + T(n-2) + T(n-1)]
What is T(n-1) ?
T(n-1) = 2 * [T(1) + T(2) + T(3) + .... + T(n-2)]
Replace with T(n-1)
T(n) = 2 * [T(1) + T(2) + T(3) + .... + T(n-2) + T(n-1)]
T(n) = 2 * [T(1) + T(2) + T(3) + .... + T(n-2)] + 2 * [T(n-1)]
T(n) = T(n-1) + 2 * [T(n-1)]
T(n) = 3 * T(n-1)
What is T(n-2) ?
T(n-2) = 2 * [T(1) + T(2) + T(3) + .... + T(n-3)]
Replace with T(n-2)
T(n) = 3 * [2 * [T(1) + T(2) + T(3) + .... + T(n-3) + T(n-2)]]
T(n) = 3 * [2 * [T(1) + T(2) + T(3) + .... + T(n-3)] + 2 * T(n-2)]]
T(n) = 3 * [T(n-2) + 2*T(n-2)]
T(n) = 3 * [3 * T(n-2)]
T(n) = 3^2 * T(n-2)
Replace with T(n-k)
T(n) = 3^k * T(n-k)
if n - k = 1 => k = n + 1
T(n) = 3^(n+1) * T(n-n+1)
T(n) = 3^(n+1) * T(1)
T(n) = 3^(n+1) * 1
Time complexiTy
O(3^n)
First of all I have to say that I can use recursive functions on easy examples like Fibonacci, but I can't understand how to dry run (solve with pen and paper) this recursion :
#include<iostream>
using namespace std;
int max(int a, int b)
{
if(a>b)return a;
return b;
}
int f(int a, int b)
{
if(a==0)return b;
return max( f(a-1,2*b), f(a-1,2*b+1) );
}
int main()
{
cout<<f(8,0);
}
How do I do this with pen and paper, with say, a = 5 and b = 6?
We have always a depth of a (8)
Each invocations calls itself 2 times, once 2b and once 2b+1 is passed
The greater result of both calls is returned
As 2b + 1 > 2b only the right site of the max call is meaningful (2b + 1)
Now lets do the first iterations mathematically:
2 * b + 1 = 2^1 * b + 2^0
2 * (2^1 * b + 2^0) + 1 = 2^2 * b + 2^1 + 2^0
2 * (2^2 * b + 2^1 + 2^0) + 1 = 2^3 * b + 2^2 + 2^1 + 2^0
2 * (2^3 * b + 2^2 + 2^1 + 2^0) + 1 = 2^4 * b + 2^3 + 2^2 + 2^1 + 2^0
As you can see there is a system behind it. Because b = 0 for the first iteration, we can ignore the left side. The final value is thus:
2^0 + 2^1 + 2^2 + 2^3 + 2^4 + 2^5 + 2^6 + 2^7
=
1 + 2 + 4 + 8 + 16 + 32 + 64 + 128
=
255
If we run the programm we get the exact same value
Just to give some information there are algorithms that use a little more complex parameters, one basic example would be mergesort
Merging is simple:
Take two elements one from each array A and B.
Compare them and place smaller of two (say from A) in sorted list.
Take next element from A and compare with element in hand (from B).
Repeat until one of the array is exhausted.
Now place all remaining elements of non-empty array one by one.
Maybe you can find this doc useful
Or maybe this one
Assuming you want to analyzse the funciton on paper, I'll paste the result for f(1, 2)
f(2, 1) =
max( f(1, 2), f(1, 3) ) =
max ( max(f(0, 4), f(0, 5) , max(f(0, 6), f(0, 7) ) =
max ( max(4, 5) , max(6, 7) ) =
max (5, 7) =
7
Is up to you to follow the computations
Note: I'm also assuming you didn't miss a parenthesis here: 2*b+1
I have a strange algorithm than is being called recursively 2 times. It's
int alg(int n)
loop body = Θ(3n+1)
alg(n-1);
alg(n-2)
Somehow i need to find this algorithm's complexity. I've tried to find it with using characteristic polynomial of the above equation but the result system is too hard to solve so i was wondering if there was any other straight way..
Complexity: alg(n) = Θ(φ^n) where φ = Golden ratio = (1 + sqrt(5)) / 2
I can't formally prove it at first, but with a night's work, I find my missing part - The substitution method with subtracting a lower-order term. Sorry for my bad expression of provement (∵ poor English).
Let loop body = Θ(3n+1) ≦ tn
Assume (guess) that cφ^n ≦ alg(n) ≦ dφ^n - 2tn for an n (n ≧ 4)
Consider alg(n+1):
Θ(n) + alg(n) + alg(n-1) ≦ alg(n+1) ≦ Θ(n) + alg(n) + alg(n-1)
c * φ^n + c * φ^(n-1) ≦ alg(n+1) ≦ tn + dφ^n - 2tn + dφ^(n-1) - 2t(n-1)
c * φ^(n+1) ≦ alg(n+1) ≦ tn + d * φ^(n+1) - 4tn + 2
c * φ^(n+1) ≦ alg(n+1) ≦ d * φ^(n+1) - 3tn + 2
c * φ^(n+1) ≦ alg(n+1) ≦ d * φ^(n+1) - 2t(n+1) (∵ n ≧ 4)
So it is correct for n + 1. By mathematical induction, we can know that it's correct for all n.
So cφ^n ≦ alg(n) ≦ dφ^n - 2tn and then alg(n) = Θ(φ^n).
johnchen902 is correct:
alg(n)=Θ(φ^n) where φ = Golden ratio = (1 + sqrt(5)) / 2
but his argument is a bit too hand-waving, so let's make it strict. His original argument was incomplete, therefore I added mine, but now he has completed the argument.
loop body = Θ(3n+1)
Let us denote the cost of the loop body for the argument n with g(n). Then g(n) ∈ Θ(n) since Θ(n) = Θ(3n+1).
Further, let T(n) be the total cost of alg(n) for n >= 0. Then, for n >= 2 we have the recurrence
T(n) = T(n-1) + T(n-2) + g(n)
For n >= 3, we can insert the recurrence applied to T(n-1) into that,
T(n) = 2*T(n-2) + T(n-3) + g(n) + g(n-1)
and for n > 3, we can continue, applying the recurrence to T(n-2). For sufficiently large n, we therefore have
T(n) = 3*T(n-3) + 2*T(n-4) + g(n) + g(n-1) + 2*g(n-2)
= 5*T(n-4) + 3*T(n-5) + g(n) + g(n-1) + 2*g(n-2) + 3*g(n-3)
...
k-1
= F(k)*T(n+1-k) + F(k-1)*T(n-k) + ∑ F(j)*g(n+1-j)
j=1
n-1
= F(n)*T(1) + F(n-1)*T(0) + ∑ F(j)*g(n+1-j)
j=1
with the Fibonacci numbers F(n) [F(0) = 0, F(1) = F(2) = 1].
T(0) and T(1) are some constants, so the first part is obviously Θ(F(n)). It remains to investigate the sum.
Since g(n) ∈ Θ(n), we only need to investigate
n-1
A(n) = ∑ F(j)*(n+1-j)
j=1
Now,
n-1
A(n+1) - A(n) = ∑ F(j) + (((n+1)+1) - ((n+1)-1))*F((n+1)-1)
j=1
n-1
= ∑ F(j) + 2*F(n)
j=1
= F(n+1) - 1 + 2*F(n)
= F(n+2) + F(n) - 1
Summing that, starting with A(2) = 2 = F(5) + F(3) - 5, we obtain
A(n) = F(n+3) + F(n+1) - (n+3)
and therefore, with
c*n <= g(n) <= d*n
the estimate
F(n)*T(1) + F(n-1)*T(0) + c*A(n) <= T(n) <= F(n)*T(1) + F(n-1)*T(0) + d*A(n)
for n >= 2. Since F(n+1) <= A(n) < F(n+4), all terms depending on n in the left and right parts of the inequality are Θ(φ^n), q.e.d.
Assumptions:
1: n >= 0
2: Θ(3n+1) = 3n + 1
Complexity:
O(2 ^ n * (3n - 2));
Reasoning:
int alg(int n)
loop body = Θ(3n+1)// for every n you have O(3n+1)
alg(n-1);
alg(n-2)
Assuming the alg does not execute for n < 1, you have the following repetitions:
Step n:
3 * n + 1
alg(n - 1) => 3 * (n - 1) + 1
alg(n - 2) => 3 * (n - 2) + 1
Now you basically have a division. You have to imagine a number tree with N as parent and n-1 and n-2 as children.
n
n-1 n-2
n - 2 n - 3 n - 3 n - 4
n - 3 n - 4 n - 4 n - 5 n - 4 n - 5 n - 5 n - 6
n-4 n-5 | n-5 n-6 |n-5 n-6 |n-6 n-7 n-5 n-6 n-6 n-7 n-6 n-6| n-6 n-8
It's obvious that there is a repetition pattern here. For every pair (n - k, n - k - 1) in A = {k, with k from 0 to n) except the first two and the last two, (n - 1, n - 2) and (n-2, n-3) there is a 3k + 1 * (2 ^ (k - 1)) complexity.
I am looking at the number of repetitions of the pair (n - k, n - k - 1). So now for each k from 0 to n I have:
(3k + 1) * (2 ^ (k - 1)) iterations.
If you sum this up from 1 to n you should get the desired result. I will expand the expression:
(3k + 1) * (2 ^ (k - 1)) = 3k * 2 ^ (k - 1) + 2 ^ (k - 1)
Update
1 + 2 + 2^2 + 2^3 + ... + 2^n = 2 ^ (n + 1) - 1
In your case, this winds up being:
2^n - 1
Based on the summation formula and k = 0, n . Now the first one:
3k * 2 ^ (k - 1)
This is equal to 3 sum from k = 0, n of k * 2 ^ (k - 1).
That sum can be determined by switching to polinomial functions, integrating, contracting using the 1 + a ^ 2 + a ^ 3 + ... + a ^ n formula, and then differentiated again to obtain the result, which is (n - 1) * 2 ^ n + 1.
So you have:
2 ^ n - 1 + 3 * (n - 1) * 2 ^ n + 1
Which contracted is:
2 ^ n * (3n - 2);
The body of the function takes Θ(n) time.
The function is called twice recursively.
For the given function the complexity is,
T(n) = T(n-1) + T(n-2) + cn ----- 1
T(n-1) = T(n-2) + T(n-3) + c(n-1) ----- 2
1-2 -> T(n) = 2T(n-1) - T(n-3) + c ----- 3
3 --> T(n-1) = 2T(n-2) + T(n-4) + c ----- 4
3-4 -> T(n) = 3T(n-1) - 2T(n-2) - T(n-3) - T(n-4) ----- 5
Let g(n) = 3g(n-1)
There for, we can approximate T(n) = O(g(n))
g(n) is Θ(3n)
There for T(n) = O(3n)