This question already has answers here:
Big O, how do you calculate/approximate it?
(24 answers)
Closed 8 years ago.
Compute the complexity of the following Algorithm?
I have the following code snippet:
i = 1;
while (i < n + 1) {
j = 1;
while (j < n + 1) {
j = j * 2;
}
i = i + 1;
}
plz explain it in detail
I want to know the the steps to solve the problem so I can solve such problems
Since j grows exponentially, the inner loop takes O(log(n)).
Since i grows linearly, the outer loop takes O(n).
Hence the overall complexity is O(n*log(n)).
i = 1;
while(i < n + 1){
j = 1;
While(j < n + 1){
j = j * 2:
}
i = i + 1;
}
outer loop takes O(n) since it increments by constant.
i = 1;
while(i < n + 1){
i = i + 1;
}
inner loop : j = 1, 2, 4, 8, 16, ...., 2^k
j = 2^k (k >= 0)
when will j stops ?
when j == n,
log(2^k) = log(n)
=> k * lg(2) = lg(n) ..... so k = lg(n).
While(j < n + 1){
j = j * 2;
}
so total O(n * lg(n))
You can simply understand outer-loop(with i) because it loops exactly n times. (1, 2, 3, ..., n). But inner-loop(j) is little difficult to understand.
Let's assume that n is 8. How much it loops? Starting with j = 1, it will be increased as exponentially : 1, 2, 4, 8. When j is over 8, loop will be terminated. It loops exactly 4 times. Then we can think general-form of this problem...
Think of that sequence 1, 2, 4, 8, .... If n is 2^k (k is non-negative integer), inner-loop will take k+1 times. (Because 2^(loop-1) = 2^k) Due to the assumption : n = 2^k, we can say that k = lg(n). So we can say inner-loop takes lg(n)+1 times.
When n is not exactly fit to 2^k, it takes one more time. ([lg(n)]+1) It's not a big deal with complexity though it has floor function. You can ingonre it this time.
So the total costs will be like this : n*(lg(n)+1). If you are familiar with Big-O notation, it can be expressed as : O(n lg n).
This one is similar to the following code :
for( int i = 1;i < n+1 ; i++){ // this loop runs n times
for(int j = 1 ; j<n+1 ; j=j*2){// this loop runs log_2(n)(log base 2 because it grows exponentially with 2)
//body
}
}
Hence in Big-Oh notation it is O(n)*O(logn) ; i.e, O(n*logn)
You can proceed like the following:
Related
The first loop runs O(log n) time but the second loop's runtime depends on the counter of the first loop, If we examine it more it should run like (1+2+4+8+16....+N) I just couldn't find a reasonable answer to this series...
for (int i = 1; i < n; i = i * 2)
{
for (int j = 1; j < i; j++)
{
//const time
}
}
Just like you said. If N is power of two, then 1+2+4+8+16....+N is exactly 2*N-1 (sum of geometric series) . This is same as O(N) that can be simplified to N.
It is like :
1 + 2 + 4 + 8 + 16 + ....+ N
= 2 ^ [O(log(N) + 1] - 1
= O(N)
Consider this function:
void func()
{
int n;
std::cin >> n;
int var = 0;
for (int i = n; i > 0; i--)
for (int j = 1; j < n; j *= 2)
for (int k = 0; k < j; k++)
var++;
}
I think that the time complexity is O(n^2 * log n)
but when n is 2^m, I have a hard time thinking what complexity it is.
How can I analyze the complexity of this function?
n isn't a constant, it's dynamic, so that's the variable in your analysis. If it was constant, your complexity would be O(1) regardless of its value because all constants are discarded in complexity analysis.
Similarly, "n is 2^m" is sort of nonsensical because m isn't a variable in the code, so I'm not sure how to analyze that. Complexity analysis is done relative to the size of the input; you don't have to introduce any more variables.
Let's break down the loops, then multiply them together:
for (int i = n; i > 0; i--) // O(n)
for (int j = 1; j < n; j *= 2) // O(log(n))
for (int k = 0; k < j; k++) // O(n / log(n))
Total time complexity: O(n * log(n) * (n / log(n))) => O(n^2).
The first two loops are trivial (if the second one isn't obvious, it's logarithmic because of repeated multiplication by 2, the sequence is 1, 2, 4, 8, 16...).
The third loop is tougher to analyze because it runs on j, not n. We can simplify matters by disregarding the outermost loop completely, analyzing the inner loops, then multiplying whatever complexity we get for the two inner loops by the outermost loop's O(n).
The trick is to look at the shape of the enclosing loop; as the j loop approaches n, k is running from 0..n linearly, giving a baseline of O(n) for the k loop. This is scaled by a logarithmic factor j, O(log(n)). The logarithmic factors cancel and we're left with O(n) for the inner loops.
Here's some empirical evidence of the inner loop complexity:
import math
from matplotlib import pyplot
def f(N):
count = 0
j = 1
while j < N:
j *= 2
count += j
return count
def linear(N):
return N
def linearithmic(N):
return N * math.log2(N) if N else 0
def plot_fn(fn, n_start, n_end, n_step):
x = []
y = []
for N in range(n_start, n_end, n_step):
x.append(N)
y.append(fn(N))
pyplot.plot(x, y, "-", label=fn.__name__)
def main():
max_n = 10 ** 10
step = 10 ** 5
for fn in [f, linear, linearithmic]:
plot_fn(fn, 0, max_n, step)
pyplot.legend()
pyplot.show()
if __name__ == "__main__":
main()
The plot this produces is:
This shows that the innermost two loops (in blue) are linear, not linearithmic, confirming the overall quadratic complexity once the outermost linear loop is re-introduced.
So, I need to find the T(n) and then Big-O (tight upper bound) for the following piece of code:
int sum = 0;
for(int i = 1; i < n; i *= 2) {
for(int j = n; j > 0; j /= 2) {
for(int k = j; k < n; k += 2) {
sum += i + j * k;
}
}
}
Now from what I calculated for the loops, first loop runs log(n) times, second loop runs (log(n) * log(n)) times and the third loop is the one which is causing confusion, because I believe it runs for (n - j)/2 times. My question is can I assume it to be n/2 times, because I think it won't be a tight upper bound if I do that. Or is there a different approach that I am missing?
for(int i = 1; i < n; i *= 2) // (1)
for(int j = n; j > 0; j /= 2) // (2)
for(int k = j; k < n; k += 2) // (3)
For the first iteration of (3) (where k = j = n) no iteration will occur. After j is divided by 2 the third loop will run (n/2)/2 or n/4 times. After the third iteration of (2), (3) will run n/4/2 or n/8 times. We can sum the running time as follows:
n/4 + n/8 + n/16 + ... + n/2^k
This can also be written as:
n * (1/4 + 1/8 + 1/16 + ... + 1/2^k)
Which asymptotically is in O(n).
This is a very interesting question. Let give n a real number and see how it's going. Say, n=100. If we only look at the two inner loops
j k
100 None
50 50, 52, ..., 98
25 25, 27, ..., 99
12 12, 14, ..., 98
6 6, 8, ..., 98
3 3, 5, ..., 99
1 1, 3, ..., 99
As you can see, the complexity of the third loop is actually O(n). Especially when n is a very large number, it will be close to Θ(n)
Okay so when my professor was going over it in class it seemed quite simple, but when I got to my homework I became confused. This is a homework example.
for (int i = 0; i < n; i++) // I know this runs at T(n)
for (int j = n - 1; j >= i; j--)
cout << i << " " << j << endl;
Here's an example I understand
for(int i=0; i<n-1; i++) {
for(int j=i+1; j<n; j++) {
1 Simple statement
}
For that example I just plugged in 0, 1, and 2. For 0, it ran for n-1, at 1 for n-2 and at 2 n-3. So I think that for the homework example if I plugged in 0 it would run for n+1 since j has to be greater than or equal to i which is 0. If it's not obvious, i'm pretty confused. If anyone could show me how to solve it, that'd make my day. Thanks guys.
Let's dig into the functon. Let's pick some numbers.
say, n = 5
So our code looks like this (magical pseudo-code uses INCLUSIVE loops, not that it's too important)
(1)for i = 0 to 4
(2)for j = 4 to i
(3)print i j
next
next
So this is a matter of preference, but usually loops are assumed to cost 1 simple statement per execution (comparison, and incrementation). So we'll assume that statements (1) and (2) have a cost of 2. Statement (3) has a cost of 1.
Now to determine T(n).
Our outer loop for i = 0 to 4 runs exactly n times.
Our inner loop for j = 4 to i . . . We'll dig in there for a minute.
For our example with n = 5 loop (2) will execute like so
j = 4; i = 0; j = 4; i = 1; j = 4; i = 2; j = 4; i = 3 j = 4; i = 4;
j = 3; i = 0; j = 3; i = 1; j = 3; i = 2; j = 3; i = 3;
j = 2; i = 0; j = 2; i = 1; j = 2; i = 2;
j = 1; i = 0; j = 1; i = 1;
j = 0; i = 0;
So it makes this kind of pyramid shape, where we do 1 less iteration each time. This particular example ran 5 + 4 + 3 + 2 + 1 = 15 times.
We can write this down as SUM(i; i = 0 to n).
Which we know from precalc: = (1/2)(n)(n+1).
And (3) will execute the exact same number of times as that inner loop since it's the only statement. So our total runtime is going to be. . .
COST(1) + COST(2) + COST(3)
(2)(n) + 2(1/2)(n)(n+1) + (1/2)(n)(n+1)
We can clean this up to be
(3/2)(n)(n+1) + 2n = T(n).
That said, this assumes that loops cost 2 and the statement costs 1. It's usually more meaningful to say loops cost 0 and statements cost 1. If that were the case, T(n) = (1/2)(n)(n+1).
And givent that T(n), we know T(n) is O(n^2).
Hope this helps!
It's not that hard.
3 examples for single loops:
for (int i = 0; i < n; i++)
for(int i = 0; i < n-1; i++)
for(int i = 2; i < n-1; i++)
The first loop executs it's content n times (i=0,1,2,3,...,n-1).
The same way, the second loop is just n-1 times.
The third would be n-3 because it starts not at 0, but 2
(and if n is less than 3, ie. n-3<0, it won't execute at all)
In a nested loop like
for(int i = 0; i < n-1; i++) {
for(int j = 0; j < n; j++) {
//something
}
}
For each pass of the outer loop, the whole inner loop is executed, ie. you can multiply both single loop counts to get how often "something" is executed in total. Here, it is (n-1) * n = n^2 - n.
If the inner loop depends on the value of the outer loop, it gets a bit more complicated:
for(int i = 0; i < n-1; i++) {
for(int j = i+1; j < n; j++) {
//something
}
}
The inner loop alone is n - (i+1) times, the outer one n-1 times (with i going from 0 to n-2).
While there are "proper" ways to calculate this, a bit logical thinking is often easier, as you did already:
i-value => inner-loop-time
0 => n-1
1 => n-2
...
n-2 => n - (n-2+1) = 1
So you´ll need the sum 1+2+3+...+(n-1).
For calculating sums from 1 to x, following formula helps:
sum[1...x] = x*(x+1)/2
So, the sum from 1 to n-1 is
sum[1...n-1] = (n-1)*(n-1+1)/2 = (n^2 - n)/2
and that´s the solution for the loops above (your second code).
About the first code:
Outer loop: n
Inner loop: From n-1 down to i included, or the other way from i up to <=n-1,
or from i up to <n, that´s n-i times
i >= innerloop
0 n
1 n-1
2 n-2
...
n-1 1
...and the sum from 1 to n is (n^2 + n)/2.
One easy way to investigate a problem is to model it and look at resulting data.
In your case, the question is: how many iterations does the inner loop depending on the the value of the outer loop variable?
let n = 10 in [0..n-1] |> List.map (fun x -> x,n-1-x);;
The 1 line above is the model showing what happens. If you now look at the resulting output, you will quickly notice something...
val it : (int * int) list =
[(0, 9); (1, 8); (2, 7); (3, 6); (4, 5); (5, 4); (6, 3); (7, 2); (8, 1);
(9, 0)]
What is it you notice? For a given N you run the outer loop N times - this is trivial. Now we need to sum up the second numbers and we have the solution:
sum(N-1..0) = sum(N-1..1) = N * (N-1) / 2.
So the total count of cout calls is N * (N-1) / 2.
Another easy way to achieve the same is to modify your function a bit:
int count(int n) {
int c = 0;
<outer for loop>
<inner for loop>
c++;
return c;
}
1) for (i = 1; i < n; i++) { > n
2) SmallPos = i; > n-1
3) Smallest = Array[SmallPos]; > n-1
4) for (j = i+1; j <= n; j++) > n*(n+1 -i-1)??
5) if (Array[j] < Smallest) { > n*(n+1 -i-1 +1) ??
6) SmallPos = j; > n*(n+1 -i-1 +1) ??
7) Smallest = Array[SmallPos] > n*(n+1 -i-1 +1) ??
}
8) Array[SmallPos] = Array[i]; > n-1
9) Array[i] = Smallest; > n-1
}
i know the big O notation is n^2 ( my bad its not n^3)
i am not sure between line 4-7 anyone care to help out?
im not sure how to get the out put for the second loop since j = i +1 as i changes so does j
also for line 4 the ans suppose to be n(n+1)/2 -1 i want to know why as i can never get that
i am not really solving for the big O i am trying to do the steps that gets to big O as constant and variables are excuded in big O notations.
I would say this is O(n^2) (although as Fred points out above, O(n^2) is a subset of O(n^3), so it's not wrong to say that it's O(n^3)).
Note that it's generally not necessary to compute the number of executions of every single line; as Big-O notation discards low-order terms, it's sufficient to focus only on the most-executed section (which will typically be inside the innermost loop).
So in your case, none of the loops are affected by the values in Array, so we can safely ignore all that. The innermost loop runs (n-1) + (n-2) + (n-3) + ... times; this is an arithmetic series, and so has a term in n^2.
Is this an algorithm given to you, or one you wrote?
I think your loop indexes are wrong.
for (i = 1; i < n; i++) {
should be either
for (i = 0; i < n; i++) {
or
for (i = 1; i <= n; i++) {
depending on whether your array indexes start at 0 or 1 (it's 0 in C and Java).
Assuming we correct it to:
for (i = 0; i < n; i++) {
SmallPos = i;
Smallest = Array[SmallPos];
for (j = i+1; j < n; j++)
if (Array[j] < Smallest) {
SmallPos = j;
Smallest = Array[SmallPos];
}
Array[SmallPos] = Array[i];
Array[i] = Smallest;
}
Then I think the complexity is n2-3/2n = O(n2).
Here's how...
The most costly operation in the innermost loop (my lecturer called this the "basic operation") is key comparison at line 5. It is done once per loop.
So now, you create a summation:
Sum(i=0 to n-1) of Sum(j=i+1 to n-1) of 1.
Now expand the innermost (rightmost) Sum to get:
Sum(i=0 to n-1) of (n-1)-(i+1)+1
and then:
Sum(i=0 to n-1) of n-i-1
and then:
[Sum(i=0 to n-1) of n] - [Sum(i=0 to n-1) of i] - [Sum (i=0 to n-1) of 1]
and then:
n[Sum(i=0 to n-1) of 1] - [(n-1)(n)/2] - [(n-1)-0+1]
and then:
n[(n-1)-0+1] - [(n^2-n)/2] - [n]
and then:
n^2 - [(n^2/2) - n/2] - n
equals:
1/2n^2 - 1/2n
is in:
O(n^2)
If you're asking why it's not O(n3)...
Consider the worst case. if (Array[j] < Smallest) will be true the most times if Array is reverse sorted.
Then you have an inner loop that looks like this:
Array[j] < Smallest;
SmallPos = j;
Smallest = Array[SmallPos];
Now we've got a constant three operations for every inner for (j...) loop.
And O(3) = O(1).
So really, it's i and j that determine how much work we do. Nothing in the inner if loop changes anything.
You can think of it as you should only count while and for loops.
As to why for (j = i+1; j <= n; j++) is n(n+1)/2. It's called an arithmetic series.
You're doing n-1 passes of the for (j...) loop when i==0, n-2 passes when i==1, n-3, etc, until 0.
So the summation is
n-1 + n-2 + n-3 + ... 3 + 2 + 1
now, you sum pairs from outside in, re-writing it as:
n-1+1 + n-2+2 + n-3+3 + ...
equals:
n + n + n + ...
and there are n/2 of these pairs, so you have:
n*(n/2)
Two for() loops, the outer loop from 1 to n, the inner loop runs between 1..n, to n. This makes it O(n^2).
If you 'draw this out', it'll be triangular, rather than rectangular, so O(n^2), while true, is hiding the fact that the constant factor term is smaller than if the inner loop also iterated from 1 to n.
It is O(n^2).
For each of the n iterations of the outer loop you have n iterations in the inner loop.