1) for (i = 1; i < n; i++) { > n
2) SmallPos = i; > n-1
3) Smallest = Array[SmallPos]; > n-1
4) for (j = i+1; j <= n; j++) > n*(n+1 -i-1)??
5) if (Array[j] < Smallest) { > n*(n+1 -i-1 +1) ??
6) SmallPos = j; > n*(n+1 -i-1 +1) ??
7) Smallest = Array[SmallPos] > n*(n+1 -i-1 +1) ??
}
8) Array[SmallPos] = Array[i]; > n-1
9) Array[i] = Smallest; > n-1
}
i know the big O notation is n^2 ( my bad its not n^3)
i am not sure between line 4-7 anyone care to help out?
im not sure how to get the out put for the second loop since j = i +1 as i changes so does j
also for line 4 the ans suppose to be n(n+1)/2 -1 i want to know why as i can never get that
i am not really solving for the big O i am trying to do the steps that gets to big O as constant and variables are excuded in big O notations.
I would say this is O(n^2) (although as Fred points out above, O(n^2) is a subset of O(n^3), so it's not wrong to say that it's O(n^3)).
Note that it's generally not necessary to compute the number of executions of every single line; as Big-O notation discards low-order terms, it's sufficient to focus only on the most-executed section (which will typically be inside the innermost loop).
So in your case, none of the loops are affected by the values in Array, so we can safely ignore all that. The innermost loop runs (n-1) + (n-2) + (n-3) + ... times; this is an arithmetic series, and so has a term in n^2.
Is this an algorithm given to you, or one you wrote?
I think your loop indexes are wrong.
for (i = 1; i < n; i++) {
should be either
for (i = 0; i < n; i++) {
or
for (i = 1; i <= n; i++) {
depending on whether your array indexes start at 0 or 1 (it's 0 in C and Java).
Assuming we correct it to:
for (i = 0; i < n; i++) {
SmallPos = i;
Smallest = Array[SmallPos];
for (j = i+1; j < n; j++)
if (Array[j] < Smallest) {
SmallPos = j;
Smallest = Array[SmallPos];
}
Array[SmallPos] = Array[i];
Array[i] = Smallest;
}
Then I think the complexity is n2-3/2n = O(n2).
Here's how...
The most costly operation in the innermost loop (my lecturer called this the "basic operation") is key comparison at line 5. It is done once per loop.
So now, you create a summation:
Sum(i=0 to n-1) of Sum(j=i+1 to n-1) of 1.
Now expand the innermost (rightmost) Sum to get:
Sum(i=0 to n-1) of (n-1)-(i+1)+1
and then:
Sum(i=0 to n-1) of n-i-1
and then:
[Sum(i=0 to n-1) of n] - [Sum(i=0 to n-1) of i] - [Sum (i=0 to n-1) of 1]
and then:
n[Sum(i=0 to n-1) of 1] - [(n-1)(n)/2] - [(n-1)-0+1]
and then:
n[(n-1)-0+1] - [(n^2-n)/2] - [n]
and then:
n^2 - [(n^2/2) - n/2] - n
equals:
1/2n^2 - 1/2n
is in:
O(n^2)
If you're asking why it's not O(n3)...
Consider the worst case. if (Array[j] < Smallest) will be true the most times if Array is reverse sorted.
Then you have an inner loop that looks like this:
Array[j] < Smallest;
SmallPos = j;
Smallest = Array[SmallPos];
Now we've got a constant three operations for every inner for (j...) loop.
And O(3) = O(1).
So really, it's i and j that determine how much work we do. Nothing in the inner if loop changes anything.
You can think of it as you should only count while and for loops.
As to why for (j = i+1; j <= n; j++) is n(n+1)/2. It's called an arithmetic series.
You're doing n-1 passes of the for (j...) loop when i==0, n-2 passes when i==1, n-3, etc, until 0.
So the summation is
n-1 + n-2 + n-3 + ... 3 + 2 + 1
now, you sum pairs from outside in, re-writing it as:
n-1+1 + n-2+2 + n-3+3 + ...
equals:
n + n + n + ...
and there are n/2 of these pairs, so you have:
n*(n/2)
Two for() loops, the outer loop from 1 to n, the inner loop runs between 1..n, to n. This makes it O(n^2).
If you 'draw this out', it'll be triangular, rather than rectangular, so O(n^2), while true, is hiding the fact that the constant factor term is smaller than if the inner loop also iterated from 1 to n.
It is O(n^2).
For each of the n iterations of the outer loop you have n iterations in the inner loop.
Related
I am having trouble deciding between N^2 and NlogN as the Big O? Whats throwing me off is the third nested for loop from k <=j. How do I reconcile this?
int Max_Subsequence_Sum( const int A[], const int N )
{
int This_Sum = 0, Max_Sum = 0;
for (int i=0; i<N; i++)
{
for (int j=i; j<N; j++)
{
This_Sum = 0;
for (int k=i; k<=j; k++)
{
This_Sum += A[k];
}
if (This_Sum > Max_Sum)
{
Max_Sum = This_Sum;
}
}
}
return Max_Sum;
}
This can be done with estimation or analysis. Looking at the inner most loop there are j-i operations inside the second loop. To get the total number of operations one would sum to get :
(1+N)(2 N + N^2) / 6
Making the algorithm O(N^3). To estimate one can see that there are three loops which at some point have O(N) calls thus it's O(N^3).
Let us analyze the most inner loop first:
for (int k=i; k <= j; k++) {
This_Sum += A[k];
}
Here the counter k iterates from i (inclusive) to j (inclusive), this thus means that the body of the for loop is performed j-i+1 times. If we assume that fetching the k-th number from an array is done in constant time, and the arithmetic operations (incrementing k, calculating the sum of This_Sum and A[k], and comparking k with j), then this thus runs in O(j-i).
The initialization of This_Sum and the if statement is not significant:
This_Sum = 0;
// ...
if (This_Sum > Max_Sum) {
Max_Sum = This_Sum;
}
indeed, if we can compare two numbers in constant time, and set one variable to the value hold by another value in constant time, then regardless whether the condition holds or not, the number of operations is fixed.
Now we can take a look at the loop in the middle, and abstract away the most inner loop:
for (int j=i; j < N; j++) {
// constant number of oprations
// j-i+1 operations
// constant number of operations
}
Here j ranges from i to N, so that means that the total number of operations is:
N
---
\
/ j - i + 1
---
j=i
This sum is equivalent to:
N
---
\
(N-j) * (1 - i) + / j
---
j=i
This is an arithmetic sum [wiki] and it is equivalent to:
(N - i + 1) × ((1 - i) + (i+N) / 2) = (N - i + 1) × ((N-i) / 2 + 1)
or when we expand this:
i2/2 + 3×N/2 - 3×i/2 + N2/2 - N×i + 1
So that means that we can now focus on the outer loop:
for (int i=0; i<N; i++) {
// i2/2 + 3×N/2 - 3×i/2 + N2/2 - N×i + 1
}
So now we can again calculate the number of operations with:
N
---
\
/ i2/2 + 3×N/2 - 3×i/2 + N2/2 - N×i + 1
---
i=0
We can use Faulhaber's formula [wiki] here to solve this sum, and obtain:
(N+1)×(N2+5×N+6)/6
or in expanded form:
N3/6 + N2 + 11×N/6 + 1
which is thus an O(n3) algorithm.
I want to design an algorithm with O(n(log(2)n)^2) time complexity. I wrote this:
for(int i=1; i<=n; i++){
j=i;
while(j != 1)
j=j/2;
j=i;
while(j !=1)
j=j/2;
}
Does it have O(n(log(2)n)^2) time complexity? If not, where I am going wrong and how can I fix it so that its time complexity is O(n(log(2)n)^2)?
Slight digression:
As the guys said in the comments, the algorithm is indeed O(n log n). This is coincidentally identical to the result obtained by multiplying the complexity of the inner loop by the outer loop, i.e. O(log i) x O(n).
This may lead you to believe that we can simply add another iteration of the inner loop to obtain the (log n)2 part:
for (int i = 1; i < n; i++) {
int k = i;
while (k >= 1)
k /= 2;
int j = i;
while (j >= 1)
j /= 2;
}
}
But let's look at how the original complexity is derived:
(Using Sterling's approximation)
Therefore the proposed modification would give:
Which is not what we want.
An example I can think of from a recent personal project is semi-naive KD-tree construction. The pseudocode is given below:
def kd_recursive_cons (list_points):
if length(list_points) < predefined_threshold:
return new leaf(list_points)
A <- random axis (X, Y, Z)
sort list_points by their A-coordinate
mid <- find middle element in list_points
list_L, list_R <- split list_points at mid
node_L <- kd_recursive_cons(list_L)
node_R <- kd_recursive_cons(list_R)
return new node (node_L, node_R)
end
The time complexity function is therefore given by:
Where the n log n part is from sorting. We can obviously ignore the Dn linear part, and also the constant C. Thus:
Which is what we wanted.
Now to write a simpler piece of code with the same time complexity. We can make use of the summation we obtained in the above derivation...
And noting that the parameter passed to the log function is divided by two in every loop, we can thus write the code:
for (int i = 1; i < n; i++) {
for (int k = n; k >= 1; k /= 2) {
int j = k;
while (j >= 1)
j /= 2;
}
}
This looks like the "naive" but incorrect solution mentioned at the beginning, with the difference being that the nested loops there had different bounds (j did not depend on k, but k depended on i instead of directly on n).
EDIT: some numerical tests to confirm that the complexity is as intended:
Test function code:
int T(int n) {
int o = 0;
for (int i = 1; i < n; i++)
for (int j = n; j >= 1; j /= 2)
for (int k = j; k >= 1; k /= 2, o++);
return o;
}
Numerical results:
n T(n)
-------------------
2 3
4 18
8 70
16 225
32 651
64 1764
128 4572
256 11475
512 28105
1024 67518
2048 159666
4096 372645
8192 860055
16384 1965960
32768 4456312
65536 10026855
131072 22413141
262144 49807170
524288 110100270
Then I plotted sqrt(T(n) / n) against n. If the complexity is correct this should give a log(n) graph, or a straight line if plotted with a log-scale horizontal axis.
And this is indeed what we get:
for (int i = 0; i < n; ++i ) { //n
for (int j = 0; j < i; ++j) { //n
cout<< i* j<<endl;
cout<< ("j = " + j);
}
for (int k = 0; k < n * 3; ++k) //n?
cout<<"k = " + k);
}
In this loop I see that the first for loop is O(n), the second loop is also O(n) but the 3rd for loop is confusing for me. K being less than something expanding would this also be O(n) for this loop? If so, what does two loops within another loop's time complexity come out to be in this context?
I am assuming O(n^2) due to the two n's in the middle not being multiplied in any way. Is this correct? Also if I'm correct and the second loop is O(n), what would the time complexity be if it was O(logn)?
(Not homework, simply for understanding purposes)
A good rule of thumb for big-O notation is the following:
When in doubt, work inside-out!
Here, let's start by analyzing the two inner loops and then work outward to get the overall time complexity. The two inner loops are shown here:
for (int j = 0; j < i; ++j) {
cout<< i* j<<endl;
cout<< (”j = ” + j);
}
for (int k = 0; k < n * 3; ++k)
cout<<”k = ” + k);
The first loop runs O(i) times and does O(1) work per iteration, so it does O(i) total work. That second loop runs O(n) times (it runs 3n times, and since big-O notation munches up constants, that's O(n) total times) and does O(1) work per iteration, so it does O(n) total work. This means that your overall loop can be rewritten as
for (int i = 0; i < n; ++i) {
do O(i) work;
do O(n) work;
}
If you do O(i) work and then do O(n) work, the total work done is O(i + n), so we can rewrite this even further as
for (int i = 0; i < n; ++i) {
do O(i + n) work;
}
If we look at the loop bounds here, we can see that i ranges from 0 up to n-1, so i is never greater than n. As a result, the O(i + n) term is equivalent to an O(n) term, since i + n = O(n). This makes our overall loop
for (int i = 0; i < n; ++i) {
do O(n) work;
}
From here, it should be a bit clearer that the overall runtime is O(n2), so we do O(n) iterations, each of which does O(n) total work.
You asked in a comment in another answer about what would happen if the second of the nested loops only ran O(log n) times instead of O(n) times. That's a great exercise, so let's see what happens if we try that out!
Imagine the code looked like this:
for (int i = 0; i < n; ++i) {
for (int j = 0; j < i; ++j) {
cout<< i* j<<endl;
cout<< ("j = " + j);
}
for (int k = 0; k < n; k *= 2)
cout<<"k = " + k);
}
Here, the second loop runs only O(log n) times because k grows geometrically. Let's again apply the idea of working from the inside out. The inside now consists of these two loops:
for (int j = 0; j < i; ++j) {
cout<< i* j<<endl;
cout<< ("j = " + j);
}
for (int k = 0; k < n; k *= 2)
cout<<"k = " + k);
Here, that first loop runs in time O(i) (as before) and the new loop runs in time O(log n), so the total work done per iteration is O(i + log n). If we rewrite our original loops using this, we get something like this:
for (int i = 0; i < n; ++i) {
do O(i + log n) work;
}
This one is a bit trickier to analyze, because i changes from one iteration of the loop to the next. In this case, it often helps to approach the analysis not by multiplying the work done per iteration by the number of iterations, but rather by just adding up the work done across the loop iterations. If we do this here, we'll see that the work done is proportional to
(0 + log n) + (1 + log n) + (2 + log n) + ... + (n-1 + log n).
If we regroup these terms, we get
(0 + 1 + 2 + ... + n - 1) + (log n + log n + ... + log n) (n times)
That simplifies to
(0 + 1 + 2 + ... + n - 1) + n log n
That first part of the summation is Gauss's famous sum 0 + 1 + 2 + ... + n - 1, which happens to be equal to n(n-1) / 2. (It's good to know this!) This means we can rewrite the total work done as
n(n - 1) / 2 + n log n
= O(n2) + O(n log n)
= O(n2)
with that last step following because O(n log n) is dominated by the O(n2) term.
Hopefully this shows you both where the result comes from and how to come up with it. Work from the inside out, working out how much work each loop does and replacing it with a simpler "do O(X) work" statement to make things easier to follow. When you have some amount of work that changes as a loop counter changes, sometimes it's easiest to approach the problem by bounding the value and showing that it never leaves some range, and other times it's easiest to solve the problem by explicitly working out how much work is done from one loop iteration to the next.
When you have multiple loops in sequence, the time complexity of all of them is the worst complexity of any of them. Since both of the inner loops are O(n), the worst is also O(n).
So since you have O(n) code inside an O(n) loop, the total complexity of everything is O(n2).
O n squared; calculate the area of a triangle.
We get 1+2+3+4+5+...+n, which is the nth triangular number. If you graph it, it is basically a triangle of height and width n.
A triangle with base n and height n has area 1/2 n^2. O doesn't care about constants like 1/2.
I have a situation where I am using openMP for the Xeon Phi Coprocessor, and I have an opportunity to parallelize an "embarrassingly parallel" double for loop.
However, the for loop is looping through the upper triangle (including the diagonal):
for (int i = 0; i < n; i++)
// access some arrays with the value of i
for (int j = i; j < n; j++)
// do some stuff with the values of j and i
So, I've got the total size of the loop,
for (int q = 0; q < ((n*n - n)/2)+n; q++)
// Do stuff
But where I am struggling is:
How do I calculate i and j from q? Is it possible?
In the meantime, I'm just doing the full matrix, calculating i and j from there, and only doing my stuff when j >= i...but that still leaves a hefty amount of thread overhead.
If I restate your problem, to find i and j from q, you need the greatest i such that
q >= i*n - (i-1)*i/2
to define j as
j = i + (q - i*n - (i-1)*i/2)
If you have such a greatest i, then
(i+1)*n - i*(i+1)/2 > q >= i*n - (i-1)*i/2
n-i > (q - i*n - (i-1)*i/2) >= 0
n > j = i + (q - i*n - (i-1)*i/2) >= i
Here is a first iterative method to find i:
for (i = 0; q >= i*n - (i-1)*i/2; ++i);
i = i-1;
If you iterate over q, the computation of i is likely to exploit the iterative process.
A second method could use sqrt since
i*n - i²/2 + i/2 ~ q
i²/2 - i(n+1/2) + q ~ 0
delta = (n+0.5)² - 2q
i ~ (n+0.5) - sqrt(delta)
i could be defined as floor((n+0.5) - sqrt((n+0.5)² - 2q))
OK, this isn't an answer as far as I can tell. But, it is a workaround (for now).
I was thinking about it, and creating an array (or corresponding pointer), a[(n*n + n)/2 + n][2], and reading in the corresponding i and j values in my calling code, and passing this to my function would allow for the speed up.
You can make your loop to iterate as it is iterating on the whole matrix.
And just to keep track on your current line and each time you are entering a new line increment the index with the value of that line.
Then: i == line and j == i%n.
See this code:
int main() {
int n = 10;
int line = 0;
for (int i=0; i<n*n; i++){
if (i%n == 0 && i!=0){
line++;
i += line;
cout << endl;
}
cout << "("<<line<<","<<i%n<<")";
}
return 0;
}
Running Example
This question already has answers here:
Big O, how do you calculate/approximate it?
(24 answers)
Closed 8 years ago.
Compute the complexity of the following Algorithm?
I have the following code snippet:
i = 1;
while (i < n + 1) {
j = 1;
while (j < n + 1) {
j = j * 2;
}
i = i + 1;
}
plz explain it in detail
I want to know the the steps to solve the problem so I can solve such problems
Since j grows exponentially, the inner loop takes O(log(n)).
Since i grows linearly, the outer loop takes O(n).
Hence the overall complexity is O(n*log(n)).
i = 1;
while(i < n + 1){
j = 1;
While(j < n + 1){
j = j * 2:
}
i = i + 1;
}
outer loop takes O(n) since it increments by constant.
i = 1;
while(i < n + 1){
i = i + 1;
}
inner loop : j = 1, 2, 4, 8, 16, ...., 2^k
j = 2^k (k >= 0)
when will j stops ?
when j == n,
log(2^k) = log(n)
=> k * lg(2) = lg(n) ..... so k = lg(n).
While(j < n + 1){
j = j * 2;
}
so total O(n * lg(n))
You can simply understand outer-loop(with i) because it loops exactly n times. (1, 2, 3, ..., n). But inner-loop(j) is little difficult to understand.
Let's assume that n is 8. How much it loops? Starting with j = 1, it will be increased as exponentially : 1, 2, 4, 8. When j is over 8, loop will be terminated. It loops exactly 4 times. Then we can think general-form of this problem...
Think of that sequence 1, 2, 4, 8, .... If n is 2^k (k is non-negative integer), inner-loop will take k+1 times. (Because 2^(loop-1) = 2^k) Due to the assumption : n = 2^k, we can say that k = lg(n). So we can say inner-loop takes lg(n)+1 times.
When n is not exactly fit to 2^k, it takes one more time. ([lg(n)]+1) It's not a big deal with complexity though it has floor function. You can ingonre it this time.
So the total costs will be like this : n*(lg(n)+1). If you are familiar with Big-O notation, it can be expressed as : O(n lg n).
This one is similar to the following code :
for( int i = 1;i < n+1 ; i++){ // this loop runs n times
for(int j = 1 ; j<n+1 ; j=j*2){// this loop runs log_2(n)(log base 2 because it grows exponentially with 2)
//body
}
}
Hence in Big-Oh notation it is O(n)*O(logn) ; i.e, O(n*logn)
You can proceed like the following: