What is the time complexity of this for loop (be related to `n`)? - c++

What is the time complexity of this for loop (be related to n)?
for(int i = 1, j; i <= n; i = j + 1)
{
j = n / (n / i);
}
Please note that i, j and n are integer variables and they follow integer arithmetic. In particular, the expression n/(n/i) inside the loop should be interpreted as below:

If we use j = i; instead of j = n / (n / i);, the time complexity is O(n).
Now it's j = n / (n / i);, suppose that n = i*k+r, where k and r are all integers and r = n%i. Thus j = (i*k+r)/((i*k+r)/i) = (i*k+r)/k = i+r/k >= i, which means i will increment faster than the case where you use j = i;. So at least the time complexity is less than O(n), which I suppose gives you another O(n).
And besides the big O notation, there are another two notations(Θ and Ω) which means the lower and upper bound of O(n). You can get time complexity by finding these two bounds. And there's another rule if I remember correctly, O(k*n) = O(n), the coefficient k doesn't matter no matter how big it is.

As elaborated by taotsi, the increment for i in each iteration is
inc = 1 + r/k
where r=n%i and k=n/i. Since r<i, the increment is 1 as long as i<sqrt(n) (because then i*i/n<1 become 0 in integer division). Thereafter, the increment is (typically) 2 as long as i<2*sqrt(n). This continues similar to the geometric series, giving a factor 2 over sqrt(n), i.e. 2 sqrt(n) iterations.
If we write n = a*a+b with integers 0 <= b <= 2*a (i.e. a=int(sqrt(n)) and b=n-a*a), then the total number of iterations in simple experiments is always
b < a? 2*a-1 : 2*a
Thus, the complexity is O(√n) (provided some useful work is done inside the loop, for example counting the number of total iterations, such that the compiler is not allowed to elide the whole loop).

As #Walter has already offered a proof, I am too late for that part, but here is a Python3 version of your code and a plot of the number of iterations as a function of n vs the 2*sqrt(n) function. They look approximately the same (up to n = 1e9).
import matplotlib.pyplot as plt
from numba import jit
import math
#jit
def weird_increment_loop(n):
i = 1
j = 0
iterations = 0
while i <= n:
j = n // (n // i)
i = j + 1
iterations = iterations + 1
return iterations
iterations = []
func_2sqrt = []
domain = range(0,1000000001,1000000)
for n in domain:
iterations.append(weird_increment_loop(n))
func_2sqrt.append(math.sqrt(n)*2)
plt.plot(domain,iterations)
plt.plot(domain,func_2sqrt)
plt.xlabel("n")
plt.ylabel("iterations(n) and 2*sqrt(n)")
plt.show()
Here is the plot:
If you see no difference, it is because there is close to none :D Of course, one should always trust Mathematics ;)

Strictly by the rules of C++, it's O(1). Either the loop terminates after some finite amount of doing no observable work, or it loops forever (which is undefined behaviour). A conforming implementation may assume that undefined behaviour is not encountered, so we may assume it terminates.
Because the observable effects of the program does not depend on what happens inside the loop, an implementation is allowed to "As-if" it into nothingness.

Related

What's the time complexity of for (int i = 2; i < n; i = i*i)?

What would be the time complexity of the following loop?
for (int i = 2; i < n; i = i * i) {
++a;
}
While practicing runtime complexities, I came across this code and can't find the answer. I thought this would be sqrt(n), though it doesn't seem correct, since the loop has the sequence of 2, 4, 16, 256, ....
To understand the answer you must understand that: Inverse of Exponent is not SQRT, but log is.
This loop is multiplying i by itself(.i.e. exponential increment) and will stop only when i >= n, therefore the complexity would be O(log(n)) (log to the base 2 to be precise because i=2 at initialization)
To illustrate this:
In the above image, you can see that SQRT is giving correct number of steps only when i is a even power of 2. However log2 is giving accurate number of steps everytime.
Each time i is powered by 2. Hence, if A(n) shows the current value of i in the last step (which is n), it can be written in a recursive for like the following (suppose n is power of 2):
A(n) = A(n-1)^2
Now, you can expand it to find a pattern:
A(n) = A(n-2)^4 = A(n-3)^8 = ... = A(n-(n-1))^(2^(n-1)) = 2^(2^(n-1))
Hence, the loop iterates k step such that n = 2 ^ (2^ (k-1)). Therefore, this loop iterates Theta(log(log(n)).

Counting the basic operations of a given program

I am looking at the following: Operations Counting Example
Which is supposed to present the operations count of the following pseudocode:
Algorithm prefixAverages(A)
Input array A of n numbers
Output array B of n numbers such that B[i] is the average
of elements A[0], A[1], … , A[i]
for i = 0 to n - 1 do
b = 0
for j = 0 to i do
b = b + A[j]
j++;
B[i] = b / (i + 1)
return B
But I don't see how the counts on the inner for loop are reached. It says that for case i=0; j=0; the inner for loop runs twice? But it strikes me that it should only run once to see that 0 < 0. Can anyone provide insight into where the given operations count comes from or provide their own operations count?
This is under the assumption that primitive operations are:
Assignment
Array access
Mathematical operators (+, -, /, *)
Comparison
Increment/Decrement (math in disguise)
Return statements
Let me know if anything is unclear or you need more information
When the article you are following says "for var <- 0 to var2", it is like "for (var = 0; var <= var2; var++), so yes, when i = 0, it enters the "for" twice (once when i = 0, and again when i = 1, then it goes out).
(Sorry if bad english)
Edit and improve: When I calculate the complexity of a program, the only thing that interest me is the big O complexity; in this case, you have that the 'i' loop run 'n' times, and the 'j' loop run 'i' times, so the 'i' loop runs (1+2+3+...+n) times, that is n(n+1)/2 times, and that is an O(n**2) complexity.
In the first line, you have an assignament (i = something), and a comparison (i <= n-1) ("2 operations") for each i value, and as the last value is i=n, it does those 2 operations since i=0, until i=n, and as those are n+1 values (from 0 to n), this line do 2(n+1) operations.
The second line is a little obvious, as it enters the loop n times (since i=0, until i=n-1).
On the second loop, it do 2 things, an assignament, and a comparison (just as the first loop), and it do this i+2 times (for example, when i=0, it enters the loop 1 time, but it has to do the i=1 assignament, and the 1<=0 comparison, so its 2 times in total), so it do this calculus 2(i+2) times, but it do this since i=0, until i=n-1, so to calculate all of this, we have to do the sum (sum from i=0 until i=n-1: 2(i+2)) = 2((sum of i from 0 to n-1) + (sum of 2 from i=0 to i=n-1)) = 2((n(n-1)/2) + 2n) = n(n-1) + 4n = n"2 - n + 4n = n"2 + 3n.
I'll continue this later, I hope my answer so far is helpful for you. (again, sorry if some bad english)

Big O Notation for Algorithm

I'm busy doing an assignment and I'm struggling with a question. I know I'm not supposed to ask assignment questions outright so I understand if I don't get straight answers. But here goes anyway.
We must calculate the run time complexity of different algorithms, the one I'm stuck on is this.
for(int i = 1 ; i < n ; i++)
for(int j = 0 ; j < i ; j +=2)
sum++;
Now with my understanding, my first thought would be less than O(n2), because the nested loop isn't running the full n times, and still the j variable is incrementing by 2 each loop rather than iterating like a normal for loop. Although, when I did some code simulations with N=10, N=100, N=1000, etc. I got the following results when I outputted the sum variable.
N = 10 : 25,
N = 100 : 2500,
N = 1000 : 250000,
N = 10000 : 25000000
When I look at these results, the O Notations seems like it should be much larger than just O(n).
The 4 options we have been given in the assignment are : O(1), O(n2), O(n) and O(logn). As I said earlier, I cannot see how it can be as large as O(n2), but the results are pointing to that. So I just think I don't fully understand this, or I'm missing some link.
Any help would be appreciated!
Big O notation does not give you the number of operations. It just tells you how fast it will grow with growing input. And this is what you observe.
When you increased input c times, the total number of operations grows c^2.
If you calculated (nearly) exact number of operations precisely you would get (n^2)/4.
Of course you can calculate it with sums, but since I dunno how to use math on SO I will give an "empirical" explanation. Simple loop-within-a-loop with the same start and end conditions gives n^2. Such loop produces a matrix of all possible combinations for "i" and "j". So if start is 1 and end is N in both cases you get N*N combinations (or iterations effectively).
However, yours inner loop is for i < j. This basically makes a triangle out of this square, that is the 1st 0.5 factor, and then you skip every other element, this is another 0.5 factor; multiplied you get 1/4.
And O(0.25 * n^2) = O(n^2). Sometimes people like to leave the factor in there because it lets you compare two algorithms with the same complexity. But it does not change the ratio of growth in respect to n.
Bear in mind that big-O is asymptotic notation. Constants (additive or multiplicative) have zero impact on it.
So, the outer loop runs n times, and on the ith time, the inner loop runs i / 2 times. If it weren't for the / 2 part, it would be the sum of all numbers 1 .. n, which is the well known n * (n + 1) / 2. That expands to a * n^2 + b * n + c for a non-zero a, so it's O(n^2).
Instead of summing n numbers, we're summing n / 2 numbers. But that's still somewhere around (n/2) * ((n/2) + 1) / 2. Which still expands to d * n^2 + e * n + f for a non-zero d, so it's still O(n^2).
From your output it seems like:
sum ~= (n^2)/4.
This is obviously O(n^2) (actually you can replace the O with teta).
You should recall the definition for Big-O notation. See http://en.wikipedia.org/wiki/Big_O_notation.
The thing is that number of operations here is dependant on the square of n, even though the overall number is less than n². Nevertheless, the scaling is what matters for Big-O notation, thus it's O(n²)
With:
for (int i = 1 ; i < n ; i++)
for (int j = 0 ; j < i ; j +=2)
sum++;
We have:
0+2+4+6+...+2N == 2 * (0+1+2+3+...+N) == 2 * (N * (N+1) / 2) == N * (N+1)
so, with n == 2N, we have (n / 2) * (n / 2 + 1) ~= (n * n) / 4
so O(n²)
Your understanding regarding time complexity is not appropriate.Time Complexity is not only the matter of 'sum' variable.'sum' only calculates how many times the inner loop iterates,but you also have to consider total number of outer loop iterations.
now consider your program:
for(int i = 1 ; i < n ; i++)
for(int j = 0 ; j < i ; j +=2)
sum++;
Time complexity means running time of your program with respect to input values(here n).Here running time does not mean actual required time to execute your program in your computer .Actual required time varies from machine to machine.so to get a machine independent running time, Big O notation is very useful.Bog O actually comes from mathematics and it describes the running time in terms of mathematical functions.
The outer loop is executed total (n-1) times.for each of these (n-1) values (starting from i=1), the inner loop iterates i/2 times.so total number of inner loop iterations=1+1+2+2+3+3+...+(n/2)+(n/2)=2(1+2+3+...+n/2)=2*(n/2(n/2+1))/2=n^2/4+n/2.
similarly 'sum++' also executed total n^2/4+n/2 times.Now consider cost of line 1 of the program=c1,cost of line 2=c2 and cost of line 3=c3 .These casts can be different for different machine. so total time required for executing the program =c1*(n-1)+c2*(n^2/4+n/2)+c3*(n^2/4+n/2)=(c2+c3)n^2/4+(c2+c3)n/2+c1*n-c1.Thus the required time can be expressed in terms of mathematical function.In Big O notation you can say it is O((c2+c3)n^2/4+(c2+c3)n/2+c1*n-c1).In case of Big O notation, lower order terms and coefficient of highest order term can be ignored. because for large value of n ,n^2 is much greater than n. so you can say it is O((c1+c2)n^2/4).Also as for any value of n , n^2 is greater than (c1+c2)n^2/4 by a constant factor, so you can say it is O(n^2).

Confusion with determining Big-O notation?

So, I really don't get Big O notation. I have been tasked with determining "O value" for this code segment.
for (int count =1; count < n; count++) // Runs n times, so linear, or O(N)
{
int count2 = 1; // Declares an integer, so constant, O(1)
while (count2 < count) // Here's where I get confused. I recognize that it is a nested loop, but does that make it O(N^2)?
{
count2 = count2 * 2; // I would expect this to be constant as well, O(N)
}
}
O(f(n))=g(n)
This implies that for some value k, f(n)>g(n) where n>k. This gives the upper bound for the function g(n).
When you are asked to find Big O for some code,
1) Try to count the number of computations being performed in terms of n and thus getting g(n).
2) Now try estimating the upper bound function of g(n). That will be your answer.
Lets apply this procedure to your code.
Lets count the number of computations made. The statements declaring and multiply by 2 take O(1) time. But these are executed repeatedly. We need to find how many times they are executed.
The outer loop executes for n times. Hence the first statement executes for n times. Now the number of times inner loop gets executed depends on value of n. For a given value of n it executes for logn times.
Now lets count the total number of computations performed,
log(1) + log(2) + log(3) +.... log(n) + n
Note that the last n is for the first statement. Simplifying the above series we get:
= log(1*2*3*...n) + n
= log(n!) + n
We have
g(n)=log(n!) + n
Lets guess the upper bound for log(n!).
Since,
1.2.3.4...n < n.n.n...(n times)
Hence,
log(n!) < log(n^n) for n>1
which implies
log(n!) = O(nlogn).
If you want a formal proof for this, check this out. Since nlogn increases faster than n , we therefore have:
O(nlogn + n) = O(nlogn)
Hence your final answer is O(nlogn).

Algorithm analysis: Am I analyzing these algorithms correctly? How to approach problems like these [closed]

This question is unlikely to help any future visitors; it is only relevant to a small geographic area, a specific moment in time, or an extraordinarily narrow situation that is not generally applicable to the worldwide audience of the internet. For help making this question more broadly applicable, visit the help center.
Closed 10 years ago.
1)
x = 25;
for (int i = 0; i < myArray.length; i++)
{
if (myArray[i] == x)
System.out.println("found!");
}
I think this one is O(n).
2)
for (int r = 0; r < 10000; r++)
for (int c = 0; c < 10000; c++)
if (c % r == 0)
System.out.println("blah!");
I think this one is O(1), because for any input n, it will run 10000 * 10000 times. Not sure if this is right.
3)
a = 0
for (int i = 0; i < k; i++)
{
for (int j = 0; j < i; j++)
a++;
}
I think this one is O(i * k). I don't really know how to approach problems like this where the inner loop is affected by variables being incremented in the outer loop. Some key insights here would be much appreciated. The outer loop runs k times, and the inner loop runs 1 + 2 + 3 + ... + k times. So that sum should be (k/2) * (k+1), which would be order of k^2. So would it actually be O(k^3)? That seems too large. Again, don't know how to approach this.
4)
int key = 0; //key may be any value
int first = 0;
int last = intArray.length-1;;
int mid = 0;
boolean found = false;
while( (!found) && (first <= last) )
{
mid = (first + last) / 2;
if(key == intArray[mid])
found = true;
if(key < intArray[mid])
last = mid - 1;
if(key > intArray[mid])
first = mid + 1;
}
This one, I think is O(log n). But, I came to this conclusion because I believe it is a binary search and I know from reading that the runtime is O(log n). I think it's because you divide the input size by 2 for each iteration of the loop. But, I don't know if this is the correct reasoning or how to approach similar algorithms that I haven't seen and be able to deduce that they run in logarithmic time in a more verifiable or formal way.
5)
int currentMinIndex = 0;
for (int front = 0; front < intArray.length; front++)
{
currentMinIndex = front;
for (int i = front; i < intArray.length; i++)
{
if (intArray[i] < intArray[currentMinIndex])
{
currentMinIndex = i;
}
}
int tmp = intArray[front];
intArray[front] = intArray[currentMinIndex];
intArray[currentMinIndex] = tmp;
}
I am confused about this one. The outer loop runs n times. And the inner for loop runs
n + (n-1) + (n-2) + ... (n - k) + 1 times? So is that O(n^3) ??
More or less, yes.
1 is correct - it seems you are searching for a specific element in what I assume is an un-sorted collection. If so, the worst case is that the element is at the very end of the list, hence O(n).
2 is correct, though a bit strange. It is O(1) assuming r and c are constants and the bounds are not variables. If they are constant, then yes O(1) because there is nothing to input.
3 I believe that is considered O(n^2) still. There would be some constant factor like k * n^2, drop the constant and you got O(n^2).
4 looks a lot like a binary search algorithm for a sorted collection. O(logn) is correct. It is log because at each iteration you are essentially halving the # of possible choices in which the element you are looking for could be in.
5 is looking like a bubble sort, O(n^2), for similar reasons to 3.
O() doesn't mean anything in itself: you need to specify if you are counting the "worst-case" O, or the average-case O. For some sorting algorithm, they have a O(n log n) on average but a O(n^2) in worst case.
Basically you need to count the overall number of iterations of the most inner loop, and take the biggest component of the result without any constant (for example if you have k*(k+1)/2 = 1/2 k^2 + 1/2 k, the biggest component is 1/2 k^2 therefore you are O(k^2)).
For example, your item 4) is in O(log(n)) because, if you work on an array of size n, then you will run one iteration on this array, and the next one will be on an array of size n/2, then n/4, ..., until this size reaches 1. So it is log(n) iterations.
Your question is mostly about the definition of O().
When someone say this algorithm is O(log(n)), you have to read:
When the input parameter n becomes very big, the number of operations performed by the algorithm grows at most in log(n)
Now, this means two things:
You have to have at least one input parameter n. There is no point in talking about O() without one (as in your case 2).
You need to define the operations that you are counting. These can be additions, comparison between two elements, number of allocated bytes, number of function calls, but you have to decide. Usually you take the operation that's most costly to you, or the one that will become costly if done too many times.
So keeping this in mind, back to your problems:
n is myArray.Length, and the number of operations you're counting is '=='. In that case the answer is exactly n, which is O(n)
you can't specify an n
the n can only be k, and the number of operations you count is ++. You have exactly k*(k+1)/2 which is O(n2) as you say
this time n is the length of your array again, and the operation you count is ==. In this case, the number of operations depends on the data, usually we talk about 'worst case scenario', meaning that of all the possible outcome, we look at the one that takes the most time. At best, the algorithm takes one comparison. For the worst case, let's take an example. If the array is [[1,2,3,4,5,6,7,8,9]] and you are looking for 4, your intArray[mid] will become successively, 5, 3 and then 4, and so you would have done the comparison 3 times. In fact, for an array which size is 2^k + 1, the maximum number of comparison is k (you can check). So n = 2^k + 1 => k = ln(n-1)/ln(2). You can extend this result to the case when n is not = 2^k + 1, and you will get complexity = O(ln(n))
In any case, I think you are confused because you don't exactly know what O(n) means. I hope this is a start.