I am looking at the following: Operations Counting Example
Which is supposed to present the operations count of the following pseudocode:
Algorithm prefixAverages(A)
Input array A of n numbers
Output array B of n numbers such that B[i] is the average
of elements A[0], A[1], … , A[i]
for i = 0 to n - 1 do
b = 0
for j = 0 to i do
b = b + A[j]
j++;
B[i] = b / (i + 1)
return B
But I don't see how the counts on the inner for loop are reached. It says that for case i=0; j=0; the inner for loop runs twice? But it strikes me that it should only run once to see that 0 < 0. Can anyone provide insight into where the given operations count comes from or provide their own operations count?
This is under the assumption that primitive operations are:
Assignment
Array access
Mathematical operators (+, -, /, *)
Comparison
Increment/Decrement (math in disguise)
Return statements
Let me know if anything is unclear or you need more information
When the article you are following says "for var <- 0 to var2", it is like "for (var = 0; var <= var2; var++), so yes, when i = 0, it enters the "for" twice (once when i = 0, and again when i = 1, then it goes out).
(Sorry if bad english)
Edit and improve: When I calculate the complexity of a program, the only thing that interest me is the big O complexity; in this case, you have that the 'i' loop run 'n' times, and the 'j' loop run 'i' times, so the 'i' loop runs (1+2+3+...+n) times, that is n(n+1)/2 times, and that is an O(n**2) complexity.
In the first line, you have an assignament (i = something), and a comparison (i <= n-1) ("2 operations") for each i value, and as the last value is i=n, it does those 2 operations since i=0, until i=n, and as those are n+1 values (from 0 to n), this line do 2(n+1) operations.
The second line is a little obvious, as it enters the loop n times (since i=0, until i=n-1).
On the second loop, it do 2 things, an assignament, and a comparison (just as the first loop), and it do this i+2 times (for example, when i=0, it enters the loop 1 time, but it has to do the i=1 assignament, and the 1<=0 comparison, so its 2 times in total), so it do this calculus 2(i+2) times, but it do this since i=0, until i=n-1, so to calculate all of this, we have to do the sum (sum from i=0 until i=n-1: 2(i+2)) = 2((sum of i from 0 to n-1) + (sum of 2 from i=0 to i=n-1)) = 2((n(n-1)/2) + 2n) = n(n-1) + 4n = n"2 - n + 4n = n"2 + 3n.
I'll continue this later, I hope my answer so far is helpful for you. (again, sorry if some bad english)
Related
What is the time complexity of this for loop (be related to n)?
for(int i = 1, j; i <= n; i = j + 1)
{
j = n / (n / i);
}
Please note that i, j and n are integer variables and they follow integer arithmetic. In particular, the expression n/(n/i) inside the loop should be interpreted as below:
If we use j = i; instead of j = n / (n / i);, the time complexity is O(n).
Now it's j = n / (n / i);, suppose that n = i*k+r, where k and r are all integers and r = n%i. Thus j = (i*k+r)/((i*k+r)/i) = (i*k+r)/k = i+r/k >= i, which means i will increment faster than the case where you use j = i;. So at least the time complexity is less than O(n), which I suppose gives you another O(n).
And besides the big O notation, there are another two notations(Θ and Ω) which means the lower and upper bound of O(n). You can get time complexity by finding these two bounds. And there's another rule if I remember correctly, O(k*n) = O(n), the coefficient k doesn't matter no matter how big it is.
As elaborated by taotsi, the increment for i in each iteration is
inc = 1 + r/k
where r=n%i and k=n/i. Since r<i, the increment is 1 as long as i<sqrt(n) (because then i*i/n<1 become 0 in integer division). Thereafter, the increment is (typically) 2 as long as i<2*sqrt(n). This continues similar to the geometric series, giving a factor 2 over sqrt(n), i.e. 2 sqrt(n) iterations.
If we write n = a*a+b with integers 0 <= b <= 2*a (i.e. a=int(sqrt(n)) and b=n-a*a), then the total number of iterations in simple experiments is always
b < a? 2*a-1 : 2*a
Thus, the complexity is O(√n) (provided some useful work is done inside the loop, for example counting the number of total iterations, such that the compiler is not allowed to elide the whole loop).
As #Walter has already offered a proof, I am too late for that part, but here is a Python3 version of your code and a plot of the number of iterations as a function of n vs the 2*sqrt(n) function. They look approximately the same (up to n = 1e9).
import matplotlib.pyplot as plt
from numba import jit
import math
#jit
def weird_increment_loop(n):
i = 1
j = 0
iterations = 0
while i <= n:
j = n // (n // i)
i = j + 1
iterations = iterations + 1
return iterations
iterations = []
func_2sqrt = []
domain = range(0,1000000001,1000000)
for n in domain:
iterations.append(weird_increment_loop(n))
func_2sqrt.append(math.sqrt(n)*2)
plt.plot(domain,iterations)
plt.plot(domain,func_2sqrt)
plt.xlabel("n")
plt.ylabel("iterations(n) and 2*sqrt(n)")
plt.show()
Here is the plot:
If you see no difference, it is because there is close to none :D Of course, one should always trust Mathematics ;)
Strictly by the rules of C++, it's O(1). Either the loop terminates after some finite amount of doing no observable work, or it loops forever (which is undefined behaviour). A conforming implementation may assume that undefined behaviour is not encountered, so we may assume it terminates.
Because the observable effects of the program does not depend on what happens inside the loop, an implementation is allowed to "As-if" it into nothingness.
If n numbers are given, how would I find the total number of possible triangles? Is there any method that does this in less than O(n^3) time?
I am considering a+b>c, b+c>a and a+c>b conditions for being a triangle.
Assume there is no equal numbers in given n and it's allowed to use one number more than once. For example, we given a numbers {1,2,3}, so we can create 7 triangles:
1 1 1
1 2 2
1 3 3
2 2 2
2 2 3
2 3 3
3 3 3
If any of those assumptions isn't true, it's easy to modify algorithm.
Here I present algorithm which takes O(n^2) time in worst case:
Sort numbers (ascending order).
We will take triples ai <= aj <= ak, such that i <= j <= k.
For each i, j you need to find largest k that satisfy ak <= ai + aj. Then all triples (ai,aj,al) j <= l <= k is triangle (because ak >= aj >= ai we can only violate ak < a i+ aj).
Consider two pairs (i, j1) and (i, j2) j1 <= j2. It's easy to see that k2 (found on step 2 for (i, j2)) >= k1 (found one step 2 for (i, j1)). It means that if you iterate for j, and you only need to check numbers starting from previous k. So it gives you O(n) time complexity for each particular i, which implies O(n^2) for whole algorithm.
C++ source code:
int Solve(int* a, int n)
{
int answer = 0;
std::sort(a, a + n);
for (int i = 0; i < n; ++i)
{
int k = i;
for (int j = i; j < n; ++j)
{
while (n > k && a[i] + a[j] > a[k])
++k;
answer += k - j;
}
}
return answer;
}
Update for downvoters:
This definitely is O(n^2)! Please read carefully "An Introduction of Algorithms" by Thomas H. Cormen chapter about Amortized Analysis (17.2 in second edition).
Finding complexity by counting nested loops is completely wrong sometimes.
Here I try to explain it as simple as I could. Let's fix i variable. Then for that i we must iterate j from i to n (it means O(n) operation) and internal while loop iterate k from i to n (it also means O(n) operation). Note: I don't start while loop from the beginning for each j. We also need to do it for each i from 0 to n. So it gives us n * (O(n) + O(n)) = O(n^2).
There is a simple algorithm in O(n^2*logn).
Assume you want all triangles as triples (a, b, c) where a <= b <= c.
There are 3 triangle inequalities but only a + b > c suffices (others then hold trivially).
And now:
Sort the sequence in O(n * logn), e.g. by merge-sort.
For each pair (a, b), a <= b the remaining value c needs to be at least b and less than a + b.
So you need to count the number of items in the interval [b, a+b).
This can be simply done by binary-searching a+b (O(logn)) and counting the number of items in [b,a+b) for every possibility which is b-a.
All together O(n * logn + n^2 * logn) which is O(n^2 * logn). Hope this helps.
If you use a binary sort, that's O(n-log(n)), right? Keep your binary tree handy, and for each pair (a,b) where a b and c < (a+b).
Let a, b and c be three sides. The below condition must hold for a triangle (Sum of two sides is greater than the third side)
i) a + b > c
ii) b + c > a
iii) a + c > b
Following are steps to count triangle.
Sort the array in non-decreasing order.
Initialize two pointers ‘i’ and ‘j’ to first and second elements respectively, and initialize count of triangles as 0.
Fix ‘i’ and ‘j’ and find the rightmost index ‘k’ (or largest ‘arr[k]‘) such that ‘arr[i] + arr[j] > arr[k]‘. The number of triangles that can be formed with ‘arr[i]‘ and ‘arr[j]‘ as two sides is ‘k – j’. Add ‘k – j’ to count of triangles.
Let us consider ‘arr[i]‘ as ‘a’, ‘arr[j]‘ as b and all elements between ‘arr[j+1]‘ and ‘arr[k]‘ as ‘c’. The above mentioned conditions (ii) and (iii) are satisfied because ‘arr[i] < arr[j] < arr[k]'. And we check for condition (i) when we pick 'k'
4.Increment ‘j’ to fix the second element again.
Note that in step 3, we can use the previous value of ‘k’. The reason is simple, if we know that the value of ‘arr[i] + arr[j-1]‘ is greater than ‘arr[k]‘, then we can say ‘arr[i] + arr[j]‘ will also be greater than ‘arr[k]‘, because the array is sorted in increasing order.
5.If ‘j’ has reached end, then increment ‘i’. Initialize ‘j’ as ‘i + 1′, ‘k’ as ‘i+2′ and repeat the steps 3 and 4.
Time Complexity: O(n^2).
The time complexity looks more because of 3 nested loops. If we take a closer look at the algorithm, we observe that k is initialized only once in the outermost loop. The innermost loop executes at most O(n) time for every iteration of outer most loop, because k starts from i+2 and goes upto n for all values of j. Therefore, the time complexity is O(n^2).
I have worked out an algorithm that runs in O(n^2 lgn) time. I think its correct...
The code is wtitten in C++...
int Search_Closest(A,p,q,n) /*Returns the index of the element closest to n in array
A[p..q]*/
{
if(p<q)
{
int r = (p+q)/2;
if(n==A[r])
return r;
if(p==r)
return r;
if(n<A[r])
Search_Closest(A,p,r,n);
else
Search_Closest(A,r,q,n);
}
else
return p;
}
int no_of_triangles(A,p,q) /*Returns the no of triangles possible in A[p..q]*/
{
int sum = 0;
Quicksort(A,p,q); //Sorts the array A[p..q] in O(nlgn) expected case time
for(int i=p;i<=q;i++)
for(int j =i+1;j<=q;j++)
{
int c = A[i]+A[j];
int k = Search_Closest(A,j,q,c);
/* no of triangles formed with A[i] and A[j] as two sides is (k+1)-2 if A[k] is small or equal to c else its (k+1)-3. As index starts from zero we need to add 1 to the value*/
if(A[k]>c)
sum+=k-2;
else
sum+=k-1;
}
return sum;
}
Hope it helps........
possible answer
Although we can use binary search to find the value of 'k' hence improve time complexity!
N0,N1,N2,...Nn-1
sort
X0,X1,X2,...Xn-1 as X0>=X1>=X2>=...>=Xn-1
choice X0(to Xn-3) and choice form rest two item x1...
choice case of (X0,X1,X2)
check(X0<X1+X2)
OK is find and continue
NG is skip choice rest
It seems there is no algorithm better than O(n^3). In the worst case, the result set itself has O(n^3) elements.
For Example, if n equal numbers are given, the algorithm has to return n*(n-1)*(n-2) results.
I'm busy doing an assignment and I'm struggling with a question. I know I'm not supposed to ask assignment questions outright so I understand if I don't get straight answers. But here goes anyway.
We must calculate the run time complexity of different algorithms, the one I'm stuck on is this.
for(int i = 1 ; i < n ; i++)
for(int j = 0 ; j < i ; j +=2)
sum++;
Now with my understanding, my first thought would be less than O(n2), because the nested loop isn't running the full n times, and still the j variable is incrementing by 2 each loop rather than iterating like a normal for loop. Although, when I did some code simulations with N=10, N=100, N=1000, etc. I got the following results when I outputted the sum variable.
N = 10 : 25,
N = 100 : 2500,
N = 1000 : 250000,
N = 10000 : 25000000
When I look at these results, the O Notations seems like it should be much larger than just O(n).
The 4 options we have been given in the assignment are : O(1), O(n2), O(n) and O(logn). As I said earlier, I cannot see how it can be as large as O(n2), but the results are pointing to that. So I just think I don't fully understand this, or I'm missing some link.
Any help would be appreciated!
Big O notation does not give you the number of operations. It just tells you how fast it will grow with growing input. And this is what you observe.
When you increased input c times, the total number of operations grows c^2.
If you calculated (nearly) exact number of operations precisely you would get (n^2)/4.
Of course you can calculate it with sums, but since I dunno how to use math on SO I will give an "empirical" explanation. Simple loop-within-a-loop with the same start and end conditions gives n^2. Such loop produces a matrix of all possible combinations for "i" and "j". So if start is 1 and end is N in both cases you get N*N combinations (or iterations effectively).
However, yours inner loop is for i < j. This basically makes a triangle out of this square, that is the 1st 0.5 factor, and then you skip every other element, this is another 0.5 factor; multiplied you get 1/4.
And O(0.25 * n^2) = O(n^2). Sometimes people like to leave the factor in there because it lets you compare two algorithms with the same complexity. But it does not change the ratio of growth in respect to n.
Bear in mind that big-O is asymptotic notation. Constants (additive or multiplicative) have zero impact on it.
So, the outer loop runs n times, and on the ith time, the inner loop runs i / 2 times. If it weren't for the / 2 part, it would be the sum of all numbers 1 .. n, which is the well known n * (n + 1) / 2. That expands to a * n^2 + b * n + c for a non-zero a, so it's O(n^2).
Instead of summing n numbers, we're summing n / 2 numbers. But that's still somewhere around (n/2) * ((n/2) + 1) / 2. Which still expands to d * n^2 + e * n + f for a non-zero d, so it's still O(n^2).
From your output it seems like:
sum ~= (n^2)/4.
This is obviously O(n^2) (actually you can replace the O with teta).
You should recall the definition for Big-O notation. See http://en.wikipedia.org/wiki/Big_O_notation.
The thing is that number of operations here is dependant on the square of n, even though the overall number is less than n². Nevertheless, the scaling is what matters for Big-O notation, thus it's O(n²)
With:
for (int i = 1 ; i < n ; i++)
for (int j = 0 ; j < i ; j +=2)
sum++;
We have:
0+2+4+6+...+2N == 2 * (0+1+2+3+...+N) == 2 * (N * (N+1) / 2) == N * (N+1)
so, with n == 2N, we have (n / 2) * (n / 2 + 1) ~= (n * n) / 4
so O(n²)
Your understanding regarding time complexity is not appropriate.Time Complexity is not only the matter of 'sum' variable.'sum' only calculates how many times the inner loop iterates,but you also have to consider total number of outer loop iterations.
now consider your program:
for(int i = 1 ; i < n ; i++)
for(int j = 0 ; j < i ; j +=2)
sum++;
Time complexity means running time of your program with respect to input values(here n).Here running time does not mean actual required time to execute your program in your computer .Actual required time varies from machine to machine.so to get a machine independent running time, Big O notation is very useful.Bog O actually comes from mathematics and it describes the running time in terms of mathematical functions.
The outer loop is executed total (n-1) times.for each of these (n-1) values (starting from i=1), the inner loop iterates i/2 times.so total number of inner loop iterations=1+1+2+2+3+3+...+(n/2)+(n/2)=2(1+2+3+...+n/2)=2*(n/2(n/2+1))/2=n^2/4+n/2.
similarly 'sum++' also executed total n^2/4+n/2 times.Now consider cost of line 1 of the program=c1,cost of line 2=c2 and cost of line 3=c3 .These casts can be different for different machine. so total time required for executing the program =c1*(n-1)+c2*(n^2/4+n/2)+c3*(n^2/4+n/2)=(c2+c3)n^2/4+(c2+c3)n/2+c1*n-c1.Thus the required time can be expressed in terms of mathematical function.In Big O notation you can say it is O((c2+c3)n^2/4+(c2+c3)n/2+c1*n-c1).In case of Big O notation, lower order terms and coefficient of highest order term can be ignored. because for large value of n ,n^2 is much greater than n. so you can say it is O((c1+c2)n^2/4).Also as for any value of n , n^2 is greater than (c1+c2)n^2/4 by a constant factor, so you can say it is O(n^2).
Hello everyone I trying to calculate the time complexity of Maximum Subsequence Sum.
Actually I know the answer which is O(n^3) and it follows from the function (n^3 + 3n^2 + 2n)/6
My question is how is that function obtained.
Quite simply, actually: just look at the loops in the code.
for (int i=0; i<n; i++)
for(j = i; j<n; j++) {
...
for (int k=i; k<=j; k++)
XXX;
The line XXX is executed n^3 times (modulo some constant factor and some lower powers of n), since the outer loop obviously runs from 0 to n-1, the "middle" loop runs from i (which will start out with 0, 1, ...) to n-1, meaning that the inner loop will be "started" approx n^2 times. Now, both i and j depend on n (eg., i will be 0 and j=n-1 at the end of the first outer iteration), so line XXX will be n times (for the inner loop) by n^2 times (for the outer two loops), resulting in a total of n^3.
To get the concrete function (n^3 + 3n^2 + 2n)/6, you'd have to be more thorough in your calculation and take care of all those factors I simply omitted above.
Here is how..
i=0
j=0 k=0 (count=1 )
j=1 k=0,1 (count =2)
j=2 k=0,1,2 (count = 3)
...
j=n-1 k=0,1,2,...n-1 (count = n)
Total number of times code executed = 1+2+3+...+n = n(n+1)/2
i=1
j=1 k=1 (count=1 )
j=2 k=1,2 (count =2)
j=3 k=1,2, 3 (count = 3)
...
j=n-1 k=1,2,...n-1 (count = n-2)
Total number of times code executed = 1+2+3+...+n-1 = (n-1)n/2
...
i=n-1
j=n-1 k=n-1 ( count = 1)
Total number of of times code executed = 1 = 1(1+1)/2
Now if we sum for all the values of i
n(n+1)/2 + ((n-1)((n-1)+1)/2+.....+1(1+1)/2
=∑ N(N+1)/2 =1/2∑(N^2 +N) =1/2(∑N^2+∑N)=1/2{ 1/6 N(N+1)(2N+1) + 1/2 N(N+1) } =1/2{ (2N^3 + 3N^2+N )/6 +(N^2+N)/2} =(N^3 + 3N^2 + 2N)/6
Check this solution suggested by Mark Allen Weiss (in his book).
Wikipedia claims that the failure function table can be computed in O(n) time.
Let's look at its `canonical' implementation (in C++):
vector<int> prefix_function (string s) {
int n = (int) s.length();
vector<int> pi (n);
for (int i=1; i<n; ++i) {
int j = pi[i-1];
while (j > 0 && s[i] != s[j])
j = pi[j-1];
if (s[i] == s[j]) ++j;
pi[i] = j;
}
return pi;
}
Why does it work in O(n) time, even if there is an inner while-loop? I'm not really strong at the analysis of algorithms, so may somebody explain it?
This line: if (s[i] == s[j]) ++j; is executed at most O(n) times.
It caused increase in the value of p[i]. Note that p[i] starts at same value as p[i-1].
Now this line: j = pi[j-1]; causes decrease of p[i] by at least one. And since it was increased at most O(n) times (we count also increases and decreases on previous values), it cannot be decreased more than O(n) times.
So it as also executed at most O(n) times.
Thus the whole time complexity is O(n).
There's already two answers here that are correct, but I often think a fully laid out
proof can make things clearer. You said you wanted an answer for a 9-year-old, but
I don't think it's feasible (I think it's easy to be fooled into thinking it's true
without actually having any intuition for why it's true). Maybe working through this answer will help.
First off, the outer loop runs n times clearly because i is not modified
within the loop. The only code within the loop that could run more than once is
the block
while (j > 0 && s[i] != s[j])
{
j = pi[j-1]
}
So how many times can that run? Well notice that every time that condition is
satisfied we decrease the value of j which, at this point, is at most
pi[i-1]. If it hits 0 then the while loop is done. To see why this is important,
we first prove a lemma (you're a very smart 9-year-old):
pi[i] <= i
This is done by induction. pi[0] <= 0 since it's set once in the initialization of pi and never touched again. Then inductively we let 0 < k < n and assume
the claim holds for 0 <= a < k. Consider the value of p[k]. It's set
precisely once in the line pi[i] = j. Well how big can j be? It's initialized
to pi[k-1] <= k-1 by induction. In the while block it then may be updated to pi[j-1] <= j-1 < pi[k-1]. By another mini-induction you can see that j will never increase past pi[k-1]. Hence after the
while loop we still have j <= k-1. Finally it might be incremented once so we have
j <= k and so pi[k] = j <= k (which is what we needed to prove to finish our induction).
Now returning back to the original point, we ask "how many times can we decrease the value of
j"? Well with our lemma we can now see that every iteration of the while loop will
monotonically decrease the value of j. In particular we have:
pi[j-1] <= j-1 < j
So how many times can this run? At most pi[i-1] times. The astute reader might think
"you've proven nothing! We have pi[i-1] <= i-1 but it's inside the while loop so
it's still O(n^2)!". The slightly more astute reader notices this extra fact:
However many times we run j = pi[j-1] we then decrease the value of pi[i] which shortens the next iteration of the loop!
For example, let's say j = pi[i-1] = 10. But after ~6 iterations of the while loop we have
j = 3 and let's say it gets incremented by 1 in the s[i] == s[j] line so j = 4 = pi[i].
Well then at the next iteration of the outer loop we start with j = 4... so we can only execute the while at most 4 times.
The final piece of the puzzle is that ++j runs at most once per loop. So it's not like we can have
something like this in our pi vector:
0 1 2 3 4 5 1 6 1 7 1 8 1 9 1
^ ^ ^ ^ ^
Those spots might mean multiple iterations of the while loop if this
could happen
To make this actually formal you might establish the invariants described above and then use induction
to show that the total number of times that while loop is run, summed with pi[i] is at most i.
From that, it follows that the total number of times the while loop is run is O(n) which means that the entire outer loop has complexity:
O(n) // from the rest of the outer loop excluding the while loop
+ O(n) // from the while loop
=> O(n)
Let's start with the fact the outer loop executes n times, where n is the length of the pattern we seek. The inner loop decreases the value of j by at least 1, since pi[j] < j. The loop terminates at the latest when j == -1, therefore it can decrease the value of j at most as often as it has been increased previously by j++ (the outer loop). Since j++ is executed in the outer loop exactly n times, the overall number of executions of the inner while loop is limited to n. The preprocessing algorithm therefore requires O(n) steps.
If you care, consider this simpler implementation of the preprocessing stage:
/* ff stands for 'failure function': */
void kmp_table(const char *needle, int *ff, size_t nff)
{
int pos = 2, cnd = 0;
if (nff > 1){
ff[0] = -1;
ff[1] = 0;
} else {
ff[0] = -1;
}
while (pos < nff) {
if (needle[pos - 1] == needle[cnd]) {
ff[pos++] = ++cnd;
} else if (cnd > 0) {
cnd = ff[cnd]; /* This is O(1) for the reasons above. */
} else {
ff[pos++] = 0;
}
}
}
from which it is painfully obvious the failure function is O(n), where n is the length of the pattern sought.