I have this code and want to know its time complexity:
int N,M; // let N and M be any two numbers
while(N != M && N > 0 && M > 0){
if(N > M)N -= M;
else M -= N;
}
I don't know how to analyze this, since the values of M and N decrease in unusual ways. Is there a standard way to approach this?
This code is a naive implementation of the Euclidean algorithm. At each iteration, you're subtracting out the smaller number from the bigger one, so you can divide the algorithm into "phases." Each phase consists of subtracting as many copies of the smaller number out of the bigger one until the bigger number drops below the smaller number. (This is connected to a procedure the Ancient Greeks knew about call anythpharesis) A modern version of this could would just be to mod the bigger number by the smaller number, which is known to terminate within O(log min{M, N}) steps (this is the modern Euclidean algorithm) and spend O(1) time on each step, assuming the numbers are represented as integers.
In this case, we know that there will be O(log min{M, N}) phases, but each phase won't take a constant amount of time. Using the geometric intuition behind anythpharesis, it's possible to construct pairs of numbers where it will take a very long time for each individual phase to terminate, so the best bound that I'm aware of would be to say that the runtime is O(N + M).
In short: this code is inefficient compared to a modern implementation, which runs in logarithmic time. It's hard to get a good upper bound on the runtime, but realistically speaking it doesn't matter since you would probably just rewrite the code to be more efficient anyway. :-)
Hope this helps!
Related
I have written a small algorithm, that a changes a digit of a given number to zero. This is done in base 10. And the digits are indexed from least significant to most significant positions.
Example
unsigned int num = 1234
removeDigit(0, &num); // num == 1230
void removeDigit(unsigned int digit, unsigned int *num) {
unsigned int th = 1;
for (; digit > 0; digit--) th *= 10;
while (*num / th % 10 != 0) *num -= th;
}
I was thinking about the runtime of the algorithm. I'm pretty sure its "technically" O(1), but I also see why you could say it is O(n).
The fact that an integer cannot have more than 11 digits, effectively makes it O(1), but you could still argue that for an arbitrary data type with an endless amount of digits, this algorithm is O(n).
What would you say? If someone were to ask me what the runtime of this is, what should I say?
What would you say? If someone were to ask me what the runtime of this is, what should I say?
When you give some time/space asymptotic bounds, typically it is because the algorithm can be used on arbitrarily large inputs.
In this case, your possible inputs are so constrained that providing the absolute maximum number of given operations/cycles/time that will take place worst-case can be a much more useful metric instead, in some domains.
Also consider that, given this is a very simple algorithm, you could simply provide the exact equations for the number of given operations/cycles/time, too; which is way more precise than giving either asymptotic bounds or absolute maximums.
The for loop is executed digit+1 times. The while loop is executed nd times, where nd denotes the value of the designated digit. So in total generality, the asymptotic complexity is
O(digit + nd).
This formula is valid for any number length and any numeration base, but assumes that the arithmetic operations are done in constant time, which is somewhat unrealistic.
In any base, it is clear that nd = O(1), as this number is bounded by the base. So in the worst case,
O(digit).
Now for fixed-length arithmetic, digits is bounded so that digits = O(1). And as a consequence, the worst-case complexity is also
O(1).
Note that there is no incompatibility between the complexity being O(digits + nd) and the worst-case complexity being O(1). This is justified by digits and nb both being O(1).
Other answers and comments have missed the fact that removeDigit(1000000000,p) will do a billion iterations of the first loop... So this is certainly not O(1). Sometimes in this kind of analysis we make use of the fact that machine words are of constant size, but asymptotic analysis ceases to be useful when we treat the largest possible integer value as a constant.
So... unless otherwise specified, the N in O(N) or similar refers to the size of the input. The size of an integer x would usually be considered to be log2 x when it matters, so in terms of that usual convention, your function takes O(2n) time, i.e., exponential.
But when we use asymptotic analysis in real life we try to convey useful information when we make a statement of complexity. In this case the most useful thing you can say is that removeDigit(digit,p) takes O(digit) time.
We want to find all permutations of a string of length n. Then, you are to search an array of fixed constant size, say 3000 and check if the string is in the array.
String arr[3000];
Because we will have !n permutations, we need to do !n searches.
Also, what difference does it make when you check 2 different strings against an element in the array versus just checking 1 string?
What is the time complexity?
My thoughts is that it will take at worst, log2(3000) to go through the array once. Time complexity of that is O(log2(3000)) which is O(1).
Now, you need to go through this array !n times so time complexity is O(!n).
So the binary search reducing the number of searches required should not be the focus when analyzing the time complexity of this algorithm.
My question is, binary search does reduce the number of searches and if you are gonna go through it n! times, shouldn't this be a significant difference?
Any insight to better my understanding is appreciated.
Big O complexity analysis only deals with quantities that are subject to change, by definition. That you get vacuous answers when all your quantities are constant is expected.
The constant factors are relevant when comparing two algorithms of equal Big-O, so your change from 3000 -> log2(3000) is a factor of about 200.
Thus you use the binary search because you are doing more than Big-O analysis. You have also estimated the constant factors, and see an easy 200x speedup
But equally you can have multiple terms in your complexity. You might say:
Let n be the input string length
Let m be the size of arr
Our algorithm is O( n * n! * log(m) ) (n for the string equality, n! for the permutations, log(m) for the binary searching)
It also rather depends on a model of cost. Usually this maps back to some abstract machine, e.g. we assume that operations have a certain cost. E.g. You might compare sorting algorithms by just the count of comparisons, or by just the count of swaps, or by the counts of both comparisons and swaps.
Hey guys i have a quiz tommorow i am facing some problems finding time complexity of recursive programm of selection sort can any one explain how is it n^2. And also one more question that some sorting algorithms loops have n/2 time complexity what is the meaning of /2 sorry for newbie question.
#include <iostream>
using namespace std;
// recursive function to perform selection sort on subarray arr[i..n-1]
void selectionSort(int arr[], int i, int n)
{
// find the minimum element in the unsorted subarray[i..n-1]
// and swap it with arr[i]
int min = i;
for (int j = i + 1; j < n; j++)
{
// if arr[j] element is less, then it is the new minimum
if (arr[j] < arr[min])
min = j; // update index of min element
}
// swap the minimum element in subarray[i..n-1] with arr[i]
swap(arr[min], arr[i]);
if (i + 1 < n)
selectionSort(arr, i + 1, n);
}
Finding time complexity is often described in a way that is not really very helpful. Here is how it works for Selection Sort.
passes
The very first time through the algorithm, you must scan all n elements of the data.
The very next time through (on recursion), you must scan all but one, which is (n-1).
And so on. We can write the number of times we look at/compare an element as:
n + (n-1) + (n-2) + ... + 2 + 1
(You can quibble about that last term, but for simplicity here we just won't care. You'll see why in just a second.)
math identities
This particular series (notice all the additions) is called an “arithmetic progression”. The difference between each term is 1. The first term is n and the last term is 1 (or 2, whatever).
And using some math most people don’t remember (or weren’t taught) in High School, we can rewrite that sum as:
n(n+1)
──────
2
(Yes, again, that last term is actually supposed to be a two, even though I’ve left it at one.)
as n grows arbitrarily large
Big O doesn’t care much about friendly values of n. It cares about what happens when n gets arbitrarily big. We say “n grows towards infinity”.
As it does, we can notice two things happening:
That division by 2 becomes insignificant: ∞/2 → ∞
That addition by 1 (or whatever) is also insignificant: ∞(∞+1) → ∞(∞)
So ultimately, we have infinity, er, n multiplied by itself. The equation simplifies to:
n(n+1)
────── → n²
2
complexity bounds
The worst case behavior is n², so we annotate that as “O(n²)”. But notice that the best case is also n². We annotate that as “Ω(n²)”. Finally, since the best and worst case behaviors are the same, we have a very nice tight bound on the behavior of the function. We annotate this as “Θ(n²)”.
Hence, selection sort has Θ(n²) complexity.
holy quiznak! I’m supposed to do this myself!!?
Yes, unfortunately, figuring out complexity is one of those things that people treat as if it were really easy — even when they don’t understand it well themselves. It takes some familiarity with math and some practice. The typical response is as in the links provided you above: ‘look at these common cases and choose the one that most closely matches’. I personally find that less satisfyingly instructive. It is also one of those things that university professors expect you to pick up right away.
My advice: don’t worry too hard. Do your best to understand the underlying principles. What we are doing is finding a function (like y=x²) that represents an algorithm’s behavior whenever its input is really large (n → ∞).
Being able to follow through the paths that the algorithm takes and recognizing when those paths are expensive is a more important ability than being able to come up with a mathematician’s answer. If you really need it you can always find a mathematician to help, either at the university or on the internet if you ask in the right places.
And, of course, feel free to take time to try to understand better and practice. Being able to do it right is a valuable skill, regardless of whether others see it.
:O)
I'm wondering whether the big-o complexity of the following code should be o(1) or o(n).
for(int i=0;i<n && n<100;i++){sum++;}
Here's what i think:
since n is limited to be lower than 100, worst case will be O(99) + O(98) + ... + O(1) = 99 * O(1) = O(1)
However, by intuition, the code is somehow O(n) due to the loop.
Would really appreciate it if someone can advise me on this.
Thank you!
Intuitively it is O(1) because as n increases the runtime does not increase after a certain point. However, this is an edge case, as were n bounded by a much higher number, say the maximum value of an int, it would seem to be no different than if n was not bounded at all. However, when considering runtime using complexity theory we usually ignore things like that maximum size of an int.
Another way to think of this is that the number of iterations grows linearly with n for n in (0,100) and is constant otherwise. When considering n can be any value, however, the algorithm is definitely O(1)
This is all assuming each iteration of the loop takes constant time.
For more information look up Liouville's theorem.
I have a collection of N positive integers, each bounded by a (relatively small) constant C. I want to find a subset of these numbers with the smallest sum greater than (or equal to) a value K.
The numbers involved aren't terribly large (<100), but I need good performance even in the worst case. I thought maybe I could adapt Pisinger's dynamic programming algorithm to the task; it runs in O(NC) time, and I happen to meet the requirements of bounded, positive numbers.
[Edit]: The numbers are not sorted and there may be duplicates.
However, I don't understand the algorithm well enough to do this myself. In fact, I'm not even certain if the assumptions it is based on still hold...
-Is it possible to adapt this algorithm to my needs?
-Or is there another linear algorithm I could use that is similarly efficient?
-Could anyone provide pseudocode or a detailed explanation?
Thanks.
Link to the Subset-Sum code I was investigating:
Fast solution to Subset sum algorithm by Pisinger
(Apologies if this is poorly worded/formatted/etc. I'm still new to StackOverflow...)
Pisinger's algorithm gives you the largest sum less than or equal to the capacity of the knapsack. To solve your problem, use Pisinger to figure out what not to put in the subset. Formally, let the items be w_1, ..., w_n and the minimum be K. Give w_1, ..., w_n and w_1 + ... + w_n - K to Pisinger, then take every item that Pisinger does not.
Well one solution is to:
T = {0}
for x in V
for t in T
T.insert(x+t)
for i in K to max(T)
if (T.contains(i))
return i
fail
This gives you the size of the subset, but you can adapt to output the members.
The maximum size of T is O(N) (because of C bound), so the running time is O(N^2) and the space is O(N). You can use a bit array of length NC as the backing store of T.