I recently came across an article that claimed that it can find all primes less than n in O(n) using an efficient Sieve Of Eratosthenes. However I am unable to see how it is O(n).
https://www.geeksforgeeks.org/sieve-eratosthenes-0n-time-complexity/
Could anyone please help with that?
The normal Sieve of Eratosthenes is O(n log log n).
Paul Pritchard has done some work on sieves similar to the Sieve of Eratosthenes that run in O(n) and even in O(n / log log n). They are tricky to implement, and despite improved theoretical time complexity, the bookkeeping involved in running the sieves makes them slower than the normal Sieve of Eratosthenes.
I discuss a simple version of Pritchard's sieve at my blog.
It is a version of the Gries and Misra (1978) sieve, which is an O(n) sieve. A better description can be found here:
(external link) Sieve of Eratosthenes Having Linear Time Complexity.
For a more theoretical look at this type of sieve, from an expert in the field, see Pritchard's paper:
(external link) Linear Prime-Number Sieves: A Family Tree (1987, PDF).
Pritchard is well known for his sub-linear sieve algorithm and paper as well as other early contributions.
The version at GfG uses a lot of extra memory. The version at CP uses a little less. Both are huge compared to typical byte or bit implementations of the SoE. At 10^9, it is over 60x more memory used than a simple bit array monolithic SoE, and also half the speed, even when using uint32_t types.
So in practice it is slower than a simple 4-line monolithic SoE, which is usually where we start before getting into the interesting optimizations (segmented sieves, wheels, etc.). If you actually want the factor array, then that's useful. It's also useful for learning and experimentation, though the GfG article doesn't actually do much other than give the code. The CP page does go over a bit of the history and some memory/speed analysis.
The algorithm at your link is a variation of Algorithm 3.3 in Paul Pritchard's paper "Linear Prime-Number Sieves: a Family Tree". The reason the algorithm is linear, i.e. O(n), is that each composite is removed once, because a composite c has a unique form p*f where p=lpf(c), and it is removed when the outer loop variable if f, and the inner loop variable j is such that p[j]=p.
Incidentally, the code is inelegant. There is no need for two arrays; SPF suffices. Also, the first test (on j) in the inner loop is unnecessary.
Many other linear sieves are presented in Pritchard's paper, one of which is due to Gries and Misra, which is an entirely different algorithm. The algorithm at your link is often mis-attributed to Gries and Misra.
Related
Until now, I've been using sieve of Eratosthenes for generating all 'n' prime numbers.
However, I was wondering does there exist a better algorithm, or can we improve an existing one which performs better ??
For sufficiently large N (e.g. more than a million or so), the best algorithm is to use an approximation (e.g. Logarithmic Integral or Riemann's R function), then use a fast prime count method such as LMO, then sieve the small remainder. This is many orders of magnitude faster than sieving.
See https://math.stackexchange.com/questions/507178/most-efficient-algorithm-for-nth-prime-deterministic-and-probabilistic
There are at least two open source implementations.
https://metacpan.org/pod/ntheory
https://github.com/kimwalisch/primecount
The latter has progressed past the first, and is also multithreaded.
Add: Will Ness has also pointed out a nice post from Daniel Fischer that provides a walkthrough of different ways to solve this: Calculating and printing the nth prime number
Does anyone know both the expected running times and worst case running times for different implementations of std::nth_element? I use this algorithm nearly every day.
I'm specifically interested in the STL versions shipping with the recent Microsoft Compilers, but any information on this topic is helpful.
Please note that this is not a duplicate of this question. I understand which algorithms exist, but I'm interested in which implementations use which algorithms.
For background, there are well-known algorithms to do this. One is O(n) average case and O(n log n) worst case, one is O(n) worst case but slow in practice (median of medians).
Also note that there is talk of interesting implementation strategies to get worst-case O(n) running times that are fast in practice. The standard says that this must be at worse O(n) average time.
The expected running time is O(N)
The worst case running time for most implemententations is O(N * N) because most implementations use QuickSelect and it could be that QuickSelect runs into bad partitions.
That is true for Microsoft VS2008, VS2010 & VS2012.
Now with the new ISO C++ 2011 standard, the complexity for std::sort has been tightened up - it is guaranteed to be O(N * log N) and has no worse case as David Musser's IntroSort is used: - use QuickSort and if parts of the array experience bad partitioning, swap to heapsort.
Ideally exactly the same should apply std::nth_element but the ISO C++ 2011 standard has not tightened up the complexity requirements. So std::nth_element could be O(N * N) in the worst case. This could be because in David Musser's original paper (see here) he did not mention what algorithm should be swapped to if QuickSelect goes bad.
In the worst case, the median-of-medians using groups of 5 could be used (I have seen a paper the recommended groups of 7 but cannot find it). So a quality implementation of std::nth_element could use QuickSelect and swap to median-of-medians if partitioning goes bad. This would guarantee O(N) behaviour. QuickSelect can be improved by using sampling making the worst case unlikely but not impossible.
The implementation in GCC 4.7 uses introspective selection by David Musser (here you have his paper giving details on introsort and introselect). According to those documents the worst-case execution time is O(n).
cppreference says, first it sorts and then finds the nth element, but by this way average should be O(n log n) (by comparison based sorting algorithms), but they wrote average is O(n), seems incorrect except using sorting like radix sort, ... but because it has generic comparison based input, seems it's impossible to use radix sort or any other sort which is not comparison based. anyway, using fast sorting algorithms is better than using normal selection algorithm in practice (both memory and average time).
I want to solve the min-cut problem on a lot of small DAGS (8-12 nodes,20-60 edges) very quickly. It looks like the best solution is to solve the max-flow and deduce a cut from that. There are quite a few max-flow algorithms with both theoretical and empirical timing comparisons available, but these all assume what's interesting is performance as the graphs get larger and larger. It's also often mentioned that set-up times for complicated data structures used can be quite big. So given a careful, optimized implementation (probably in C++) which algorithm turns out to be fastest for initialising and running on small graphs? (My naive assumption is that Edmonds-Karp is probably as simple in terms of data-structures so will beat more complicated algorithms, but that's just a guesstimate.)
Does anyone know both the expected running times and worst case running times for different implementations of std::nth_element? I use this algorithm nearly every day.
I'm specifically interested in the STL versions shipping with the recent Microsoft Compilers, but any information on this topic is helpful.
Please note that this is not a duplicate of this question. I understand which algorithms exist, but I'm interested in which implementations use which algorithms.
For background, there are well-known algorithms to do this. One is O(n) average case and O(n log n) worst case, one is O(n) worst case but slow in practice (median of medians).
Also note that there is talk of interesting implementation strategies to get worst-case O(n) running times that are fast in practice. The standard says that this must be at worse O(n) average time.
The expected running time is O(N)
The worst case running time for most implemententations is O(N * N) because most implementations use QuickSelect and it could be that QuickSelect runs into bad partitions.
That is true for Microsoft VS2008, VS2010 & VS2012.
Now with the new ISO C++ 2011 standard, the complexity for std::sort has been tightened up - it is guaranteed to be O(N * log N) and has no worse case as David Musser's IntroSort is used: - use QuickSort and if parts of the array experience bad partitioning, swap to heapsort.
Ideally exactly the same should apply std::nth_element but the ISO C++ 2011 standard has not tightened up the complexity requirements. So std::nth_element could be O(N * N) in the worst case. This could be because in David Musser's original paper (see here) he did not mention what algorithm should be swapped to if QuickSelect goes bad.
In the worst case, the median-of-medians using groups of 5 could be used (I have seen a paper the recommended groups of 7 but cannot find it). So a quality implementation of std::nth_element could use QuickSelect and swap to median-of-medians if partitioning goes bad. This would guarantee O(N) behaviour. QuickSelect can be improved by using sampling making the worst case unlikely but not impossible.
The implementation in GCC 4.7 uses introspective selection by David Musser (here you have his paper giving details on introsort and introselect). According to those documents the worst-case execution time is O(n).
cppreference says, first it sorts and then finds the nth element, but by this way average should be O(n log n) (by comparison based sorting algorithms), but they wrote average is O(n), seems incorrect except using sorting like radix sort, ... but because it has generic comparison based input, seems it's impossible to use radix sort or any other sort which is not comparison based. anyway, using fast sorting algorithms is better than using normal selection algorithm in practice (both memory and average time).
I have found a way that improves (as far as I have tested) upon the quicksort algorithm beyond what has already been done. I am working on testing it and then I want to get the word out about it. However, I would appreciate some help with some things. So here are my questions. All of my code is in C++ by the way.
One of the sorts I have been comparing to my quicksort is the std::sort from the C++ Standard Library. However, it appears to be extremely slow. I am only sorting arrays of ints and longs, but it appears to be around 8-10 times slower than both my quicksort and a standard quicksort by Bentley and McIlroy (and maybe Sedgewick). Does anyone have any ideas as to why it is so slow? The code I use for the sort is just
std::sort(a,a+numelem);
where a is the array of longs or ints and numelem is the number of elements in the array. The numbers are very random, and I have tried different sizes as well as different amounts of repeated elements. I also tried qsort, but it is even worse as I expected.
Edit: Ignore this first question - it's been resolved.
I would like to find more good quicksort implementations to compare with my quicksort. So far I have a Bentley-McIlroy one and I have also compared with the first published version of Vladimir Yaroslavskiy's dual-pivot quicksort. In addition, I plan on porting timsort (which is a merge sort I believe) and the optimized dual-pivot quicksort from the jdk 7 source. What other good quicksorts implementations do you know about? If they aren't in C or C++ that might be okay because I am pretty good at porting, but I would prefer C or C++ ones if you know of them.
How would you recommend getting out the word about my additions to the quicksort? So far my quicksort seems to be significantly faster than all other quicksorts that I've tested it against. The main source of its speed is that it handles repeated elements much more efficiently than other methods that I've found. It almost completely eradicates worst case behavior without adding much time in checking for repeated elements. I posted about it on the Java forums, but got no response. I also tried writing to Jon Bentley because he was working with Vladimir on his dual-pivot quicksort and got no response (though I wasn't terribly surprised by this). Should I write a paper about it and put it on arxiv.org? Should I post in some forums? Are there some mailing lists to which I should post? I have been working on this for some time now and my method is legit. I do have some experience with publishing research because I am a PhD candidate in computational physics. Should I try approaching someone in the Computer Science department of my university? By the way, I have also developed a different dual-pivot quicksort, but it isn't better than my single-pivot quicksort (though it is better than Vladimir's dual-pivot quicksort with some datasets).
I really appreciate your help. I just want to add what I can to the computing world. I'm not interested in patenting this or any absurd thing like that.
If you have confidence in your work, definitely try to discuss it with someone knowledgeable at your university as soon as possible. It's not enough to show that your code runs faster than another procedure on your machine. You have to mathematically prove whatever performance gain you claim to have achieved through analysis of your algorithm. I'd say the first thing to do is make sure both algorithms you are comparing are implemented and compiled optimally - you may just be fooling yourself here. The likelihood of an individual achieving such a marked improvement upon such an important sorting method without already having thorough knowledge of its accepted variants just seems minuscule. However, don't let me discourage you. It should be interesting anyway. Would you be willing to post the code here?
...Also, since quicksort is especially vulnerable to worst-case scenarios, the tests you choose to run may have a huge effect, as will the choice of pivots. In general, I would say that any data set with a large number of equivalent elements or one that is already highly sorted is never a good choice for quicksort - and there are already well-known ways of combating that situation, and better alternative sorting methods.
If you have truly made a breakthrough and have the math to prove it, you should try to get it published in the Journal of the ACM. It's definitely one of the more prestigious journals for computer science.
The second best would be one of the IEEE journals such as Transactions on Software Engineering.