I have a question about quick sort algorithm. I implement quick sort algorithm and play it.
The elements in initial unsorted array are random numbers chosen from certain range.
I find the range of random number effects the running time. For example, the running time for 1, 000, 000 random number chosen from the range (1 - 2000) takes 40 seconds. While it takes 9 seconds if the 1,000,000 number chosen from the range (1 - 10,000).
But I do not know how to explain it. In class, we talk about the pivot value can effect the depth of recursion tree.
For my implementation, the last value of the array is chosen as pivot value. I do not use randomized scheme to select pivot value.
int partition( vector<int> &vec, int p, int r) {
int x = vec[r];
int i = (p-1);
int j = p;
while(1) {
if (vec[j] <= x){
i = (i+1);
int temp = vec[j];
vec[j] = vec[i];
vec[i] = temp;
}
j=j+1;
if (j==r)
break;
}
int temp = vec[i+1];
vec[i+1] = vec[r];
vec[r] = temp;
return i+1;
}
void quicksort ( vector<int> &vec, int p, int r) {
if (p<r){
int q = partition(vec, p, r);
quicksort(vec, p, q-1);
quicksort(vec, q+1, r);
}
}
void random_generator(int num, int * array) {
srand((unsigned)time(0));
int random_integer;
for(int index=0; index< num; index++){
random_integer = (rand()%10000)+1;
*(array+index) = random_integer;
}
}
int main() {
int array_size = 1000000;
int input_array[array_size];
random_generator(array_size, input_array);
vector<int> vec(input_array, input_array+array_size);
clock_t t1, t2;
t1 = clock();
quicksort(vec, 0, (array_size - 1)); // call quick sort
int length = vec.size();
t2 = clock();
float diff = ((float)t2 - (float)t1);
cout << diff << endl;
cout << diff/CLOCKS_PER_SEC <<endl;
}
Most likely it's not performing well because quicksort doesn't handle lots of duplicates very well and may still result in swapping them (order of key-equal elements isn't guaranteed to be preserved). You'll notice that the number of duplicates per number is 100 for 10000 or 500 for 2000, while the time factor is also approximately a factor of 5.
Have you averaged the runtimes over at least 5-10 runs at each size to give it a fair shot of getting a good starting pivot?
As a comparison have you checked to see how std::sort and std::stable_sort also perform on the same data sets?
Finally for this distribution of data (unless this is a quicksort exercise) I think counting sort would be much better - 40K memory to store the counts and it runs in O(n).
It probably has to do with how well sorted the input is. Quicksort is O(n logn) if the input is reasonably random. If it's in reverse order, performance can degrade to O(n^2). You're probably getting closer to the O(n^2) behavior with the smaller data range.
Late answer - the effect of duplicates depends on the partition scheme. The example code in the question is a variation of Lomuto partition scheme, which takes more time as the number of duplicates increases, due to the partitioning getting worse. In the case of all equal elements, Lomuto only reduces the size by 1 element with each level of recursion.
If instead Hoare partition scheme was used (with middle value as pivot), it generally takes less time as the number of duplicates increases. Hoare will needlessly swap values equal to the pivot, due to duplicates, but the partitioning will approach the ideal case of splitting an array in nearly equally sized parts. The swap overhead is somewhat masked by memory cache. Link to Wiki example of Hoare partition scheme:
https://en.wikipedia.org/wiki/Quicksort#Hoare_partition_scheme
Related
I know there were similar questions, but not of such specificity
Input: n-elements array with unsorted emelents with values from 1 to (n-1).
one of the values is duplicate (eg. n=5, tab[n] = {3,4,2,4,1}.
Task: find duplicate with best Complexity.
I wrote alghoritm:
int tab[] = { 1,6,7,8,9,4,2,2,3,5 };
int arrSize = sizeof(tab)/sizeof(tab[0]);
for (int i = 0; i < arrSize; i++) {
tab[tab[i] % arrSize] = tab[tab[i] % arrSize] + arrSize;
}
for (int i = 0; i < arrSize; i++) {
if (tab[i] >= arrSize * 2) {
std::cout << i;
break;
}
but i dont think it is with best possible Complexity.
Do You know better method/alghoritm? I can use any c++ library, but i don't have any idea.
Is it possible to get better complexity than O(n) ?
In terms of big-O notation, you cannot beat O(n) (same as your solution here). But you can have better constants and simpler algorithm, by using the property that the sum of elements 1,...,n-1 is well known.
int sum = 0;
for (int x : tab) {
sum += x;
}
duplicate = sum - ((n*(n-1)/2))
The constants here will be significntly better - as each array index is accessed exactly once, which is much more cache friendly and efficient to modern architectures.
(Note, this solution does ignore integer overflow, but it's easy to account for it by using 2x more bits in sum than there are in the array's elements).
Adding the classic answer because it was requested. It is based on the idea that if you xor a number with itself you get 0. So if you xor all numbers from 1 to n - 1 and all numbers in the array you will end up with the duplicate.
int duplicate = arr[0];
for (int i = 1; i < arr.length; i++) {
duplicate = duplicate ^ arr[i] ^ i;
}
Don't focus too much on asymptotic complexity. In practice the fastest algorithm is not necessarily the one with lowest asymtotic complexity. That is because constants are not taken into account: O( huge_constant * N) == O(N) == O( tiny_constant * N).
You cannot inspect N values in less than O(N). Though you do not need a full pass through the array. You can stop once you found the duplicate:
#include <iostream>
#include <vector>
int main() {
std::vector<int> vals{1,2,4,6,5,3,2};
std::vector<bool> present(vals.size());
for (const auto& e : vals) {
if (present[e]) {
std::cout << "duplicate is " << e << "\n";
break;
}
present[e] = true;
}
}
In the "lucky case" the duplicate is at index 2. In the worst case the whole vector has to be scanned. On average it is again O(N) time complexity. Further it uses O(N) additional memory while yours is using no additional memory. Again: Complexity alone cannot tell you which algorithm is faster (especially not for a fixed input size).
No matter how hard you try, you won't beat O(N), because no matter in what order you traverse the elements (and remember already found elements), the best and worst case are always the same: Either the duplicate is in the first two elements you inspect or it's the last, and on average it will be O(N).
So i have an array which has even and odds numbers in it.
I have to sort it with odd numbers first and then even numbers.
Here is my approach to it:
int key,val;
int odd = 0;
int index = 0;
for(int i=0;i<max;i++)
{
if(arr[i]%2!=0)
{
int temp = arr[index];
arr[index] = arr[i];
arr[i] = temp;
index++;
odd++;
}
}
First I separate even and odd numbers then I apply sorting to it.
For sorting I have this code:
for (int i=1; i<max;i++)
{
key=arr[i];
if(i<odd)
{
val = 0;
}
if(i>=odd)
{
val = odd;
}
for(int j=i; j>val && key < arr[j-1]; j--)
{
arr[j] = arr[j-1];
arr[j-1] = key;
}
}
The problem i am facing is this i cant find the complexity of the above sorting code.
Like insertion sort is applied to first odd numbers.
When they are done I skip that part and start sorting the even numbers.
Here is my approach for sorting if i have sorted array e.g: 3 5 7 9 2 6 10 12
complexity table
How all this works?
in first for loop i traverse through the loop and put all the odd numbers before the even numbers.
But since it doesnt sort them.
in next for loop which has insertion sort. I basically did is only like sorted only odd numbers first in array using if statement. Then when i == odd the nested for loop then doesnt go through all the odd numbers instead it only counts the even numbers and then sorts them.
I'm assuming you know the complexity of your partitioning (let's say A) and sorting algorithms (let's call this one B).
You first partition your n element array, then sort m element, and finally sort n - m elements. So the total complexity would be:
A(n) + B(m) + B(n - m)
Depending on what A and B actually are you should probably be able to simplify that further.
Edit: Btw, unless the goal of your code is to try and implement partitioning/sorting algorithms, I believe this is much clearer:
#include <algorithm>
#include <iterator>
template <class T>
void partition_and_sort (T & values) {
auto isOdd = [](auto const & e) { return e % 2 == 1; };
auto middle = std::partition(std::begin(values), std::end(values), isOdd);
std::sort(std::begin(values), middle);
std::sort(middle, std::end(values));
}
Complexity in this case is O(n) + 2 * O(n * log(n)) = O(n * log(n)).
Edit 2: I wrongly assumed std::partition keeps the relative order of elements. That's not the case. Fixed the code example.
I've understood how the partitioning part is done in the quicksort algorithm, but I'm having trouble understanding the quicksort recursive function. Could someone please explain to me step by step how it works? I'm pasting here the C++ code.
using namespace std;
void swap(int* a, int* b) {
int temp = *a;
*a = *b;
*b = temp;
}
int partition(int myArray[], int low, int high) {
int i = (low - 1);
int pivot = myArray[high];
for (int j = low; j <= high - 1; j++) {
if (myArray[j] < pivot) {
i++;
swap(&myArray[i], &myArray[j]);
}
}
swap(&myArray[i + 1], &myArray[high]);
return (i + 1);
}
void quickSort(int myArray[], int low, int high) {
if (low < high) {
int pi = partition(myArray, low, high);
quickSort(myArray, low, pi - 1);
quickSort(myArray, pi + 1, high);
}
}
void cinArray(int myArray[], int n) {
for (int p = 0; p < n; p++) {
cin >> myArray[p];
}
}
void print(int myArray[], int n) {
for (int z = 0; z < n; z++) {
cout << myArray[z] << "\t";
}
}
int main() {
int myArray[10];
int size = sizeof(myArray) / sizeof(myArray[0]);
cout << "Write 10 numbers: ";
cinArray(myArray, size);
quickSort(myArray, 0, size - 1);
print(myArray, size);
return 0;
}
My logic by far (step by step) is the following:
if (low < high) will always be true. The first time, the
low (0) and high (9) values are taken from the int main.
Pi will be equal to the returned value of the partition funcion
(i+1), and I suppose that in order to return that value the function
has to run first.
We call the quicksort function giving new arguments for the function twice. Once for
the values before and once for the values after i+1 from the original partitioning. I want to focus on what happens on the first one, the one that has the values before i+1.
Function starts again, the if statement is true (always is), pi calls the function partition and returns i+1, pi is equal to i+1. And what if at this point the values are still not sorted? I suppose the quicksort function restarts again (It feels like a loop). But since the IF statement will always be true, when will this loop situation stop?
Also, assuming my logic at point 4 is correct, how is the code run the first time? Does it start with the first quickSort(myArray, low, pi - 1); function call and loops until something stops it and then does the same for the second call quickSort(myArray, pi + 1, high);? Or does it partition before i+1 then after i+1 and restarts the function?
I know it's a basic question, but I'm really having a hard time wraping my head around this algorithm.
if (low < high) will always be true.
Not correct. It will be true on the first call, but QuickSort calls itself recursive with progressively smaller intervals between low and high. This if is why the algorithm eventually terminates - you ask about that below.
Pi will be equal to the returned value of the partition funcion (i+1)
Right. pi is short for pivot index, i.e. the location where the chosen pivot ended up after partitioning.
And what if at this point the values are still not sorted?
After partitioning, you know that no value in the left partition is greater than the pivot value and that no value in the right partition is less than the pivot value. That's all you know, and all the algorithm needs to know in order to eventually succeed. Each partition is recursively partitioned until it only has one element in it.
when will this loop situation stop?
See my first point.
The partition function places the pivot element in place and returns the index to the pivot element. The next two calls exclude the element, and in the worst case, one call will be for zero elements, the other for n-1 elements, and continuing with the worst case, the size of what is passed on only decreases by 1 element for each level of recursion, with time complexity O(n^2). The best case is if the pivot ends up in the middle, with an even split on each level of recursion, or near even split for time complexity O(n log(n)).
//Time sorting O(nlogn) + binary search for N items logN = 2NLogN =
//Time: O(NLogN).
//space - O(1).
bool TwoSum::TwoSumSortAndBinarySearch(int* arr, int size, int sum)
{
sort(arr, arr + size);
for (int i = 0; i < size; i++)
{
if (binary_search(arr + i + 1, arr + size, sum - arr[i]))
return true;
}
return false;
}
//Time: O(N) as time complexity of Add and Search in hashset/unordered_set is O(1).
//Space: O(N)
bool TwoSum::TwoSumHashSet(int* arr, int size, int sum)
{
unordered_set<int> hash;
for (int i = 0; i < size; i++)
{
if (hash.find(sum - arr[i]) != hash.end())
return true;
hash.insert(arr[i]);
}
return false;
}
int* TwoSum::Testcase(int size)
{
int* in = new int[size];
for (int i = 0; i < size; i++)
{
in[i] = rand() % (size + 1);//random number b/w 0 to N.
}
return in;
}
int main()
{
int size = 5000000;
int* in = TwoSum::Testcase(size);
auto start = std::chrono::system_clock::now();//clock start
bool output = TwoSum::TwoSumHashSet(in, size, INT_MAX);
auto end = std::chrono::system_clock::now();//clock end
std::chrono::duration<double> elapsed_seconds = end - start;
cout << "elapsed time: " << elapsed_seconds.count() << "s\n";
}
I measured the performance of the above two methods, where I would like to find the TwoSum problem.
In the First approach, I am sorting the array then using binary search.
Time: O(NLogN).
space - O(1).
In the second approach, unordered_set is used whose complexity is constant on average, worst case linear in the size of the container.
//Time: O(N) as time complexity of Add and Search in hashset/unordered_set is O(1).
//Space: O(N)
Here are the three runs time taken by these two methods
TwoSumSortAndBinarySearch---------------TwoSumHashSet
8.05---------------------------------------15.15
7.76---------------------------------------14.47
7.74---------------------------------------14.28
So, it is clear that TwoSumSortAndBinarySearch performs definitely better than unordered_Set.
Which approach is more preferable and suggested in real scenario and why?
This is because computational complexity doesn’t take into account the behavior of multi-level memory system present in every modern computer. And it is precisely because you measure that behavior via proxy using time (!!), that your measurement is not “like” theoretical computational complexity. Computational complexity predicts execution times only in very well controlled situations, when the code is optimal for the platform. If you want to measure complexity, you can’t measure time. Measure operation counts. It will agree with theory then.
In my limited experience, it is rather rare that computational complexity theory would predict runtimes on reasonably sized data sets, when the behavior is neither exponential nor cubic (or higher terms). Cache access patterns and utilization of architectural parallelism are major predictors of performance, before computational complexity comes into play.
Could anyone please tell me how this code sorts the array? i don't get it! and how is this code reducing the complexity of a regular insertion sort?
// Function to sort an array a[] of size 'n'
void insertionSort(int a[], int n)
{
int i, loc, j, k, selected;
for (i = 1; i < n; ++i)
{
j = i - 1;
selected = a[i];
// find location where selected sould be inseretd
loc = binarySearch(a, selected, 0, j);
// Move all elements after location to create space
while (j >= loc)
{
a[j+1] = a[j];
j--;
}
a[j+1] = selected;
}
}
This code uses the fact that the portion of the array from zero, inclusive, to i, exclusive, is already sorted. That's why it can run binarySearch for the insertion location of a[i], rather than searching for it linearly.
This clever trick does not change the asymptotic complexity of the algorithm, because the part where elements from loc to i are moved remains linear. In the worst case (which happens when the array is sorted in reverse) each of the N insertion steps will make i moves, for a total of N(N-1)/2 moves.
The only improvement that this algorithm has over the classic insertion sort is the number of comparisons. If comparisons of objects being sorted are computationally expensive, this algorithm can significantly reduce the constant factor.