I read from many sources that Dijkstra's Shortest Path also will run in O(V^2) complexity if using a naive way to get the min element (linear search). However, it can be optimised to O(VLogV) if priority queue is used as this data structure will return min element in O(1) time but takes O(LogV) time to restore the heap property after deleting the min element.
I have implemented Dijkstra's algo in the following code for the UVA problem at this link: https://uva.onlinejudge.org/index.php?option=com_onlinejudge&Itemid=8&category=16&page=show_problem&problem=1927:
#include<iostream>
#include<vector>
#include <climits>
#include <cmath>
#include <set>
using namespace std;
#define rep(a,b,c) for(int c=a;c<b;c++)
typedef std::vector<int> VI;
typedef std::vector<VI> VVI;
struct cmp {
bool operator()(const pair<int,int> &a,const pair<int,int> &b) const {
return a.second < b.second;
}
};
void sp(VVI &graph,set<pair<int,int>,cmp> &minv,VI &ans,int S,int T) {
int e = -1;
minv.insert(pair<int,int>(S,0));
rep(0,graph.size() && !minv.empty() && minv.begin()->first != T,s) {
e = minv.begin()->first;
minv.erase(minv.begin());
int nb = 0;
rep(0,graph[e].size(),d) {
nb = d;
if(graph[e][d] != INT_MAX && ans[e] + graph[e][d] < ans[d]) {
set<pair<int,int>,cmp>::iterator si = minv.find(pair<int,int>(d,ans[d]));
if(si != minv.end())
minv.erase(*si);
ans[d] = ans[e] + graph[e][d];
minv.insert(pair<int,int>(d,ans[d]));
}
}
}
}
int main(void) {
int cc = 0,N = 0,M = 0,S = -1,T = -1,A=-1,B=-1,W=-1;
VVI graph;
VI ans;
set<pair<int,int>,cmp> minv;
cin >> cc;
rep(0,cc,i) {
cin >> N >> M >> S >> T;
graph.clear();
ans.clear();
graph.assign(N,VI());
ans.assign(graph.size(),INT_MAX);
minv.clear();
rep(0,N,j) {
graph[j].assign(N,INT_MAX);
}
ans[S] = 0;
graph[S][S] = 0;
rep(0,M,j) {
cin >> A >> B >> W;
graph[A][B] = min(W,graph[A][B]);
graph[B][A] = min(W,graph[B][A]);
}
sp(graph,minv,ans,S,T);
cout << "Case #" << i + 1 << ": ";
if(ans[T] != INT_MAX)
cout << ans[T] << endl;
else
cout << "unreachable" << endl;
}
}
Based on my analysis, my algorithm has a O(VLogV) complexity. The STL std::set is implemented as a binary search tree. Furthermore, the set is sorted too.
Hence getting the minimum element from it is O(1), insertion and deletion is O(LogV) each. However, I am still getting a TLE from this problem which should be solvable in O(VLogV) based on the given time limit.
This led me to think deeper. What if all nodes were interconnected such that each vertex V has V-1 neighbours? Won't it make Dijkstra's algorithm run in O(V^2) since each vertex has to look at V-1,V-2,V-3... nodes every round?
On second thoughts, I might have misinterpreted the worst case complexity. Could someone please advise me on the following issues:
How is Dijkstra's algo O(VLogV) especially given the above counterexample?
How could I optimise my code so that it could achieve O(VLogV) complexity (or better)?
Edit:
I realised that my program does not run in O(ElogV) after all. The bottleneck is caused by my input processing which runs in O(V^2). The dijkstra part indeed runs in (ElogV).
In order to understand the time complexity of Dijkstra's algorithm, we need to study the operations that are performed on the data structure that is used to implement the Frontier set (i.e. the data structure used for minv in your algorithm):
Insert
Update
Find/Delete minimum
There are O(|V|) inserts, O(|E|) updates, O(|V|) Find/Delete Minimums in total that occur on the data structure for the entire duration of the algorithm.
Originally Dijkstra implemented the Frontier set using an unsorted array. Thus it was O(1) for Insert and Update, but O(|V|) for Find/Delete minimum, resulting in O(|E| + |V|^2), but since |E| < |V|^2, you have O(|V|^2).
If a binary min-heap is used to implement the Frontier set, you have log(|v|) for all operations, resulting in O(|E|log|V| + |V|log|V|), but since it is reasonable to assume |E| > |V|, you have O(|E|log|V|).
Then came the Fibonacci heap, where you have O(1) amortized time for Insert/Update/Find minimum, but O(log|V|) amortized time for Delete minimum, giving you the currently best known time bound of O(|E| + |V|log|V|) for Dijkstra's algorithm.
Finally, an algorithm for solving the Single Source Shortest Paths problem in O(|V|log|V|) worst case time complexity is not possible if (|V|log|V| < |E|), since the problem has the trivial lower time bound of O(|E| + |V|) i.e. you need to inspect each vertex and edge at least once to solve the problem.
Improving Dijkstra by using a BST or heap will lead to time complexities like O(|E|log|V|) or O(|E|+|V|log|V|), see Dijkstra's running time. Each edge has to be checked at some point.
Related
I am solving a problem on LeetCode:
Given an unsorted array of integers nums, return the length of the longest consecutive elements sequence. You must write an algorithm that runs in O(n) time. So for nums = [100,4,200,1,3,2], the output is 4.
The Union Find solution to solve this is as below:
class Solution {
public:
vector<int> parent, sz;
int find(int i) {
if(parent[i]==i) return i;
return parent[i]=find(parent[i]);
}
void merge(int i, int j) {
int p1=find(i);
int p2=find(j);
if(p1==p2) return;
if(sz[p1]>sz[p2]) {
sz[p1]+=sz[p2];
parent[p2]=p1;
} else {
sz[p2]+=sz[p1];
parent[p1]=p2;
}
}
int longestConsecutive(vector<int>& nums) {
sz.resize(nums.size(),1);
parent.resize(nums.size(),0);
iota(begin(parent),end(parent),0);
unordered_map<int, int> m;
for(int i=0; i<nums.size(); i++) {
int n=nums[i];
if(m.count(n)) continue;
if(m.count(n-1)) merge(i,m[n-1]);
if(m.count(n+1)) merge(i,m[n+1]);
m[n]=i;
}
int res=0;
for(int i=0; i<parent.size(); i++) {
if(parent[i]==i && sz[i]>res) {
res=sz[i];
}
}
return res;
}
};
This gets accepted by the OJ (Runtime: 80 ms, faster than 76.03% of C++ online submissions for Longest Consecutive Sequence), but is this really O(n), as claimed by many answers, such as this one? My understanding is that Union Find is an O(NlogN) algorithm.
Are they right? Or, am I missing something?
They are right. A properly implemented Union Find with path compression and union by rank has linear run time complexity as a whole, while any individual operation has an amortized constant run time complexity. The exact complexity of m operations of any type is O(m * alpha(n)) where alpha is the inverse Ackerman function. For any possible n in the physical world, the inverse Ackerman function doesn't exceed 4. Thus, we can state that individual operations are constant and algorithm as a whole linear.
The key part for path compression in your code is here:
return parent[i]=find(parent[i])
vs the following that doesn't employ path compression:
return find(parent[i])
What this part of the code does is that it flattens the structure of the nodes in the hierarchy and links each node directly to the final root. Only in the first run of find will you traverse the whole structure. The next time you'll get a direct hit since you set the node's parent to its ultimate root. Notice that the second code snippet works perfectly fine, but it just does redundant work when you are not interested in the path itself and only in the final root.
Union by rank is evident here:
if(sz[p1]>sz[p2]) {...
It makes sure that the node with more children becomes the root of the node with less children. Therefore, less nodes need to be reassigned a new parent, hence less work.
Note: The above was updated and corrected based on feedback from #Matt-Timmermans and #kcsquared.
I have a very interesting problem, I've started developing a genetic learning algorithm and have succeeded on doing so. its a simple GA designed to find a phrase by randomly selecting characters to store into strings and using the standard selection and mutation methods to progress until the it has the final answer, and sometimes this works perfectly.
However, sometimes there's one character incorrect.
I think this is due to the sorting algorithm being slow.
this is what i have so far
This is the loop code
while (!word.Get_found())
{
generation++;
word.Calculate_fitness();
word.Selection(); //selection
word.Crossover(); //crossover
system("cls");
std::cout << "Generation: " << generation << " Highest fitness: " << word.get_fittest() << " with string: " << word.get_item() << "\n";
}
This is the code for the fitness function
void Guess_word::Calculate_fitness()// calculates fittness based on guess
word against matching string;
{
for (int i = 0; i < population.size(); i++)
{
population.at(i).second = 0;
for (int j = 0; j < population.at(i).first.size(); j++)
{
if (population.at(i).first.at(j) == Phrase.at(j))
{
population.at(i).second += 1;//calculate fitness
}
}
if (population.at(i).second == Phrase.size() && population.at(i).first == Phrase)
{
found = true;
}
}
}
And this is the selection function
void Guess_word::Selection()//determine highest fitness of population and make them parents
{
//i hate stable sort....
//it indicates to sort in pairs and keep them together
std::sort(population.begin(), population.end(), [](auto &a, auto &b) { return a.second > b.second; });
//select two random parent from mating pool
parents.clear();
parents.push_back(population.at(0));
parents.push_back(population.at(1));
}
The population entities are in vector pairs with strings and ints representing the guess and fitness respectively.
after debugging the code i found that the population does indeed contain the correct guess but with the wrong fitness, i think the sorting algorithm moves the ints faster than the paired strings. meaning that during the fitness function it selects an item as the answer that is one character incorrect however with the correct fitness moved from another vector entity.
I've tried using stable sort and moving the algorithm around to see if timing is a problem. however, no dice.
is there a way to either make the program wait for the sort to complete (which is inefficient in terms of time) or a way to either make the sort faster or implement a faster custom sorting algorithm which would be much more efficient especially on older hardware.
Any suggestions would be greatly appreciated!
The issue was simply, the code doing a cross over and storing it back in position 0 of the population making it change randomly just before the final result was displayed
I'll start by giving some context. I'm learning to write a raytracer, a very simple one. I don't have any acceleration structures yet, so the code in question is intended to find the closest object that the ray hits. Since I'm learning yet, I'd greatly appreciate if the answers concentrated on the seemingly strange problem that I'm observing - I know the RT logic is very wrong as it is right now. It produces correct results, anyway.
1. The first approach: for every hit, add a hit-result structure object to the list, then apply std::sort with a predicate that compares the distance form the hit point to the ray origin. Should be O(N log N) according to the textbook, and I think it is suboptimal, since I only need the first result, not the whole sorted list.
2. The second approach: whenever there is a hit, take the distance and compare it to the minimum, which is first initialized to std::numeric_limits<float>::max(). Well, your standard "find min in the array" algorithm. Should be O(N) and thus faster.
These pieces of code reside in a recursive function. Tested on the very same scene of 10 spheres, 1 is faster by an order of magnitude. The amount of calls to the distance function is a few times less than in 2. What am I missing?
I'm not sure if the context is required, in case there are "branches to be cut" off this question, tell me.
Code piece 1:
result rt_function(...) {
static int count{};
std::vector<result> hitList;
for(const auto& obj : objList) {
const result res = obj->testOuter(ray);
if ( res.hit ) {
hitList.push_back(res);
}
}
if (!hitList.empty()) {
sort(hitList.begin(), hitList.end(), [=](result& hit1, result& hit2) -> bool {
std::cerr << ++count << '\n';
return cv::norm(hit1.point - ray.origin) <
cv::norm(hit2.point - ray.origin);
});
const result res = hitList.front();
const SceneObject* near = res.obj;
// the raytracing continues...
count == 180771
Code piece 2:
result rt_function(...) {
static int count{};
float min_distance = std::numeric_limits<float>::max(), distance{};
result closest_res{}; bool have_hit{};
for(const auto& obj : objList) {
const result res = obj->testOuter(ray);
if ( res.hit ) {
have_hit = true;
std::cerr << ++count << '\n';
distance = cv::norm(res.point - ray.origin);
if (distance < min_distance) {
min_distance = distance; closest_res = res;
}
}
}
if (have_hit) {
const result res = closest_res;
const SceneObject* near = res.obj;
// the raytracing continues...
count == 349633
I want to (a) understand why there are less comparisons and (b) where the bottleneck is, since the run time is significantly higher, as I've noted above.
Statements like O(N²) are like a dimension; double the number of points and time taken quadruples. An O(log N) algorithm can be slow for small N , the point being if N doubles or is increased by a factor of 10 running time doesn't.
Compare with finding a specific word in a 1000 page dictionary and one in a 20 word sentence. Sorting a 20 word sentence before finding a specific word takes longer than reading it straight through once.
I created a program that finds the median of a list of numbers. The list of numbers is dynamic in that numbers can be removed and inserted (duplicate numbers can be entered) and during this time, the new median is re-evaluated and printed out.
I created this program using a multimap because
1) the benefit of it being already being sorted,
2) easy insertion, deletion, searching (since multimap implements binary search)
3) duplicate entries are allowed.
The constraints for the number of entries + deletions (represented as N) are: 0 < N <= 100,000.
The program I wrote works and prints out the correct median, but it isn't fast enough. I know that the unsorted_multimap is faster than multimap, but then the problem with unsorted_multimap is that I would have to sort it. I have to sort it because to find the median you need to have a sorted list. So my question is, would it be practical to use an unsorted_multimap and then quick sort the entries, or would that just be ridiculous? Would it be faster to just use a vector, quicksort the vector, and use a binary search? Or maybe I am forgetting some fabulous solution out there that I haven't even thought of.
Though I'm not new to C++, I will admit, that my skills with time-complexity are somewhat medicore.
The more I look at my own question, the more I'm beginning to think that just using a vector with quicksort and binary search would be better since the data structures basically already implement vectors.
the more I look at my own question, the more I'm beginning to think that just using vector with quicksort and binary search would be better since the data structures basically already implement vectors.
If you have only few updates - use unsorted std::vector + std::nth_element algorithm which is O(N). You don't need full sorting which is O(N*ln(N)).
live demo of nth_element:
#include <algorithm>
#include <iterator>
#include <iostream>
#include <ostream>
#include <vector>
using namespace std;
template<typename RandomAccessIterator>
RandomAccessIterator median(RandomAccessIterator first,RandomAccessIterator last)
{
RandomAccessIterator m = first + distance(first,last)/2; // handle even middle if needed
nth_element(first,m,last);
return m;
}
int main()
{
vector<int> values = {5,1,2,4,3};
cout << *median(begin(values),end(values)) << endl;
}
Output is:
3
If you have many updates and only removing from middle - use two heaps as comocomocomocomo suggests. If you would use fibonacci_heap - then you would also get O(N) removing from arbitary position (if don't have handle to it).
If you have many updates and need O(ln(N)) removing from arbitary places - then use two multisets as ipc suggests.
If your purpose is to keep track of the median on the fly, as elements are inserted/removed, you should use a min-heap and a max-heap. Each one would contain one half of the elements... There was a related question a couple of days ago: How to implement a Median-heap
Though, if you need to search for specific values in order to remove elements, you still need some kind of map.
You said that it is slow. Are you iterating from the beginning of the map to the (N/2)'th element every time you need the median? You don't need to. You can keep track of the median by maintaining an iterator pointing to it at all times and a counter of the number of elements less than that one. Every time you insert/remove, compare the new/old element with the median and update both iterator and counter.
Another way of seeing it is as two multimaps containing half the elements each. One holds the elements less than the median (or equal) and the other holds those greater. The heaps do this more efficiently, but they don't support searches.
If you only need the median a few times you can use the "select" algorithm. It is described in Sedgewick's book. It takes O(n) time on average. It is similar to quick sort but it does not sort completely. It just partitions the array with random pivots until, eventually, it gets to "select" on one side the smaller m elements (m=(n+1)/2). Then you search for the greatest of those m elements, and this is the median.
Here is how you could implement that in O(log N) per update:
template <typename T>
class median_set {
public:
std::multiset<T> below, above;
// O(log N)
void rebalance()
{
int diff = above.size() - below.size();
if (diff > 0) {
below.insert(*above.begin());
above.erase(above.begin());
} else if (diff < -1) {
above.insert(*below.rbegin());
below.erase(below.find(*below.rbegin()));
}
}
public:
// O(1)
bool empty() const { return below.empty() && above.empty(); }
// O(1)
T const& median() const
{
assert(!empty());
return *below.rbegin();
}
// O(log N)
void insert(T const& value)
{
if (!empty() && value > median())
above.insert(value);
else
below.insert(value);
rebalance();
}
// O(log N)
void erase(T const& value)
{
if (value > median())
above.erase(above.find(value));
else
below.erase(below.find(value));
rebalance();
}
};
(Work in action with tests)
The idea is the following:
Keep track of the values above and below the median in two sets
If a new value is added, add it to the corresponding set. Always ensure that the set below has exactly 0 or 1 more then the other
If a value is removed, remove it from the set and make sure that the condition still holds.
You can't use priority_queues because they won't let you remove one item.
Can any one help me what is Space and Time complexity of my following C# program with details.
//Passing Integer array to Find Extreme from that Integer Array
public int extreme(int[] A)
{
int N = A.Length;
if (N == 0)
{
return -1;
}
else
{
int average = CalculateAverage(A);
return FindExtremes(A, average);
}
}
// Calaculate Average of integerArray
private int CalculateAverage(int[] integerArray)
{
int sum = 0;
foreach (int value in integerArray)
{
sum += value;
}
return Convert.ToInt32(sum / integerArray.Length);
}
//Find Extreme from that Integer Array
private int FindExtremes(int[] integerArray, int average) {
int Index = -1; int ExtremeElement = integerArray[0];
for (int i = 0; i < integerArray.Length; i++)
{
int absolute = Math.Abs(integerArray[i] - average);
if (absolute > ExtremeElement)
{
ExtremeElement = integerArray[i];
Index = i;
}
}
return Index;
}
You are almost certainly better off using a vector. Possibly maintaining an auxiliary vector of indexes to be removed between median calculations so you can delete them in batches. New additions can also be put into an auxiliary vector, sorted, then merged in.
I have coded DFS as the way it is on my mind and didn't referred any Text book or Pseudo-code for ideas. I think I have some lines of codes that are making unnecessary calculations. Any ideas on reducing the complexity of my algorithm ?
vector<int>visited;
bool isFound(vector<int>vec,int value)
{
if(std::find(vec.begin(),vec.end(),value)==vec.end())
return false;
else
return true;
}
void dfs(int **graph,int numOfNodes,int node)
{
if(isFound(visited,node)==false)
visited.push_back(node);
vector<int>neighbours;
for(int i=0;i<numOfNodes;i++)
if(graph[node][i]==1)
neighbours.push_back(i);
for(int i=0;i<neighbours.size();i++)
if(isFound(visited,neighbours[i])==false)
dfs(graph,numOfNodes,neighbours[i]);
}
void depthFirstSearch(int **graph,int numOfNodes)
{
for(int i=0;i<numOfNodes;i++)
dfs(graph,numOfNodes,i);
}
PS: Could somebody please sent me a link teaching me how can to insert C++ code with good quality. I've tried syntax highlighting but it didn't work out.
Your DFS has O(n^2) time complexity, which is really bad (it should run in O(n + m)).
This line ruins your implementation, because searching in vector takes time proportional to its length:
if(std::find(vec.begin(),vec.end(),value)==vec.end())
To avoid this, you can remember what was visited in an array of boolean values.
Second problem with your DFS is that for bigger graph it will probably cause stack overflow, because worst case recursion depth is equal to number of vertices in graph. Remedy to this problem is also simple: use std::list<int> as your own stack.
So, code that does DFS should look more or less like this:
// n is number of vertices in graph
bool visited[n]; // in this array we save visited vertices
std::list<int> stack;
std::list<int> order;
for(int i = 0; i < n; i++){
if(!visited[i]){
stack.push_back(i);
while(!stack.empty()){
int top = stack.back();
stack.pop_back();
if(visited[top])
continue;
visited[top] = true;
order.push_back(top);
for(all neighbours of top)
if(!visited[neighbour])
stack.push_back(neighbour);
}
}
}