Related
I need help adjusting the createTree function.
Which accepts a string and after that character by character traverses it, creating a binary tree based on it
If it encounters the character 0, it recursively creates two sub-branches.
If it encounters another character, it saves it in the leaf node.
For the string in the example, I need to make a tree as in the picture, but the function does not work properly for me. Thank you in advance for your advice.
int x = 0;
Node* createTree(string str, int si, int ei)
{
if (si > ei)
return NULL;
Node *root = new Node((str[si] - '0'));
if(str[si] != '0')
{
x++;
root->m_Data = (str[si] - '0');
return root;
}
if(str[si]=='0')
{
x++;
root->m_Left = createTree(str,x,ei);
root->m_Right = createTree(str,x,ei);
}
return root;
}
int main ()
{
string str = "050067089";
Node *node = createTree(str,0,str.length());
printPreorder(node);
return 0;
}
The problem can quite easily be broken down into small steps (what you partly did in your question).
Start iterating at the first character
Create the root node
If the current character is non-zero, set the value of this node to this character
If current character is a zero, set this node to zero, create a left and a right node and get back to step 3 for every one of them. (That's the recursive part.)
Below is my implementation of this algorithm.
First, a little bit of setting up:
#include <iostream>
#include <string>
#include <memory>
struct Node;
// Iterator to a constant character, NOT a constant iterator
using StrConstIt = std::string::const_iterator;
using UniqueNode = std::unique_ptr<Node>;
struct Node
{
int value;
UniqueNode p_left;
UniqueNode p_right;
Node(int value)
: value(value) {}
Node(int value, UniqueNode p_left, UniqueNode p_right)
: value(value), p_left(std::move(p_left)), p_right(std::move(p_right)) {}
};
As you can see, I'm using std::unique_ptr for managing memory. This way, you don't have to worry about manually deallocating memory. Using smart pointers is often considered the more "modern" approach, and they should virtually always be preferred over raw pointers.
UniqueNode p_createNodeAndUpdateIterator(StrConstIt& it, StrConstIt stringEnd)
{
if (it >= stringEnd)
return nullptr;
UniqueNode node;
if (*it == '0')
// Create node with appropriate value
// Create branches and increment iterator
node = std::make_unique<Node>(
0,
p_createNodeAndUpdateIterator(++it, stringEnd),
p_createNodeAndUpdateIterator(it, stringEnd)
);
else
{
// Create leaf node with appropriate value
node = std::make_unique<Node>(*it - '0');
// Increment iterator
++it;
}
return node;
}
UniqueNode p_createTree(StrConstIt begin, StrConstIt end)
{
return p_createNodeAndUpdateIterator(begin, end);
}
The first function takes a reference to the iterator to the next character it should process. That is because you can't know how much characters a branch will have in its leaf nodes beforehand. Therefore, as the function's name suggests, it will update the iterator with the processing of each character.
I'm using iterators instead of a string and indices. They are clearer and easier to work with in my opinion — changing it back should be fairly easy anyway.
The second function is basically syntactic sugar: it is just there so that you don't have to pass an lvalue as the first argument.
You can then just call p_createTree with:
int main()
{
std::string str = "050067089";
UniqueNode p_root = p_createTree(str.begin(), str.end());
return 0;
}
I also wrote a function to print out the tree's nodes for debugging:
void printTree(const UniqueNode& p_root, int indentation = 0)
{
// Print the value of the node
for (int i(0); i < indentation; ++i)
std::cout << "| ";
std::cout << p_root->value << '\n';
// Do nothing more in case of a leaf node
if (!p_root->p_left.get() && !p_root->p_right.get())
;
// Otherwise, print a blank line for empty children
else
{
if (p_root->p_left.get())
printTree(p_root->p_left, indentation + 1);
else
std::cout << '\n';
if (p_root->p_right.get())
printTree(p_root->p_right, indentation + 1);
else
std::cout << '\n';
}
}
Assuming that the code which is not included in your question is correct, there is only one issue that could pose a problem if more than one tree is built. The problem is that x is a global variable which your functions change as a side-effect. But if that x is not reset before creating another tree, things will go wrong.
It is better to make x a local variable, and pass it by reference.
A minor thing: don't use NULL but nullptr.
Below your code with that change and the class definition included. I also include a printSideways function, which makes it easier to see that the tree has the expected shape:
#include <iostream>
using namespace std;
class Node {
public:
int m_Data;
Node* m_Left = nullptr;
Node* m_Right = nullptr;
Node(int v) : m_Data(v) {}
};
// Instead of si, accept x by reference:
Node* createTree(string str, int &x, int ei)
{
if (x >= ei)
return nullptr;
Node *root = new Node((str[x] - '0'));
if(str[x] != '0')
{
root->m_Data = (str[x] - '0');
x++;
return root;
}
if(str[x]=='0')
{
x++;
root->m_Left = createTree(str,x,ei);
root->m_Right = createTree(str,x,ei);
}
return root;
}
// Overload with a wrapper that defines x
Node* createTree(string str)
{
int x = 0;
return createTree(str, x, str.length());
}
// Utility function to visualise the tree with the root at the left
void printSideways(Node *node, string tab) {
if (node == nullptr) return;
printSideways(node->m_Right, tab + " ");
cout << tab << node->m_Data << "\n";
printSideways(node->m_Left, tab + " ");
}
// Wrapper for above function
void printSideways(Node *node) {
printSideways(node, "");
}
int main ()
{
string str = "050067089";
Node *node = createTree(str);
printSideways(node);
return 0;
}
So, as you see, nothing much was altered. Just si was replaced with x, which is passed around by reference, and x is defined locally in a wrapper function.
Here is the output:
9
0
8
0
7
0
6
0
5
I have been trying to implement a C++ implementation of insertion of trie data-structure, working through a blog in which there are a few things I am unable to understand http://theoryofprogramming.com/2015/01/16/trie-tree-implementation/
#define ALPHABETS 26
#define CASE 'a'
#define MAX_WORD_SIZE 25
using namespace std;
struct Node
{
struct Node * parent;
struct Node * children[ALPHABETS];
vector<int> occurrences;
};
// Inserts a word 'text' into the Trie Tree
// 'trieTree' and marks it's occurence as 'index'.
void InsertWord(struct Node * trieTree, char word[], int index)
{
struct Node * traverse = trieTree;
while (*word != '\0') { // Until there is something to process
if (traverse->children[*word - CASE] == NULL) {
// There is no node in 'trieTree' corresponding to this alphabet
// Allocate using calloc(), so that components are initialised
traverse->children[*word - CASE] = (struct Node *) calloc(1, sizeof(struct Node));
traverse->children[*word - CASE]->parent = traverse; // Assigning parent
}
traverse = traverse->children[*word - CASE];
++word; // The next alphabet
}
traverse->occurrences.push_back(index); // Mark the occurence of the word
}
// Prints the 'trieTree' in a Pre-Order or a DFS manner
// which automatically results in a Lexicographical Order
void LexicographicalPrint(struct Node * trieTree, vector<char> word)
{
int i;
bool noChild = true;
if (trieTree->occurrences.size() != 0) {
// Condition trie_tree->occurrences.size() != 0,
// is a neccessary and sufficient condition to
// tell if a node is associated with a word or not
vector<char>::iterator charItr = word.begin();
while (charItr != word.end()) {
printf("%c", *charItr);
++charItr;
}
printf(" -> # index -> ");
vector<int>::iterator counter = trieTree->occurrences.begin();
// This is to print the occurences of the word
while (counter != trieTree->occurrences.end()) {
printf("%d, ", *counter);
++counter;
}
printf("\n");
}
for (i = 0; i < ALPHABETS; ++i) {
if (trieTree->children[i] != NULL) {
noChild = false;
word.push_back(CASE + i); // Select a child
// and explore everything associated with the cild
LexicographicalPrint(trieTree->children[i], word);
word.pop_back();
// Remove the alphabet as we dealt
// everything associated with it
}
}
word.pop_back();
}
int main()
{
int n, i;
vector<char> printUtil; // Utility variable to print tree
// Creating the Trie Tree using calloc
// so that the components are initialised
struct Node * trieTree = (struct Node *) calloc(1, sizeof(struct Node));
char word[MAX_WORD_SIZE];
printf("Enter the number of words-\n");
scanf("%d", &n);
for (i = 1; i <= n; ++i) {
scanf("%s", word);
InsertWord(trieTree, word, i);
}
printf("\n"); // Just to make the output more readable
LexicographicalPrint(trieTree, printUtil);
return 0;
}
I am unable to understand what this statement in insertword does:
if (traverse->children[*word - CASE] == NULL)
Also as we have initialised all the elements to 1 in the main function then how we can it be null?
The function InsertWord() dynamically adds a new word into the trie, and in the process, creates new nodes whenever that word's prefix does not match the prefix of another word already added in the trie.
This is exactly what your line is testing for. From what I can see, traverse is a pointer to the current Node of a prefix of the word. *word is the next character in the word after the prefix. If the node corresponding to the current k-prefix of the word doesn't have a child (pointer is NULL) with label corresponding to the next character, that means we have to allocate a new node for the next k+1-prefix of the word.
I'm currently working on a trie implementation for practice and have run into a mental roadbloack.
The issue is with my searching function. I am attempting to have my trie tree be able to retrieve a list of strings from a supplied prefix after they are loaded into the programs memory.
I also understand I could be using a queue/shouldnt use C functions in C++ ect.. This is just a 'rough draft' so to speak.
This is what I have so far:
bool SearchForStrings(vector<string> &output, string data)
{
Node *iter = GetLastNode("an");
Node *hold = iter;
stack<char> str;
while (hold->visited == false)
{
int index = GetNextChild(iter);
if (index > -1)
{
str.push(char('a' + index));
//current.push(iter);
iter = iter->next[index];
}
//We've hit a leaf so we want to unwind the stack and print the string
else if (index < 0 && IsLeaf(iter))
{
iter->visited = true;
string temp("");
stringstream ss;
while (str.size() > 0)
{
temp += str.top();
str.pop();
}
int i = 0;
for (std::string::reverse_iterator it = temp.rbegin(); it != temp.rend(); it++)
ss << *it;
//Store the string we have
output.push_back(data + ss.str());
//Move our iterator back to the root node
iter = hold;
}
//We know this isnt a leaf so we dont want to print out the stack
else
{
iter->visited = true;
iter = hold;
}
}
return (output.size() > 0);
}
int GetNextChild(Node *s)
{
for (int i = 0; i < 26; i++)
{
if (s->next[i] != nullptr && s->next[i]->visited == false)
return i;
}
return -1;
}
bool IsLeaf(Node *s)
{
for (int i = 0; i < 26; i++)
{
if (s->next[i] != nullptr)
return false;
}
return true;
}
struct Node{
int value;
Node *next[26];
bool visited;
};
The code is too long or i'd post it all, GetLastNode() retrieves the node at the end of the data passed in, so if the prefix was 'su' and the string was 'substring' the node would be pointing to the 'u' to use as an artificial root node
(might be completely wrong... just typed it here, no testing)
something like:
First of all, we need a way of indicating that a node represents an entry.
So let's have:
struct Node{
int value;
Node *next[26];
bool entry;
};
I've removed your visited flag because I don't have a use for it.
You should modify your insert/update/delete functions to support this flag. If the flag is true it means there's an actual entry up to that node.
Now we can modify the
bool isLeaf(Node *s) {
return s->entry;
}
Meaning that we consider a leaf when there's an entry... perhaps the name is wrong now, as the leaf might have childs ("y" node with "any" and "anywhere" is a leaf, but it has childs)
Now for the search:
First a public function that can be called.
bool searchForStrings(std::vector<string> &output, const std::string &key) {
// start the recursion
// theTrieRoot is the root node for the whole structure
return searchForString(theTrieRoot,output,key);
}
Then the internal function that will use for recursion.
bool searchForStrings(Node *node, std::vector<string> &output, const std::string &key) {
if(isLeaf(node->next[i])) {
// leaf node - add an empty string.
output.push_back(std::string());
}
if(key.empty()) {
// Key is empty, collect all child nodes.
for (int i = 0; i < 26; i++)
{
if (node->next[i] != nullptr) {
std::vector<std::string> partial;
searchForStrings(node->next[i],partial,key);
// so we got a list of the childs,
// add the key of this node to them.
for(auto s:partial) {
output.push_back(std::string('a'+i)+s)
}
}
} // end for
} // end if key.empty
else {
// key is not empty, try to get the node for the
// first character of the key.
int c=key[0]-'a';
if((c<0 || (c>26)) {
// first character was not a letter.
return false;
}
if(node->next[c]==nullptr) {
// no match (no node where we expect it)
return false;
}
// recurse into the node matching the key
std::vector<std::string> partial;
searchForStrings(node->next[c],partial,key.substr(1));
// add the key of this node to the result
for(auto s:partial) {
output.push_back(std::string(key[0])+s)
}
}
// provide a meaningful return value
if(output.empty()) {
return false;
} else {
return true;
}
}
And the execution for "an" search is.
Call searchForStrings(root,[],"an")
root is not leaf, key is not empty. Matched next node keyed by "a"
Call searchForStrings(node(a),[],"n")
node(a) is not leaf, key is not empty. Matched next node keyed by "n"
Call searchForStrings(node(n),[],"")
node(n) is not leaf, key is empty. Need to recurse on all not null childs:
Call searchForStrings(node(s),[],"")
node(s) is not leaf, key is empty, Need to recurse on all not null childs:
... eventually we will reach Node(r) which is a leaf node, so it will return an [""], going back it will get added ["r"] -> ["er"] -> ["wer"] -> ["swer"]
Call searchForStings(node(y),[],"")
node(y) is leaf (add "" to the output), key is empty,
recurse, we will get ["time"]
we will return ["","time"]
At this point we will add the "y" to get ["y","ytime"]
And here we will add the "n" to get ["nswer","ny","nytime"]
Adding the "a" to get ["answer","any","anytime"]
we're done
I inadvertently let my students overconstrain a shared class used to solve the following problem. I realized it might be a problem denizens of this site might enjoy.
The first team/function, getNodes, takes a string representing a prefix expression using signed integers and the four operations +, -, *, and / and produces the corresponding null terminated linked list of tokens, using the class Node, with tokens linked through the "right" pointer.
The second team/function, getTree, takes a similar string, passes it to getNodes, and relinks the resultant nodes to be an expression tree.
The third team/function, evaluate, takes a similar string, passes it to getTree, and evaluates the resultant expression tree to form an answer.
The over-constrained exptree.h follows. The problem has to be solved by writing just the three functions defined above, no additional functions.
#ifndef EXPTREE_H_
#define EXPTREE_H_
using namespace std;
enum Ops{ADD, SUB, MUL, DIV, NUM};
class Node {
private:
int num;
Ops op;
Node *left, *right;
public:
friend Node *getNodes(string d);
friend Node *getTree(string d);
friend int evaluate (string);
};
int evaluate(string d);
Node *getNodes(string d);
Node *getTree(string d);
#endif
The only libraries that can be used are these
#include <iostream>
#include <vector>
#include <string>
#include "exptree.h"
For those of you worried about my students, I will be pointing out today how just a couple of more well placed functions would allow this problem to be easily solved. I know the expression tree can code rational numbers and not just integers. I'll be pointing that out today as well.
Here is the driver program I gave them based on their specs.
#include <iostream>
#include <string>
#include "exptree.h"
using namespace std;
void test(string s, int target) {
int result = evaluate(s);
if (result == target)
cout << s << " correctly evaluates to " << target << endl;
else
cout << s << "(" << result
<< ") incorrectly evaluates to " << target << endl;
}
int main() {
test("42", 42);
test("* - / 4 2 1 42", 42);
test("* - / -4 +2 -1 2", -2);
test("* - / -4 +2 -1 2 ", -2);
test("* 9 6", 54);
return 0;
}
Can you write the three functions in as elegant a fashion as possible to solve this nightmarish problem?
The getNodes and getTree functions would be pretty trivial to write under these constraints, so I just skipped ahead to the interesting part. You would naturally evaluate an expression tree recursively, but that is not an option here because the eval function only takes a string. Sure, you could restringify the remaining tree into a prefix expression and call eval recursively on that, but that would just be stupid.
First, I convert the expression tree into a postfix expression, using an explicit stack as the poor man's recursion. Then I evaluate that with the standard operand stack.
#include <iostream>
#include <vector>
#include <string>
using namespace std;
#include "exptree.h"
int evaluate(string d){
Node* tree = getTree(d);
//convert tree to postfix for simpler evaluation
vector<Node*> node_stack;
node_stack.push_back(tree);
Node postfix_head;
Node* postfix_tail = &postfix_head;
while(node_stack.size() > 0){
Node* place = node_stack.back();
if(place->left == 0){
if(place->right == 0){
postfix_tail->right = place;
node_stack.pop_back();
} else {
node_stack.push_back(place->right);
place->right = 0;
}
} else {
node_stack.push_back(place->left);
place->left = 0;
}
}
//evaluate postfix
Node* place = postfix_head.right;
vector<int> stack;
while(place != 0){
if(place->op != NUM){
int operand_a, operand_b;
operand_b = stack.back();
stack.pop_back();
operand_a = stack.back();
stack.pop_back();
switch(place->op){
case ADD:
stack.push_back(operand_a + operand_b);
break;
case SUB:
stack.push_back(operand_a - operand_b);
break;
case MUL:
stack.push_back(operand_a * operand_b);
break;
case DIV:
stack.push_back(operand_a / operand_b);
break;
}
} else {
stack.push_back(place->num);
}
place = place->right;
}
return stack.back();
}
I think that "no additional functions" is a too tough requirement. The easiest way to implement e.g. getTree is probably recursive, and it requires defining an additional function.
Node* relink(Node* start) // builds a tree; returns the following node
{
if (start->op == NUM)
{
Node* result = start->right;
start->left = start->right = NULL;
return result;
}
else
{
start->left = start->right;
start->right = relink(start->left);
return relink(start->right);
}
}
Node* getTree(string d)
{
Node* head = getNodes(d);
relink(head);
return head;
}
I could implement recursion by using an explicit stack (implemented by std::vector) but that is ugly and obscure (unless you want you students to practice exactly that).
For what its worth, here is the solution I coded up just before I posted the question
#include <iostream>
#include <vector>
#include "exptree.h"
using namespace std;
Node *getNodes(string s) {
const int MAXINT =(int)(((unsigned int)-1) >> 1), MININT = -MAXINT -1;
Node *list;
int sign, num;
s += " "; // this simplifies a lot of logic, allows trailing white space to always close off an integer
list = (Node *) (num = sign = 0);
for (int i=0; i<s.size(); ++i) {
char c = s[i]; // more efficient and cleaner reference to the current character under scrutiny
if (isdigit(c)) {
if (sign == 0) sign = 1; // if sign not set, then set it. A blank with a sign==0 now signifies a blank that can be skipped
num = 10*num + c - '0';
} else if (((c=='+') || (c=='-')) && isdigit(s[i+1])) { // another advantage of adding blank to string above so don't need a special case
sign = (c=='+') ? 1 : -1;
} else if ( !isspace(c) && (c != '+') && (c != '-') && (c != '*') && (c != '/')) {
cout << "unexpected character " << c << endl;
exit(1);
} else if (!isspace(c) || (sign != 0)) { // have enough info to create next Node
list->left = (list == 0) ? (list = new Node) : (list->left->right = new Node); // make sure left pointer of first Node points to last Node
list->left->right = 0; // make sure list is still null terminated
list->left->op = (c=='+' ? ADD : (c=='-' ? SUB : (c=='*' ? MUL : (c=='/' ? DIV : NUM)))); // choose right enumerated type
list->left->num = (list->left->op==NUM) ? sign*num : MININT; // if interior node mark number for evaluate function
num = sign = 0; // prepare for next Node
}
}
return list;
}
Node *getTree(string s) {
Node *nodes = getNodes(s), *tree=0, *root, *node;
vector<Node *> stack;
if (nodes == 0) return tree;
root = tree = nodes;
nodes = nodes->right;
for (node=nodes; node != 0; node=nodes) {
nodes = nodes->right;
if (root->op != NUM) { // push interior operator Node on stack til time to point to its right tree
stack.push_back(root);
root = (root->left = node); // set interior operator Node's left tree and prepare to process that left tree
} else {
root->left = root->right = 0; // got a leaf number Node so finish it off
if (stack.size() == 0) break;
root = stack.back(); // now pop operator Node off the stack
stack.pop_back();
root = (root->right = node); // set its left tree and prepare to process that left tree
}
}
if ((stack.size() != 0) || (nodes != 0)) {
cout << "prefix expression has missing or extra terms" << endl;
exit(1);
}
return tree;
}
int evaluate(string s) {
// MININT is reserved value signifying operator waiting for a left side value, low inpact since at edge of representable integers
const int MAXINT =(int)(((unsigned int)-1) >> 1), MININT = -MAXINT -1;
Node *tree = getTree(s);
vector<Node *> stack;
int v = 0; // this is value of a leaf node (a number) or the result of evaluating an interior node
if (tree == 0) return v;
do {
v = tree->num;
if (tree->op != NUM) {
stack.push_back(tree);
tree = tree->left; // prepare to process the left subtree
} else while (stack.size() != 0) { // this while loop zooms us up the right side as far as we can go (till we come up left side or are done)
delete tree; // done with leaf node or an interior node we just finished evaluating
tree = stack.back(); // get last interior node from stack
if (tree->num == MININT) { // means returning up left side of node, so save result for later
tree->num = v;
tree = tree->right; // prepare to evaluate the right subtree
break; // leave the "else while" for the outer "do while" which handles evaluating an expression tree
} else { // coming up right side of an interior node (time to calculate)
stack.pop_back(); // all done with interior node
v = tree->op==ADD ? tree->num+v : (tree->op==SUB ? tree->num-v : (tree->op==MUL ? tree->num*v : tree->num/v)) ;
}
}
} while (stack.size() != 0);
return v;
}
I have an array of a few million numbers.
double* const data = new double (3600000);
I need to iterate through the array and find the range (the largest value in the array minus the smallest value). However, there is a catch. I only want to find the range where the smallest and largest values are within 1,000 samples of each other.
So I need to find the maximum of: range(data + 0, data + 1000), range(data + 1, data + 1001), range(data + 2, data + 1002), ...., range(data + 3599000, data + 3600000).
I hope that makes sense. Basically I could do it like above, but I'm looking for a more efficient algorithm if one exists. I think the above algorithm is O(n), but I feel that it's possible to optimize. An idea I'm playing with is to keep track of the most recent maximum and minimum and how far back they are, then only backtrack when necessary.
I'll be coding this in C++, but a nice algorithm in pseudo code would be just fine. Also, if this number I'm trying to find has a name, I'd love to know what it is.
Thanks.
This type of question belongs to a branch of algorithms called streaming algorithms. It is the study of problems which require not only an O(n) solution but also need to work in a single pass over the data. the data is inputted as a stream to the algorithm, the algorithm can't save all of the data and then and then it is lost forever. the algorithm needs to get some answer about the data, such as for instance the minimum or the median.
Specifically you are looking for a maximum (or more commonly in literature - minimum) in a window over a stream.
Here's a presentation on an article that mentions this problem as a sub problem of what they are trying to get at. it might give you some ideas.
I think the outline of the solution is something like that - maintain the window over the stream where in each step one element is inserted to the window and one is removed from the other side (a sliding window). The items you actually keep in memory aren't all of the 1000 items in the window but a selected representatives which are going to be good candidates for being the minimum (or maximum).
read the article. it's abit complex but after 2-3 reads you can get the hang of it.
The algorithm you describe is really O(N), but i think the constant is too high. Another solution which looks reasonable is to use O(N*log(N)) algorithm the following way:
* create sorted container (std::multiset) of first 1000 numbers
* in loop (j=1, j<(3600000-1000); ++j)
- calculate range
- remove from the set number which is now irrelevant (i.e. in index *j - 1* of the array)
- add to set new relevant number (i.e. in index *j+1000-1* of the array)
I believe it should be faster, because the constant is much lower.
This is a good application of a min-queue - a queue (First-In, First-Out = FIFO) which can simultaneously keep track of the minimum element it contains, with amortized constant-time updates. Of course, a max-queue is basically the same thing.
Once you have this data structure in place, you can consider CurrentMax (of the past 1000 elements) minus CurrentMin, store that as the BestSoFar, and then push a new value and pop the old value, and check again. In this way, keep updating BestSoFar until the final value is the solution to your question. Each single step takes amortized constant time, so the whole thing is linear, and the implementation I know of has a good scalar constant (it's fast).
I don't know of any documentation on min-queue's - this is a data structure I came up with in collaboration with a coworker. You can implement it by internally tracking a binary tree of the least elements within each contiguous sub-sequence of your data. It simplifies the problem that you'll only pop data from one end of the structure.
If you're interested in more details, I can try to provide them. I was thinking of writing this data structure up as a paper for arxiv. Also note that Tarjan and others previously arrived at a more powerful min-deque structure that would work here, but the implementation is much more complex. You can google for "mindeque" to read about Tarjan et al.'s work.
std::multiset<double> range;
double currentmax = 0.0;
for (int i = 0; i < 3600000; ++i)
{
if (i >= 1000)
range.erase(range.find(data[i-1000]));
range.insert(data[i]);
if (i >= 999)
currentmax = max(currentmax, *range.rbegin());
}
Note untested code.
Edit: fixed off-by-one error.
read in the first 1000 numbers.
create a 1000 element linked list which tracks the current 1000 number.
create a 1000 element array of pointers to linked list nodes, 1-1 mapping.
sort the pointer array based on linked list node's values. This will rearrange the array but keep the linked list intact.
you can now calculate the range for the first 1000 numbers by examining the first and last element of the pointer array.
remove the first inserted element, which is either the head or the tail depending on how you made your linked list. Using the node's value perform a binary search on the pointer array to find the to-be-removed node's pointer, and shift the array one over to remove it.
add the 1001th element to the linked list, and insert a pointer to it in the correct position in the array, by performing one step of an insertion sort. This will keep the array sorted.
now you have the min. and max. value of the numbers between 1 and 1001, and can calculate the range using the first and last element of the pointer array.
it should now be obvious what you need to do for the rest of the array.
The algorithm should be O(n) since the delete and insertion is bounded by log(1e3) and everything else takes constant time.
I decided to see what the most efficient algorithm I could think of to solve this problem was using actual code and actual timings. I first created a simple solution, one that tracks the min/max for the previous n entries using a circular buffer, and a test harness to measure the speed. In the simple solution, each data value is compared against the set of min/max values, so that's about window_size * count tests (where window size in the original question is 1000 and count is 3600000).
I then thought about how to make it faster. First off, I created a solution that used a fifo queue to store window_size values and a linked list to store the values in ascending order where each node in the linked list was also a node in the queue. To process a data value, the item at the end of the fifo was removed from the linked list and the queue. The new value was added to the start of the queue and a linear search was used to find the position in the linked list. The min and max values could then be read from the start and end of the linked list. This was quick, but wouldn't scale well with increasing window_size (i.e. linearly).
So I decided to add a binary tree to the system to try to speed up the search part of the algorithm. The final timings for window_size = 1000 and count = 3600000 were:
Simple: 106875
Quite Complex: 1218
Complex: 1219
which was both expected and unexpected. Expected in that using a sorted linked list helped, unexpected in that the overhead of having a self balancing tree didn't offset the advantage of a quicker search. I tried the latter two with an increased window size and found that the were always nearly identical up to a window_size of 100000.
Which all goes to show that theorising about algorithms is one thing, implementing them is something else.
Anyway, for those that are interested, here's the code I wrote (there's quite a bit!):
Range.h:
#include <algorithm>
#include <iostream>
#include <ctime>
using namespace std;
// Callback types.
typedef void (*OutputCallback) (int min, int max);
typedef int (*GeneratorCallback) ();
// Declarations of the test functions.
clock_t Simple (int, int, GeneratorCallback, OutputCallback);
clock_t QuiteComplex (int, int, GeneratorCallback, OutputCallback);
clock_t Complex (int, int, GeneratorCallback, OutputCallback);
main.cpp:
#include "Range.h"
int
checksum;
// This callback is used to get data.
int CreateData ()
{
return rand ();
}
// This callback is used to output the results.
void OutputResults (int min, int max)
{
//cout << min << " - " << max << endl;
checksum += max - min;
}
// The program entry point.
void main ()
{
int
count = 3600000,
window = 1000;
srand (0);
checksum = 0;
std::cout << "Simple: Ticks = " << Simple (count, window, CreateData, OutputResults) << ", checksum = " << checksum << std::endl;
srand (0);
checksum = 0;
std::cout << "Quite Complex: Ticks = " << QuiteComplex (count, window, CreateData, OutputResults) << ", checksum = " << checksum << std::endl;
srand (0);
checksum = 0;
std::cout << "Complex: Ticks = " << Complex (count, window, CreateData, OutputResults) << ", checksum = " << checksum << std::endl;
}
Simple.cpp:
#include "Range.h"
// Function to actually process the data.
// A circular buffer of min/max values for the current window is filled
// and once full, the oldest min/max pair is sent to the output callback
// and replaced with the newest input value. Each value inputted is
// compared against all min/max pairs.
void ProcessData
(
int count,
int window,
GeneratorCallback input,
OutputCallback output,
int *min_buffer,
int *max_buffer
)
{
int
i;
for (i = 0 ; i < window ; ++i)
{
int
value = input ();
min_buffer [i] = max_buffer [i] = value;
for (int j = 0 ; j < i ; ++j)
{
min_buffer [j] = min (min_buffer [j], value);
max_buffer [j] = max (max_buffer [j], value);
}
}
for ( ; i < count ; ++i)
{
int
index = i % window;
output (min_buffer [index], max_buffer [index]);
int
value = input ();
min_buffer [index] = max_buffer [index] = value;
for (int k = (i + 1) % window ; k != index ; k = (k + 1) % window)
{
min_buffer [k] = min (min_buffer [k], value);
max_buffer [k] = max (max_buffer [k], value);
}
}
output (min_buffer [count % window], max_buffer [count % window]);
}
// A simple method of calculating the results.
// Memory management is done here outside of the timing portion.
clock_t Simple
(
int count,
int window,
GeneratorCallback input,
OutputCallback output
)
{
int
*min_buffer = new int [window],
*max_buffer = new int [window];
clock_t
start = clock ();
ProcessData (count, window, input, output, min_buffer, max_buffer);
clock_t
end = clock ();
delete [] max_buffer;
delete [] min_buffer;
return end - start;
}
QuiteComplex.cpp:
#include "Range.h"
template <class T>
class Range
{
private:
// Class Types
// Node Data
// Stores a value and its position in various lists.
struct Node
{
Node
*m_queue_next,
*m_list_greater,
*m_list_lower;
T
m_value;
};
public:
// Constructor
// Allocates memory for the node data and adds all the allocated
// nodes to the unused/free list of nodes.
Range
(
int window_size
) :
m_nodes (new Node [window_size]),
m_queue_tail (m_nodes),
m_queue_head (0),
m_list_min (0),
m_list_max (0),
m_free_list (m_nodes)
{
for (int i = 0 ; i < window_size - 1 ; ++i)
{
m_nodes [i].m_list_lower = &m_nodes [i + 1];
}
m_nodes [window_size - 1].m_list_lower = 0;
}
// Destructor
// Tidy up allocated data.
~Range ()
{
delete [] m_nodes;
}
// Function to add a new value into the data structure.
void AddValue
(
T value
)
{
Node
*node = GetNode ();
// clear links
node->m_queue_next = 0;
// set value of node
node->m_value = value;
// find place to add node into linked list
Node
*search;
for (search = m_list_max ; search ; search = search->m_list_lower)
{
if (search->m_value < value)
{
if (search->m_list_greater)
{
node->m_list_greater = search->m_list_greater;
search->m_list_greater->m_list_lower = node;
}
else
{
m_list_max = node;
}
node->m_list_lower = search;
search->m_list_greater = node;
}
}
if (!search)
{
m_list_min->m_list_lower = node;
node->m_list_greater = m_list_min;
m_list_min = node;
}
}
// Accessor to determine if the first output value is ready for use.
bool RangeAvailable ()
{
return !m_free_list;
}
// Accessor to get the minimum value of all values in the current window.
T Min ()
{
return m_list_min->m_value;
}
// Accessor to get the maximum value of all values in the current window.
T Max ()
{
return m_list_max->m_value;
}
private:
// Function to get a node to store a value into.
// This function gets nodes from one of two places:
// 1. From the unused/free list
// 2. From the end of the fifo queue, this requires removing the node from the list and tree
Node *GetNode ()
{
Node
*node;
if (m_free_list)
{
// get new node from unused/free list and place at head
node = m_free_list;
m_free_list = node->m_list_lower;
if (m_queue_head)
{
m_queue_head->m_queue_next = node;
}
m_queue_head = node;
}
else
{
// get node from tail of queue and place at head
node = m_queue_tail;
m_queue_tail = node->m_queue_next;
m_queue_head->m_queue_next = node;
m_queue_head = node;
// remove node from linked list
if (node->m_list_lower)
{
node->m_list_lower->m_list_greater = node->m_list_greater;
}
else
{
m_list_min = node->m_list_greater;
}
if (node->m_list_greater)
{
node->m_list_greater->m_list_lower = node->m_list_lower;
}
else
{
m_list_max = node->m_list_lower;
}
}
return node;
}
// Member Data.
Node
*m_nodes,
*m_queue_tail,
*m_queue_head,
*m_list_min,
*m_list_max,
*m_free_list;
};
// A reasonable complex but more efficent method of calculating the results.
// Memory management is done here outside of the timing portion.
clock_t QuiteComplex
(
int size,
int window,
GeneratorCallback input,
OutputCallback output
)
{
Range <int>
range (window);
clock_t
start = clock ();
for (int i = 0 ; i < size ; ++i)
{
range.AddValue (input ());
if (range.RangeAvailable ())
{
output (range.Min (), range.Max ());
}
}
clock_t
end = clock ();
return end - start;
}
Complex.cpp:
#include "Range.h"
template <class T>
class Range
{
private:
// Class Types
// Red/Black tree node colours.
enum NodeColour
{
Red,
Black
};
// Node Data
// Stores a value and its position in various lists and trees.
struct Node
{
// Function to get the sibling of a node.
// Because leaves are stored as null pointers, it must be possible
// to get the sibling of a null pointer. If the object is a null pointer
// then the parent pointer is used to determine the sibling.
Node *Sibling
(
Node *parent
)
{
Node
*sibling;
if (this)
{
sibling = m_tree_parent->m_tree_less == this ? m_tree_parent->m_tree_more : m_tree_parent->m_tree_less;
}
else
{
sibling = parent->m_tree_less ? parent->m_tree_less : parent->m_tree_more;
}
return sibling;
}
// Node Members
Node
*m_queue_next,
*m_tree_less,
*m_tree_more,
*m_tree_parent,
*m_list_greater,
*m_list_lower;
NodeColour
m_colour;
T
m_value;
};
public:
// Constructor
// Allocates memory for the node data and adds all the allocated
// nodes to the unused/free list of nodes.
Range
(
int window_size
) :
m_nodes (new Node [window_size]),
m_queue_tail (m_nodes),
m_queue_head (0),
m_tree_root (0),
m_list_min (0),
m_list_max (0),
m_free_list (m_nodes)
{
for (int i = 0 ; i < window_size - 1 ; ++i)
{
m_nodes [i].m_list_lower = &m_nodes [i + 1];
}
m_nodes [window_size - 1].m_list_lower = 0;
}
// Destructor
// Tidy up allocated data.
~Range ()
{
delete [] m_nodes;
}
// Function to add a new value into the data structure.
void AddValue
(
T value
)
{
Node
*node = GetNode ();
// clear links
node->m_queue_next = node->m_tree_more = node->m_tree_less = node->m_tree_parent = 0;
// set value of node
node->m_value = value;
// insert node into tree
if (m_tree_root)
{
InsertNodeIntoTree (node);
BalanceTreeAfterInsertion (node);
}
else
{
m_tree_root = m_list_max = m_list_min = node;
node->m_tree_parent = node->m_list_greater = node->m_list_lower = 0;
}
m_tree_root->m_colour = Black;
}
// Accessor to determine if the first output value is ready for use.
bool RangeAvailable ()
{
return !m_free_list;
}
// Accessor to get the minimum value of all values in the current window.
T Min ()
{
return m_list_min->m_value;
}
// Accessor to get the maximum value of all values in the current window.
T Max ()
{
return m_list_max->m_value;
}
private:
// Function to get a node to store a value into.
// This function gets nodes from one of two places:
// 1. From the unused/free list
// 2. From the end of the fifo queue, this requires removing the node from the list and tree
Node *GetNode ()
{
Node
*node;
if (m_free_list)
{
// get new node from unused/free list and place at head
node = m_free_list;
m_free_list = node->m_list_lower;
if (m_queue_head)
{
m_queue_head->m_queue_next = node;
}
m_queue_head = node;
}
else
{
// get node from tail of queue and place at head
node = m_queue_tail;
m_queue_tail = node->m_queue_next;
m_queue_head->m_queue_next = node;
m_queue_head = node;
// remove node from tree
node = RemoveNodeFromTree (node);
RebalanceTreeAfterDeletion (node);
// remove node from linked list
if (node->m_list_lower)
{
node->m_list_lower->m_list_greater = node->m_list_greater;
}
else
{
m_list_min = node->m_list_greater;
}
if (node->m_list_greater)
{
node->m_list_greater->m_list_lower = node->m_list_lower;
}
else
{
m_list_max = node->m_list_lower;
}
}
return node;
}
// Rebalances the tree after insertion
void BalanceTreeAfterInsertion
(
Node *node
)
{
node->m_colour = Red;
while (node != m_tree_root && node->m_tree_parent->m_colour == Red)
{
if (node->m_tree_parent == node->m_tree_parent->m_tree_parent->m_tree_more)
{
Node
*uncle = node->m_tree_parent->m_tree_parent->m_tree_less;
if (uncle && uncle->m_colour == Red)
{
node->m_tree_parent->m_colour = Black;
uncle->m_colour = Black;
node->m_tree_parent->m_tree_parent->m_colour = Red;
node = node->m_tree_parent->m_tree_parent;
}
else
{
if (node == node->m_tree_parent->m_tree_less)
{
node = node->m_tree_parent;
LeftRotate (node);
}
node->m_tree_parent->m_colour = Black;
node->m_tree_parent->m_tree_parent->m_colour = Red;
RightRotate (node->m_tree_parent->m_tree_parent);
}
}
else
{
Node
*uncle = node->m_tree_parent->m_tree_parent->m_tree_more;
if (uncle && uncle->m_colour == Red)
{
node->m_tree_parent->m_colour = Black;
uncle->m_colour = Black;
node->m_tree_parent->m_tree_parent->m_colour = Red;
node = node->m_tree_parent->m_tree_parent;
}
else
{
if (node == node->m_tree_parent->m_tree_more)
{
node = node->m_tree_parent;
RightRotate (node);
}
node->m_tree_parent->m_colour = Black;
node->m_tree_parent->m_tree_parent->m_colour = Red;
LeftRotate (node->m_tree_parent->m_tree_parent);
}
}
}
}
// Adds a node into the tree and sorted linked list
void InsertNodeIntoTree
(
Node *node
)
{
Node
*parent = 0,
*child = m_tree_root;
bool
greater;
while (child)
{
parent = child;
child = (greater = node->m_value > child->m_value) ? child->m_tree_more : child->m_tree_less;
}
node->m_tree_parent = parent;
if (greater)
{
parent->m_tree_more = node;
// insert node into linked list
if (parent->m_list_greater)
{
parent->m_list_greater->m_list_lower = node;
}
else
{
m_list_max = node;
}
node->m_list_greater = parent->m_list_greater;
node->m_list_lower = parent;
parent->m_list_greater = node;
}
else
{
parent->m_tree_less = node;
// insert node into linked list
if (parent->m_list_lower)
{
parent->m_list_lower->m_list_greater = node;
}
else
{
m_list_min = node;
}
node->m_list_lower = parent->m_list_lower;
node->m_list_greater = parent;
parent->m_list_lower = node;
}
}
// Red/Black tree manipulation routine, used for removing a node
Node *RemoveNodeFromTree
(
Node *node
)
{
if (node->m_tree_less && node->m_tree_more)
{
// the complex case, swap node with a child node
Node
*child;
if (node->m_tree_less)
{
// find largest value in lesser half (node with no greater pointer)
for (child = node->m_tree_less ; child->m_tree_more ; child = child->m_tree_more)
{
}
}
else
{
// find smallest value in greater half (node with no lesser pointer)
for (child = node->m_tree_more ; child->m_tree_less ; child = child->m_tree_less)
{
}
}
swap (child->m_colour, node->m_colour);
if (child->m_tree_parent != node)
{
swap (child->m_tree_less, node->m_tree_less);
swap (child->m_tree_more, node->m_tree_more);
swap (child->m_tree_parent, node->m_tree_parent);
if (!child->m_tree_parent)
{
m_tree_root = child;
}
else
{
if (child->m_tree_parent->m_tree_less == node)
{
child->m_tree_parent->m_tree_less = child;
}
else
{
child->m_tree_parent->m_tree_more = child;
}
}
if (node->m_tree_parent->m_tree_less == child)
{
node->m_tree_parent->m_tree_less = node;
}
else
{
node->m_tree_parent->m_tree_more = node;
}
}
else
{
child->m_tree_parent = node->m_tree_parent;
node->m_tree_parent = child;
Node
*child_less = child->m_tree_less,
*child_more = child->m_tree_more;
if (node->m_tree_less == child)
{
child->m_tree_less = node;
child->m_tree_more = node->m_tree_more;
node->m_tree_less = child_less;
node->m_tree_more = child_more;
}
else
{
child->m_tree_less = node->m_tree_less;
child->m_tree_more = node;
node->m_tree_less = child_less;
node->m_tree_more = child_more;
}
if (!child->m_tree_parent)
{
m_tree_root = child;
}
else
{
if (child->m_tree_parent->m_tree_less == node)
{
child->m_tree_parent->m_tree_less = child;
}
else
{
child->m_tree_parent->m_tree_more = child;
}
}
}
if (child->m_tree_less)
{
child->m_tree_less->m_tree_parent = child;
}
if (child->m_tree_more)
{
child->m_tree_more->m_tree_parent = child;
}
if (node->m_tree_less)
{
node->m_tree_less->m_tree_parent = node;
}
if (node->m_tree_more)
{
node->m_tree_more->m_tree_parent = node;
}
}
Node
*child = node->m_tree_less ? node->m_tree_less : node->m_tree_more;
if (node->m_tree_parent->m_tree_less == node)
{
node->m_tree_parent->m_tree_less = child;
}
else
{
node->m_tree_parent->m_tree_more = child;
}
if (child)
{
child->m_tree_parent = node->m_tree_parent;
}
return node;
}
// Red/Black tree manipulation routine, used for rebalancing a tree after a deletion
void RebalanceTreeAfterDeletion
(
Node *node
)
{
Node
*child = node->m_tree_less ? node->m_tree_less : node->m_tree_more;
if (node->m_colour == Black)
{
if (child && child->m_colour == Red)
{
child->m_colour = Black;
}
else
{
Node
*parent = node->m_tree_parent,
*n = child;
while (parent)
{
Node
*sibling = n->Sibling (parent);
if (sibling && sibling->m_colour == Red)
{
parent->m_colour = Red;
sibling->m_colour = Black;
if (n == parent->m_tree_more)
{
LeftRotate (parent);
}
else
{
RightRotate (parent);
}
}
sibling = n->Sibling (parent);
if (parent->m_colour == Black &&
sibling->m_colour == Black &&
(!sibling->m_tree_more || sibling->m_tree_more->m_colour == Black) &&
(!sibling->m_tree_less || sibling->m_tree_less->m_colour == Black))
{
sibling->m_colour = Red;
n = parent;
parent = n->m_tree_parent;
continue;
}
else
{
if (parent->m_colour == Red &&
sibling->m_colour == Black &&
(!sibling->m_tree_more || sibling->m_tree_more->m_colour == Black) &&
(!sibling->m_tree_less || sibling->m_tree_less->m_colour == Black))
{
sibling->m_colour = Red;
parent->m_colour = Black;
break;
}
else
{
if (n == parent->m_tree_more &&
sibling->m_colour == Black &&
(sibling->m_tree_more && sibling->m_tree_more->m_colour == Red) &&
(!sibling->m_tree_less || sibling->m_tree_less->m_colour == Black))
{
sibling->m_colour = Red;
sibling->m_tree_more->m_colour = Black;
RightRotate (sibling);
}
else
{
if (n == parent->m_tree_less &&
sibling->m_colour == Black &&
(!sibling->m_tree_more || sibling->m_tree_more->m_colour == Black) &&
(sibling->m_tree_less && sibling->m_tree_less->m_colour == Red))
{
sibling->m_colour = Red;
sibling->m_tree_less->m_colour = Black;
LeftRotate (sibling);
}
}
sibling = n->Sibling (parent);
sibling->m_colour = parent->m_colour;
parent->m_colour = Black;
if (n == parent->m_tree_more)
{
sibling->m_tree_less->m_colour = Black;
LeftRotate (parent);
}
else
{
sibling->m_tree_more->m_colour = Black;
RightRotate (parent);
}
break;
}
}
}
}
}
}
// Red/Black tree manipulation routine, used for balancing the tree
void LeftRotate
(
Node *node
)
{
Node
*less = node->m_tree_less;
node->m_tree_less = less->m_tree_more;
if (less->m_tree_more)
{
less->m_tree_more->m_tree_parent = node;
}
less->m_tree_parent = node->m_tree_parent;
if (!node->m_tree_parent)
{
m_tree_root = less;
}
else
{
if (node == node->m_tree_parent->m_tree_more)
{
node->m_tree_parent->m_tree_more = less;
}
else
{
node->m_tree_parent->m_tree_less = less;
}
}
less->m_tree_more = node;
node->m_tree_parent = less;
}
// Red/Black tree manipulation routine, used for balancing the tree
void RightRotate
(
Node *node
)
{
Node
*more = node->m_tree_more;
node->m_tree_more = more->m_tree_less;
if (more->m_tree_less)
{
more->m_tree_less->m_tree_parent = node;
}
more->m_tree_parent = node->m_tree_parent;
if (!node->m_tree_parent)
{
m_tree_root = more;
}
else
{
if (node == node->m_tree_parent->m_tree_less)
{
node->m_tree_parent->m_tree_less = more;
}
else
{
node->m_tree_parent->m_tree_more = more;
}
}
more->m_tree_less = node;
node->m_tree_parent = more;
}
// Member Data.
Node
*m_nodes,
*m_queue_tail,
*m_queue_head,
*m_tree_root,
*m_list_min,
*m_list_max,
*m_free_list;
};
// A complex but more efficent method of calculating the results.
// Memory management is done here outside of the timing portion.
clock_t Complex
(
int count,
int window,
GeneratorCallback input,
OutputCallback output
)
{
Range <int>
range (window);
clock_t
start = clock ();
for (int i = 0 ; i < count ; ++i)
{
range.AddValue (input ());
if (range.RangeAvailable ())
{
output (range.Min (), range.Max ());
}
}
clock_t
end = clock ();
return end - start;
}
Idea of algorithm:
Take the first 1000 values of data and sort them
The last in the sort - the first is range(data + 0, data + 999).
Then remove from the sort pile the first element with the value data[0]
and add the element data[1000]
Now, the last in the sort - the first is range(data + 1, data + 1000).
Repeat until done
// This should run in (DATA_LEN - RANGE_WIDTH)log(RANGE_WIDTH)
#include <set>
#include <algorithm>
using namespace std;
const int DATA_LEN = 3600000;
double* const data = new double (DATA_LEN);
....
....
const int RANGE_WIDTH = 1000;
double range = new double(DATA_LEN - RANGE_WIDTH);
multiset<double> data_set;
data_set.insert(data[i], data[RANGE_WIDTH]);
for (int i = 0 ; i < DATA_LEN - RANGE_WIDTH - 1 ; i++)
{
range[i] = *data_set.end() - *data_set.begin();
multiset<double>::iterator iter = data_set.find(data[i]);
data_set.erase(iter);
data_set.insert(data[i+1]);
}
range[i] = *data_set.end() - *data_set.begin();
// range now holds the values you seek
You should probably check this for off by 1 errors, but the idea is there.