ConcurrentHashMap in JDK7 code explanation (scanAndLockForPut) - concurrency

The source codes of the method scanAndLockForPut in ConcurrentHashMap in JDK7 says:
private HashEntry<K,V> scanAndLockForPut(K key, int hash, V value) {
HashEntry<K,V> first = entryForHash(this, hash);
HashEntry<K,V> e = first;
HashEntry<K,V> node = null;
int retries = -1; // negative while locating node
while (!tryLock()) {
HashEntry<K,V> f; // to recheck first below
if (retries < 0) {
if (e == null) {
if (node == null) // speculatively create node
node = new HashEntry<K,V>(hash, key, value, null);
retries = 0;
}
else if (key.equals(e.key))
retries = 0;
else
e = e.next;
}
else if (++retries > MAX_SCAN_RETRIES) {
lock();
break;
}
else if ((retries & 1) == 0 &&
(f = entryForHash(this, hash)) != first) {
e = first = f; // re-traverse if entry changed
retries = -1;
}
}
return node;
}
I understand what the codes mean, but what I don't is this else if entry:
else if ((retries & 1) == 0 && (f = entryForHash(this, hash)) != first)
My question is:
Why do we have to do "(retries & 1) == 0"?
EDIT:
I kind of figure it out. It's all because the constant MAX_SCAN_RETRIES:
static final int MAX_SCAN_RETRIES = Runtime.getRuntime().availableProcessors() > 1 ? 64 : 1;
In single core processor, MAX_SCAN_RETRIES = 1. So the second time the thread steps into the loop "while(tryLock)", it doesn't have to check whether the first node was changed.
However, in multi cores processor, this will behave like checking whether the first node is changed every 2 times in the while loop.
Is the above explanation correct?

Let's break this down:
1:
(retries & 1) == 0
This returns 1 for odd numbers, 0 for even numbers. Basically, to get past, there's a 1 in 2 chance, if the number is even.
2:
f = entryForHash(this, hash)
f is a temporary variable used to store the value of the latest entry in the segment.
3:
(/* ... */) != first
Checks if the value changed. If it did, it would move the current entry to the start, and re-iterate the linked nodes again in attempt to acquire the lock.

I've asked this question on the concurrency-interest mailing list, and the author(Doug Lea) himself replied:
Yes. We need only ensure that staleness is eventually detected.
Alternating the head-checks works fine, and simplifies use of
the same code for both uni- and multi- processors.
link
So I think this is the end of this question.

I think there are some bugs for the method!
first let us see the put method:
final V put(K key, int hash, V value, boolean onlyIfAbsent) {
HashEntry<K,V> node = tryLock() ? null :
scanAndLockForPut(key, hash, value);//1. scanAndLockForPut only return
// null or a new Entry
V oldValue;
try {
HashEntry<K,V>[] tab = table;
int index = (tab.length - 1) & hash;
HashEntry<K,V> first = entryAt(tab, index);
for (HashEntry<K,V> e = first;;) {
if (e != null) {
K k;
if ((k = e.key) == key ||
(e.hash == hash && key.equals(k))) {
oldValue = e.value;
if (!onlyIfAbsent) {
e.value = value;
++modCount;
}
break;
}
e = e.next;
}
else {
// 2. here the node is null or a new Entry
// and the node.next is the origin head node
if (node != null)
node.setNext(first);
else
node = new HashEntry<K,V>(hash, key, value, first);
int c = count + 1;
if (c > threshold && tab.length < MAXIMUM_CAPACITY)
rehash(node);
else
setEntryAt(tab, index, node);//3. finally, the node become
// the new head,so eventually
// every thing we put will be
// the head of the entry list
// and it may appears two equals
// entry in the same entry list.
++modCount;
count = c;
oldValue = null;
break;
}
}
} finally {
unlock();
}
return oldValue;
}
step: 1. scanAndLockForPut only return null or a new Entry.
step: 2. the node eventualy a new Entry, and the node.next is the origin head node
step: 3. finally, the node become the new head,so eventually every thing we put will be the head of the entry list and it may appears two equals entry in the same entry list when the concurrentHashMap works in a concurrent environment.
That is my opinion, and I am not exactly sure about whether it is right or not. So I hope you all give me some advice,thanks a lot!!

Related

How can I optimize Astar for vast empty spaces?

I am creating a game with a 3D grid for flying entities, So I have a lot of points and connections in the air where there aren't any obstructions. I didn't want to decrease the resolution of my grid so I thought I could just skip over chunks (or empties as I call them) of the Astar map while they're not containing any obstructions, and I modified Godot's Astar algorithm to do this.
Unfortunately this ended up being slower than looping through points one at a time due to the way I implemented this modification, which needs to loop through all the edge points of an empty.
2D representation of how one edge point of an empty connects to all other edge points:
This ends up looping through a larger number of points than letting the A* algorithm work it's way through the empty.
So I'm sorta stumped on how to make this more efficient while still preserving the most optimal path.
I could potentially narrow down what faces of the empty should be scanned over by first comparing the center points of all 8 faces of the empty (as my grid consists of hexagonal prisms). Or maybe I should somehow use the face center points of the empty's faces exclusively instead of all edge points.
I mainly want to know if anyone has worked on an issue like this before, and if so what would be the recommended solution?
Here is the astar loop for reference:
bool AStar::_solve(Point *begin_point, Point *end_point, int relevant_layers) {
pass++;
//make sure parallel layers are supported
// or if *relevant_layers is 0 then use all points
bool supported = relevant_layers == 0 || (relevant_layers & end_point->parallel_support_layers) > 0;
if (!end_point->enabled || !supported) {
return false;
}
bool found_route = false;
Vector<Point *> open_list;
SortArray<Point *, SortPoints> sorter;
begin_point->g_score = 0;
begin_point->f_score = _estimate_cost(begin_point->id, end_point->id);
open_list.push_back(begin_point);
while (!open_list.empty()) {
Point *p = open_list[0]; // The currently processed point
if (p == end_point) {
found_route = true;
break;
}
sorter.pop_heap(0, open_list.size(), open_list.ptrw()); // Remove the current point from the open list
open_list.remove(open_list.size() - 1);
p->closed_pass = pass; // Mark the point as closed
//if the point is part of an empty, look through all of the edge points of said empty (as to skip over any points within the empty).
OAHashMap<int, Point*> connections;
PoolVector<Empty*> enabled_empties;
int size = p->empties.size();
PoolVector<Empty*>::Read r = p->empties.read();
for (int i = 0; i < size; i++) {
Empty* e = r[i];
supported = relevant_layers == 0 || (relevant_layers & e->parallel_support_layers) > 0;
//if the empty is enabled and the end point is not within the empty
if (e->enabled && supported && !end_point->empties.has(e)) {
enabled_empties.append(e);
//can travel to any edge point
for (OAHashMap<int, Point*>::Iterator it = e->edge_points.iter(); it.valid; it = e->edge_points.next_iter(it)) {
int id = *it.key;
Point* ep = *(it.value);
ep->is_neighbour = false;
//don't connect to the same point
if (id != p->id && (i == 0 || !connections.has(id))) {
connections.set(id, ep);
}
}
}
}
//add neighbours to connections
for (OAHashMap<int, Point*>::Iterator it = p->neighbours.iter(); it.valid; it = p->neighbours.next_iter(it)) {
int id = *it.key;
Point* np = *(it.value);// The neighbour point
np->is_neighbour = true;
//don't need to check for duplicate point connections if no empties
if (size == 0 || !connections.has(id)) {
//don't add points within enabled empties since they're meant to be skipped over
if (np->empties.size() > 0 && !np->on_empty_edge) {
bool in_enabled_empty = false;
PoolVector<Empty*>::Read r1 = np->empties.read();
for (int i = 0; i < np->empties.size(); i++) {
if (enabled_empties.has(r1[i])) {
in_enabled_empty = true;
break;
}
}
if (!in_enabled_empty) {
connections.set(id, np);
}
}
else {
connections.set(id, np);
}
}
}
for (OAHashMap<int, Point *>::Iterator it = connections.iter(); it.valid; it = connections.next_iter(it)) {
Point *e = *(it.value); // The neighbour point
//make sure parallel layers are supported
// or if *relevant_layers is 0 then use all points
supported = relevant_layers == 0 || (relevant_layers & e->parallel_support_layers) > 0;
if (!e->enabled || e->closed_pass == pass || !supported) {
continue;
}
real_t tentative_g_score = p->g_score + _compute_cost(p->id, e->id) * e->weight_scale;
bool new_point = false;
if (e->open_pass != pass) { // The point wasn't inside the open list.
e->open_pass = pass;
open_list.push_back(e);
new_point = true;
} else if (tentative_g_score >= e->g_score) { // The new path is worse than the previous.
continue;
}
e->prev_point = p;
e->prev_point_connected = e->is_neighbour;
e->g_score = tentative_g_score;
e->f_score = e->g_score + _estimate_cost(e->id, end_point->id);
if (new_point) { // The position of the new points is already known.
sorter.push_heap(0, open_list.size() - 1, 0, e, open_list.ptrw());
} else {
sorter.push_heap(0, open_list.find(e), 0, e, open_list.ptrw());
}
}
}
return found_route;
}
Note: I'm still not exactly sure what the sorter does.
the entire code can be seen here in a_star.cpp and a_star.h
Edit:
if anyone wants to reference or use this, I've modified the Astar code to add user-defined octants and to use a user-defined straight line function (they are user-defined so they can work with any type of grid) to be used between octants when possible to further decrease runtime, and it works very well in terms of speed. Though the pathing is not optimal, especially when adding a lot of obstacles/restricting the available positions.

LeetCode 380: Insert Delete GetRandom O(1)

I came across this leetcode problem Insert Delete GetRandom where it is asked to implement a Data Structure to support Insert, Delete and getRandom in average O(1) time, and solved it as using map and a vector.
My solution passes all the test cases except for the last one and I'm not able to figure out why? The last test case is really very large to debug.
I changed my code a little bit and then it passes but still didn't got why the previous one didn't pass.
Non-Accepted Solution:
class RandomizedSet {
map<int, int> mp;
vector<int> v;
public:
/** Initialize your data structure here. */
RandomizedSet() {
}
/** Inserts a value to the set. Returns true if the set did not already contain the specified element. */
bool insert(int val) {
if(mp.find(val) == mp.end()){
v.push_back(val);
mp[val] = v.size()-1;
return true;
}
else return false;
}
/** Removes a value from the set. Returns true if the set contained the specified element. */
bool remove(int val) {
if(mp.find(val) == mp.end()){
return false;
}
else{
int idx = mp[val];
mp.erase(val);
swap(v[idx], v[v.size()-1]);
v.pop_back();
if(mp.size()!=0) mp[v[idx]] = idx;
return true;
}
}
/** Get a random element from the set. */
int getRandom() {
if(v.size() == 0) return 0;
int rndm = rand()%v.size();
return v[rndm];
}
};
/**
* Your RandomizedSet object will be instantiated and called as such:
* RandomizedSet* obj = new RandomizedSet();
* bool param_1 = obj->insert(val);
* bool param_2 = obj->remove(val);
* int param_3 = obj->getRandom();
*/
Accpeted Solution:
The problem is in remove function, when i change the remove function by below code, it passes.
if(mp.find(val) == mp.end()){
return false;
}
else{
int idx = mp[val];
swap(v[idx], v[v.size()-1]);
v.pop_back();
mp[v[idx]] = idx;
mp.erase(val);
return true;
}
I don't understand why is this happening. I placed the mp.erase(val) in the last and replaced the if(mp.size()!=0) mp[v[idx]] = idx to mp[v[idx]] = idx only.
Both versions of remove function are able to handle corner case - when there is only single element left in the map and we want to remove it.
LeetCode 380
This is because of undefined behavior when the element removed is the last element.
e.g, say the operations are
insert(1) // v = [1], mp = [1->0]
insert(2) // v = [1,2], mp = [1->0, 2->1]
remove(2):
int idx = mp[val]; // val = 2, idx = 1
mp.erase(val); // mp = [1->0]
swap(v[idx], v[v.size()-1]); // idx = v.size()-1 = 1, so this does nothing.
v.pop_back(); // v = [1]
if(mp.size()!=0) mp[v[idx]] = idx; // mp[v[1]] = 1.
// But v[1] is undefined after pop_back(), since v's size is 1 at this point.
I am guessing that it doesn't clear the memory location accessed by v[1], so v[1] still points to 2, and it ends up putting 2 back into mp.

Remove Duplicates from sorted list not passing all testcases

This is a question on leetcode. For some reason my code only works for 7/164 test cases. i would like to know why my algorithm is not efficient.
what is a solution to this? what is wrong with my code?
here is link to the problem
https://leetcode.com/problems/remove-duplicates-from-sorted-list/description/
edit: i have taken suggestions and edited code but my algorithm is still not passing, i am getting a time limit exceed error. is the algorithm efficient?
class Solution {
public:
ListNode* deleteDuplicates(ListNode* head) {
ListNode* current = head;
ListNode* nex;
while(current!=NULL){
nex=current->next;
if (nex==NULL)
break;
if(current->val==nex->val)
current->next=nex->next;
if (current->next==NULL)
break;
current==current->next;
if(current->next==NULL)
break;
}
return current;
}
};
nex is not initialized. You try to compare it with NULL before it is initialized.
You remove duplicates of the neighbor pairs only. It is OK for sorted lists.
You get into memory leaks.
The final two if are odd.
I think you return from the function not what is expected.
Used == where = should be used.
Your should not change current after current->next changed.
while (current && (nex = current->next)){
if (current->val == nex->val) {
current->next = nex->next;
delete nex;
} else {
current = current->next;
}
}
return head;
Remove Duplicates from Sorted List (JavaScript) -> try this one
let deleteDuplicates = (head) => {
let inArray = []
let check = head.reduce((a, c) => {
if(a[c]) a[c] = ++a[c];
else a[c] = 1;
return a
},{})
for (var i in check) {
if (check[i] == 1) inArray.push(Number(i))
}
return inArray;
}
let output = deleteDuplicates([3,7,4,4,2])
console.log(output)

C++ Boolean Value always incorrectly returns true in 2-3 Tree Search

I'm literally ripping my hair out on this one fellas. Here's the problem. I've hard coded a 2-3 Tree and verified that it works with the use of an inorder traversal function that outputs the values of the node it's currently in. So I know the tree is built correctly.
Node *r;
Node zero,one,two,three,four,five,six,seven,eight,nine,ten;
r = &zero;
//Root
zero.small = 50;
zero.large = 90;
zero.left = &one; //Child node to the left
zero.middle = &four; //Child node in the middle
zero.right = &seven; //Child node to the right
//Left Tree
one.small = 20;
one.large = NULL;
one.left = &two;
one.middle = NULL;
one.right = &three;
two.small = 10;
two.large = NULL;
two.left = NULL;
two.middle = NULL;
two.right = NULL;
three.small = 30;
three.large = 40;
three.left = NULL;
three.middle = NULL;
three.right = NULL;
//Middle Tree
four.small = 70;
four.large = NULL;
four.left = &five;
four.middle = NULL;
four.right = &six;
five.small = 60;
five.large = NULL;
five.left = NULL;
five.middle = NULL;
five.right = NULL;
six.small = 80;
six.large = NULL;
six.left = NULL;
six.middle = NULL;
six.right = NULL;
//Right Tree
seven.small = 120;
seven.large = 150;
seven.left = &eight;
seven.middle = &nine;
seven.right = &ten;
eight.small = 100;
eight.large = 110;
eight.left = NULL;
eight.middle = NULL;
eight.right = NULL;
nine.small = 130;
nine.large = 140;
nine.left = NULL;
nine.middle = NULL;
nine.right = NULL;
ten.small = 160;
ten.large = NULL;
ten.left = NULL;
ten.middle = NULL;
ten.right = NULL;
cout<<"inorder traversal for debug"<<endl;
inOrder(*r);
Output would be: 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 160
So that proves the tree is built correctly. I've been asked to modify the code to search for a value in the tree. so I wrote this function below, that's essentially the inorder traversal function minus the outputs and a simple if statement that returns TRUE if the search key is found in the tree.
bool retrieve(Node r, int key)
{
if (r.left)
retrieve(*r.left, key);
if (r.small)
{
if (r.small == key)
{
cout<<"The node: "<<r.small<<" is equal to search key: "<<key<<endl; //for debug purposes
return true;
}
}
if (r.middle)
retrieve(*r.middle, key);
if (r.large)
if (r.right)
retrieve(*r.right, key);
}
The user is prompted for a number to search for (int key), and upon entry enters an if statement
if (retrieve(*r, key))
{
cout<<key<<" is found!"<<endl;
}
else
cout<<key<<" is not found!"<<endl;
Now the problem is that this seems logically sound to me, and yet when I enter the value "85" (which is not located on the tree AT ALL), the program outputs "85 is found!". Notice how it didn't output the COUT statement I have in the function.cout<<"The node: "<<r.small<<" is equal to search key: "<<key<<endl; I've debugged and stepped through the program and no matter what the bool function (retrieve) always returns true... What? So I switched the if statement in the bool function to return false (just for debugging purposes) upon entering "60" (which IS located on the tree), the boolean function STILL returns true. I've tried several combinations of slightly different code but to no avail.. What the heck is going on??
Thanks in advance,
Tyler
You never return a value, except in the if (r.small == key) branch.
From 2–3 tree - Wikipedia, I would say your code should compare the key with the small and large key first and depending on the comparison return the result from retrieve(*r.left/middle/right, key).
Something along these lines (untested)
if (key < r.small)
return retrieve(*r.small, key);
if (key == r.small)
return TRUE;
if (r.right == NULL)
return retrieve(*r.middle, key);
if (key < r.large)
return retrieve(*r.middle, key);
if (key == r.large)
return TRUE;
return retrieve(*r.right, key);
You need to first check if the key is found in the current node in either small or large, and if it is, return true. if it is not you need to recursively call retrieve on each of the contained nodes, and if any of them return true, return true. If your function has not returned yet you need to return false.
You need an initial test to see if the recursion should stop because you are at a least node.
// precondition: current is not 0
// returns: true or false. If true, location is set to the node
// where it was found.
bool DoSearch(Node *current, int key, Node *location)
{
/*
* Is key in current?
*/
if (current->smallValue == key || (current->isThreeNode()
&& current->largeValue == key)) {
location = current;
return true;
} else if ((current->isLeafNode())) {
location = current;
return false;
/*
* Does current have two keys?
*/
} else if (current->isThreeNode()){
if (key < current->smallValue) {
DoSearch(key, current->leftChild, location);
} else if (key < current->largeValue) {
DoSearch(key, current->middleChild, location);
} else {
DoSearch(key, current->rightChild, location);
}
} else { // ...or only one?
if (key < current->smallValue) {
DoSearch(key, current->leftChild, location);
} else {
DoSearch(key, current->rightChild, location);
}
}
}

Recursive binary tree function

I am trying to insert items from my array into my tree. My function works fine and creates nodes all the way down to the leaf node on the left hand side of my tree. The problem is when it is supposed to return recursively when detecting a leaf node to the higher level, it just stops building the tree completely. Here is the code:
void WLD::treeInsert(BSP_Node *tree_root, int node_number)
{
if ( tree_root == NULL )
{
tree_root = new BSP_Node();
tree_root->node_number = node_number;
tree_root->normalX = bsp_array[node_number].normal[0];
tree_root->normalY = bsp_array[node_number].normal[1];
tree_root->normalZ = bsp_array[node_number].normal[2];
tree_root->splitdistance = bsp_array[node_number].splitdistance;;
tree_root->region = bsp_array[node_number].region;
tree_root->left = bsp_array[node_number].left; //because the array starts at index 0
tree_root->right = bsp_array[node_number].right; //because the array starts at index 0
tree_root->left_node = NULL;
tree_root->right_node = NULL;
errorLog.OutputSuccess("Inserting new node: %i", node_number);
errorLog.OutputSuccess("Left node index: %i", bsp_array[node_number].left);
errorLog.OutputSuccess("Right node index: %i", bsp_array[node_number].right);
node_number++;
// Check for leaf nodes
if(tree_root->region != 0)
{
errorLog.OutputSuccess("This is a leaf node! Returning!");
return;
}
}
if ( tree_root->left > 0)
{
//tree_root->left_node = new BSP_Node();
errorLog.OutputSuccess("Left node not NULL, inserting it!");
treeInsert( tree_root->left_node, tree_root->left );
}
else if (tree_root->right > 0)
{
//tree_root->right_node = new BSP_Node();
errorLog.OutputSuccess("Right node not NULL, inserting it!");
treeInsert( tree_root->right_node = NULL, tree_root->right );
}
}
As you can see, when it detects a leaf node, it is supposed to return to the calling function (this function but on a level closer to the node. Does anyone have any suggestions?
if ( tree_root->left > 0) {
// implementation
}
else if (tree_root->right > 0) {
// implementation
}
Shouldn't this be two separate if statements, rather than if/else? Otherwise it only does one or the other sides, but not both.