LeetCode 380: Insert Delete GetRandom O(1) - c++

I came across this leetcode problem Insert Delete GetRandom where it is asked to implement a Data Structure to support Insert, Delete and getRandom in average O(1) time, and solved it as using map and a vector.
My solution passes all the test cases except for the last one and I'm not able to figure out why? The last test case is really very large to debug.
I changed my code a little bit and then it passes but still didn't got why the previous one didn't pass.
Non-Accepted Solution:
class RandomizedSet {
map<int, int> mp;
vector<int> v;
public:
/** Initialize your data structure here. */
RandomizedSet() {
}
/** Inserts a value to the set. Returns true if the set did not already contain the specified element. */
bool insert(int val) {
if(mp.find(val) == mp.end()){
v.push_back(val);
mp[val] = v.size()-1;
return true;
}
else return false;
}
/** Removes a value from the set. Returns true if the set contained the specified element. */
bool remove(int val) {
if(mp.find(val) == mp.end()){
return false;
}
else{
int idx = mp[val];
mp.erase(val);
swap(v[idx], v[v.size()-1]);
v.pop_back();
if(mp.size()!=0) mp[v[idx]] = idx;
return true;
}
}
/** Get a random element from the set. */
int getRandom() {
if(v.size() == 0) return 0;
int rndm = rand()%v.size();
return v[rndm];
}
};
/**
* Your RandomizedSet object will be instantiated and called as such:
* RandomizedSet* obj = new RandomizedSet();
* bool param_1 = obj->insert(val);
* bool param_2 = obj->remove(val);
* int param_3 = obj->getRandom();
*/
Accpeted Solution:
The problem is in remove function, when i change the remove function by below code, it passes.
if(mp.find(val) == mp.end()){
return false;
}
else{
int idx = mp[val];
swap(v[idx], v[v.size()-1]);
v.pop_back();
mp[v[idx]] = idx;
mp.erase(val);
return true;
}
I don't understand why is this happening. I placed the mp.erase(val) in the last and replaced the if(mp.size()!=0) mp[v[idx]] = idx to mp[v[idx]] = idx only.
Both versions of remove function are able to handle corner case - when there is only single element left in the map and we want to remove it.
LeetCode 380

This is because of undefined behavior when the element removed is the last element.
e.g, say the operations are
insert(1) // v = [1], mp = [1->0]
insert(2) // v = [1,2], mp = [1->0, 2->1]
remove(2):
int idx = mp[val]; // val = 2, idx = 1
mp.erase(val); // mp = [1->0]
swap(v[idx], v[v.size()-1]); // idx = v.size()-1 = 1, so this does nothing.
v.pop_back(); // v = [1]
if(mp.size()!=0) mp[v[idx]] = idx; // mp[v[1]] = 1.
// But v[1] is undefined after pop_back(), since v's size is 1 at this point.
I am guessing that it doesn't clear the memory location accessed by v[1], so v[1] still points to 2, and it ends up putting 2 back into mp.

Related

How can I optimize Astar for vast empty spaces?

I am creating a game with a 3D grid for flying entities, So I have a lot of points and connections in the air where there aren't any obstructions. I didn't want to decrease the resolution of my grid so I thought I could just skip over chunks (or empties as I call them) of the Astar map while they're not containing any obstructions, and I modified Godot's Astar algorithm to do this.
Unfortunately this ended up being slower than looping through points one at a time due to the way I implemented this modification, which needs to loop through all the edge points of an empty.
2D representation of how one edge point of an empty connects to all other edge points:
This ends up looping through a larger number of points than letting the A* algorithm work it's way through the empty.
So I'm sorta stumped on how to make this more efficient while still preserving the most optimal path.
I could potentially narrow down what faces of the empty should be scanned over by first comparing the center points of all 8 faces of the empty (as my grid consists of hexagonal prisms). Or maybe I should somehow use the face center points of the empty's faces exclusively instead of all edge points.
I mainly want to know if anyone has worked on an issue like this before, and if so what would be the recommended solution?
Here is the astar loop for reference:
bool AStar::_solve(Point *begin_point, Point *end_point, int relevant_layers) {
pass++;
//make sure parallel layers are supported
// or if *relevant_layers is 0 then use all points
bool supported = relevant_layers == 0 || (relevant_layers & end_point->parallel_support_layers) > 0;
if (!end_point->enabled || !supported) {
return false;
}
bool found_route = false;
Vector<Point *> open_list;
SortArray<Point *, SortPoints> sorter;
begin_point->g_score = 0;
begin_point->f_score = _estimate_cost(begin_point->id, end_point->id);
open_list.push_back(begin_point);
while (!open_list.empty()) {
Point *p = open_list[0]; // The currently processed point
if (p == end_point) {
found_route = true;
break;
}
sorter.pop_heap(0, open_list.size(), open_list.ptrw()); // Remove the current point from the open list
open_list.remove(open_list.size() - 1);
p->closed_pass = pass; // Mark the point as closed
//if the point is part of an empty, look through all of the edge points of said empty (as to skip over any points within the empty).
OAHashMap<int, Point*> connections;
PoolVector<Empty*> enabled_empties;
int size = p->empties.size();
PoolVector<Empty*>::Read r = p->empties.read();
for (int i = 0; i < size; i++) {
Empty* e = r[i];
supported = relevant_layers == 0 || (relevant_layers & e->parallel_support_layers) > 0;
//if the empty is enabled and the end point is not within the empty
if (e->enabled && supported && !end_point->empties.has(e)) {
enabled_empties.append(e);
//can travel to any edge point
for (OAHashMap<int, Point*>::Iterator it = e->edge_points.iter(); it.valid; it = e->edge_points.next_iter(it)) {
int id = *it.key;
Point* ep = *(it.value);
ep->is_neighbour = false;
//don't connect to the same point
if (id != p->id && (i == 0 || !connections.has(id))) {
connections.set(id, ep);
}
}
}
}
//add neighbours to connections
for (OAHashMap<int, Point*>::Iterator it = p->neighbours.iter(); it.valid; it = p->neighbours.next_iter(it)) {
int id = *it.key;
Point* np = *(it.value);// The neighbour point
np->is_neighbour = true;
//don't need to check for duplicate point connections if no empties
if (size == 0 || !connections.has(id)) {
//don't add points within enabled empties since they're meant to be skipped over
if (np->empties.size() > 0 && !np->on_empty_edge) {
bool in_enabled_empty = false;
PoolVector<Empty*>::Read r1 = np->empties.read();
for (int i = 0; i < np->empties.size(); i++) {
if (enabled_empties.has(r1[i])) {
in_enabled_empty = true;
break;
}
}
if (!in_enabled_empty) {
connections.set(id, np);
}
}
else {
connections.set(id, np);
}
}
}
for (OAHashMap<int, Point *>::Iterator it = connections.iter(); it.valid; it = connections.next_iter(it)) {
Point *e = *(it.value); // The neighbour point
//make sure parallel layers are supported
// or if *relevant_layers is 0 then use all points
supported = relevant_layers == 0 || (relevant_layers & e->parallel_support_layers) > 0;
if (!e->enabled || e->closed_pass == pass || !supported) {
continue;
}
real_t tentative_g_score = p->g_score + _compute_cost(p->id, e->id) * e->weight_scale;
bool new_point = false;
if (e->open_pass != pass) { // The point wasn't inside the open list.
e->open_pass = pass;
open_list.push_back(e);
new_point = true;
} else if (tentative_g_score >= e->g_score) { // The new path is worse than the previous.
continue;
}
e->prev_point = p;
e->prev_point_connected = e->is_neighbour;
e->g_score = tentative_g_score;
e->f_score = e->g_score + _estimate_cost(e->id, end_point->id);
if (new_point) { // The position of the new points is already known.
sorter.push_heap(0, open_list.size() - 1, 0, e, open_list.ptrw());
} else {
sorter.push_heap(0, open_list.find(e), 0, e, open_list.ptrw());
}
}
}
return found_route;
}
Note: I'm still not exactly sure what the sorter does.
the entire code can be seen here in a_star.cpp and a_star.h
Edit:
if anyone wants to reference or use this, I've modified the Astar code to add user-defined octants and to use a user-defined straight line function (they are user-defined so they can work with any type of grid) to be used between octants when possible to further decrease runtime, and it works very well in terms of speed. Though the pathing is not optimal, especially when adding a lot of obstacles/restricting the available positions.

How Does std::forward_list::sort Work in NlogN Time?

I can't imagine how to reorder a singly linked list with decent time complexity (The library says it takes "approximately" NlogN). Is there a name for the algorithm used that I could use to find educational material about it? I looked at the code in the standard library, but I couldn't figure much out other than a merge takes place near the end of one of the few functions named sort or sort2. Below are some of the functions used:
template <class _Pr2>
static void _Sort(_Nodeptr _BFirst, _Pr2 _Pred) {
auto _BMid = _Sort2(_BFirst, _Pred);
size_type _Bound = 2;
do {
if (!_BMid->_Next) {
return;
}
const auto _BLast = _Sort(_BMid, _Bound, _Pred);
_BMid = _Inplace_merge(_BFirst, _BMid, _BLast, _Pred);
_Bound <<= 1;
} while (_Bound != 0);
}
template <class _Pr2>
static _Nodeptr _Sort(const _Nodeptr _BFirst, size_type _Bound, _Pr2 _Pred) {
// Sort (_BFirst, _BFirst + _Bound), unless nullptr is encountered.
// Returns a pointer one before the end of the sorted region.
if (_Bound <= 2) {
return _Sort2(_BFirst, _Pred);
}
const auto _Half_bound = _Bound / 2;
const auto _BMid = _Sort(_BFirst, _Half_bound, _Pred);
if (!_BMid->_Next) {
return _BMid;
}
const auto _BLast = _Sort(_BMid, _Half_bound, _Pred);
return _Inplace_merge(_BFirst, _BMid, _BLast, _Pred);
}
template <class _Pr2>
static _Nodeptr _Inplace_merge(_Nodeptr _BFirst1, const _Nodeptr _BMid, const _Nodeptr _BLast, _Pr2 _Pred) {
// Merge the sorted ranges (_BFirst1, _BMid] and (_BMid, _BLast)
// Returns one before the new logical end of the range.
auto _First2 = _BMid->_Next;
for (;;) { // process 1 splice
_Nodeptr _First1;
for (;;) { // advance _BFirst1 over elements already in position
if (_BFirst1 == _BMid) {
return _BLast;
}
_First1 = _BFirst1->_Next;
if (_DEBUG_LT_PRED(_Pred, _First2->_Myval, _First1->_Myval)) {
// _First2->_Myval is out of order
break;
}
// _First1->_Myval is already in position; advance
_BFirst1 = _First1;
}
// find the end of the "run" of elements less than _First1->_Myval in the 2nd range
auto _BRun_end = _First2;
_Nodeptr _Run_end;
for (;;) {
_Run_end = _BRun_end->_Next;
if (_BRun_end == _BLast) {
break;
}
if (!_DEBUG_LT_PRED(_Pred, _Run_end->_Myval, _First1->_Myval)) {
// _Run_end is the first element in (_BMid->_Myval, _BLast->_Myval) that shouldn't precede
// _First1->_Myval.
// After the splice _First1->_Myval will be in position and must not be compared again.
break;
}
_BRun_end = _Run_end;
}
_BMid->_Next = _Run_end; // snip out the run from its old position
_BFirst1->_Next = _First2; // insert into new position
_BRun_end->_Next = _First1;
if (_BRun_end == _BLast) {
return _BMid;
}
_BFirst1 = _First1;
_First2 = _Run_end;
}
}
"Bottom up" variants of merge sort can sort a linked list in O(n log n) time and O(1) space. See the Wikipedia article. If O(1) space isn't a requirement then you can construct an array of pointers into the list, sort that using any O(n log n) sorting algorithm, and then rebuild the list from your sorted copy.

I am getting a TLE error while performing cycle detection

I have written a code to the leetcode problem(courseSchedule) which basically asks whether a given set of courses can be done given dependencies. my approach is to create a graph and then check for a cycle, however, it's giving a TLE error. Can you help me as to why is the TLE happening or if there's a better approach that I can use ?
bool cycle( vector<vector<int>> &adj,int i,vector<bool> vis){
if(vis[i])
return true;
vis[i]=true;
for(int k=0;k<adj[i].size();k++)
if(cycle(adj,adj[i][k],vis))
return true;
return false;
}
class Solution {
public:
bool canFinish(int numCourses, vector<vector<int>>& prerequisites) {
vector<vector<int>> adj(numCourses);
for(int i=0;i<prerequisites.size();i++)
adj[prerequisites[i][1]].push_back(prerequisites[i][0]);
vector<bool> vis(numCourses,false);
for(int i=0;i<numCourses;i++)
if(cycle(adj,i,vis))
return false;
return true;
}
};
Actually, your function is correct but so inefficient.
This is because in the cycle function performs so many redundant operations i.e check for the same node multiple times.
Your Code:
bool cycle( vector<vector<int>> &adj,int i,vector<bool> vis){
if(vis[i])
return true;
vis[i] = true;
for(int k = 0; k < adj[i].size(); k++)
if(cycle(adj, adj[i][k], vis))
return true;
return false;
}
Ex:
0 ---> 1 ---> 2 ......... (some more edges)
0 ---> 3 ---> 2 ---> 4 ........ (some more edges)
So, for this graph, for the start vertex 0 (with your code) for the bool function:
iteration - 1: you perform the DFS and check for 1 and 2 and
......
iteration - 2: you perform the DFS and check for 3 and again 2 .....
So, like this, you will be recomputing the same sub-problems. To avoid this you need to put another array just check if a node is already computed.
So I have introduced another vector var (initialized to false) which basically sets to true if node is visited and got approved as non-cycle node (which doesn't involve in a cycle) .
Improved Code:
bool cycle( vector<vector<int>> &adj,int i,vector<bool> vis, vector<bool>& var){
// if i involves in cycle and visited in the current sequence
if(!var[i] and vis[i])
return true;
vis[i] = true;
for(int k=0;k<adj[i].size();k++) {
// if adj[i][k] is true i.e doesn't involve in cycle, so no need to check it. If it is false we should check it.
if(!var[adj[i][k]] and cycle(adj,adj[i][k],vis, var))
return true;
else
var[adj[i][k]] = true; // else setting true to tell it doesn't involve in cycle
}
// setting true to tell it doesn't involve in cycle
var[i] = true;
return false;
}
class Solution {
public:
bool canFinish(int numCourses, vector<vector<int>>& prerequisites) {
vector<vector<int>> adj(numCourses);
for(int i=0;i<prerequisites.size();i++)
adj[prerequisites[i][1]].push_back(prerequisites[i][0]);
vector<bool> vis(numCourses,false);
vector<bool> var(numCourses,false);
for(int i=0;i<numCourses;i++)
if(cycle(adj,i,vis, var))
return false;
return true;
}
};
Note:
I just made small changes to make your code overcome TLE without changing the basic logic. But this is still inefficient as your logic needs to pass the vector by value. I suggest you think another way :)
I also think vis is not passed by reference would be the problem for large size test cases.
This is a similar depth first search graph method, that'd pass through:
#include <cstdint>
#include <utility>
#include <vector>
const static struct Solution {
static bool canFinish(
const int num_courses,
const std::vector<std::vector<int>>& prerequisites
) {
GraphType graph = buildCourseGraph(prerequisites, num_courses);
std::vector<bool> to_take(num_courses, false);
std::vector<bool> taken(num_courses, false);
for (SizeType course = 0; course < num_courses; ++course) {
if (!taken[course] && !validateAcyclic(graph, course, to_take, taken)) {
return false;
}
}
return true;
}
private:
using GraphType = std::vector<std::vector<int>>;
using SizeType = std::uint_fast16_t;
static GraphType buildCourseGraph(
const std::vector<std::vector<int>>& prerequisites,
const SizeType num_courses
) {
GraphType graph(num_courses);
for (const auto& prerequisite : prerequisites) {
graph[prerequisite[1]].emplace_back(prerequisite[0]);
}
return graph;
}
static bool validateAcyclic(
const GraphType& graph,
const SizeType& course,
std::vector<bool>& to_take,
std::vector<bool>& taken
) {
if (to_take[course]) {
return false;
}
if (taken[course]) {
return true;
}
to_take[course] = taken[course] = true;
for (const auto& adj_course : graph[course]) {
if (!validateAcyclic(graph, adj_course, to_take, taken)) {
return false;
}
}
to_take[course] = false;
return true;
}
};
and here is LeetCode's depth first search solution in Java (with comments):
class Solution {
public boolean canFinish(int numCourses, int[][] prerequisites) {
// course -> list of next courses
HashMap<Integer, List<Integer>> courseDict = new HashMap<>();
// build the graph first
for (int[] relation : prerequisites) {
// relation[0] depends on relation[1]
if (courseDict.containsKey(relation[1])) {
courseDict.get(relation[1]).add(relation[0]);
} else {
List<Integer> nextCourses = new LinkedList<>();
nextCourses.add(relation[0]);
courseDict.put(relation[1], nextCourses);
}
}
boolean[] checked = new boolean[numCourses];
boolean[] path = new boolean[numCourses];
for (int currCourse = 0; currCourse < numCourses; ++currCourse) {
if (this.isCyclic(currCourse, courseDict, checked, path))
return false;
}
return true;
}
/*
* postorder DFS check that no cycle would be formed starting from currCourse
*/
protected boolean isCyclic(
Integer currCourse, HashMap<Integer, List<Integer>> courseDict,
boolean[] checked, boolean[] path) {
// bottom cases
if (checked[currCourse])
// this node has been checked, no cycle would be formed with this node.
return false;
if (path[currCourse])
// come across a previously visited node, i.e. detect the cycle
return true;
// no following courses, no loop.
if (!courseDict.containsKey(currCourse))
return false;
// before backtracking, mark the node in the path
path[currCourse] = true;
boolean ret = false;
// postorder DFS, to visit all its children first.
for (Integer child : courseDict.get(currCourse)) {
ret = this.isCyclic(child, courseDict, checked, path);
if (ret)
break;
}
// after the visits of children, we come back to process the node itself
// remove the node from the path
path[currCourse] = false;
// Now that we've visited the nodes in the downstream,
// we complete the check of this node.
checked[currCourse] = true;
return ret;
}
}
References
For additional details, please see the Discussion Board which you can find plenty of well-explained accepted solutions in there, with a variety of languages including efficient algorithms and asymptotic time/space complexity analysis1, 2.

Recursive insert of a chain into memory fails

This meight be a long question but i hope someone can help me figuring out whats going wrong.
I am inserting a JSON Object into already allocated Memory with my own Datatype which basically holds a Union with Data and a ptrdiff_t to the next Datatype in 8bit steps.
template <typename T>
class BaseType
{
public:
BaseType();
explicit BaseType(T& t);
explicit BaseType(const T& t);
~BaseType();
inline void setNext(const ptrdiff_t& next);
inline std::ptrdiff_t getNext();
inline void setData(T& t);
inline void setData(const T& t);
inline T getData() const;
protected:
union DataUnion
{
T data;
::std::ptrdiff_t size;
DataUnion()
{
memset(this, 0, sizeof(DataUnion));
} //init with 0
explicit DataUnion(T& t);
explicit DataUnion(const T& t);
} m_data;
long long m_next;
};
The implementation is streight so nothing special happes there just setting/getting the values of the definition. (i'll skip the impl. here)
So here starts the code where something goes wrong:
std::pair<void*, void*> Page::insertObject(const rapidjson::GenericValue<rapidjson::UTF8<>>& value,
BaseType<size_t>* last)
{
//return ptr to the first element
void* l_ret = nullptr;
//prev element ptr
BaseType<size_t>* l_prev = last;
//position pointer
void* l_pos = nullptr;
//get the members
for (auto it = value.MemberBegin(); it != value.MemberEnd(); ++it)
{
switch (it->value.GetType())
{
case rapidjson::kNullType:
LOG_WARN << "null type: " << it->name.GetString();
continue;
case rapidjson::kFalseType:
case rapidjson::kTrueType:
{
l_pos = find(sizeof(BaseType<bool>));
void* l_new = new (l_pos) BaseType<bool>(it->value.GetBool());
if (l_prev != nullptr)
l_prev->setNext(dist(l_prev, l_new));
}
break;
case rapidjson::kObjectType:
{
//pos for the obj id
//and insert the ID of the obj
l_pos = find(sizeof(BaseType<size_t>));
std::string name = it->name.GetString();
void* l_new = new (l_pos) BaseType<size_t>(common::FNVHash()(name));
if (l_prev != nullptr)
l_prev->setNext(dist(l_prev, l_new));
//TODO something strange happens here!
// pass the objid Object to the insertobj!
// now recursive insert the obj
// the second contains the last element inserted
// l_pos current contains the last inserted element and get set to the
// last element of the obj we insert
l_pos = (insertObject(it->value, reinterpret_cast<BaseType<size_t>*>(l_new)).second);
}
break;
case rapidjson::kArrayType:
{//skip this at the moment till the bug is fixed
}
break;
case rapidjson::kStringType:
{
// find pos where the string fits
// somehow we get here sometimes and it does not fit!
// which cant be since we lock the whole page
l_pos = find(sizeof(StringType) + strlen(it->value.GetString()));
//add the String Type at the pos of the FreeType
auto* l_new = new (l_pos) StringType(it->value.GetString());
if (l_prev != nullptr)
l_prev->setNext(dist(l_prev, l_new));
}
break;
case rapidjson::kNumberType:
{
//doesnt matter since long long and double are equal on x64
//find pos where the string fits
l_pos = find(sizeof(BaseType<long long>));
void* l_new;
if (it->value.IsInt())
{
//insert INT
l_new = new (l_pos) BaseType<long long>(it->value.GetInt64());
}
else
{
//INSERT DOUBLE
l_new = new (l_pos) BaseType<double>(it->value.GetDouble());
}
if (l_prev != nullptr)
l_prev->setNext(dist(l_prev, l_new));
}
break;
default:
LOG_WARN << "Unknown member Type: " << it->name.GetString() << ":" << it->value.GetType();
continue;
}
//so first element is set now, store it to return it.
if(l_ret == nullptr)
{
l_ret = l_pos;
}
//prev is the l_pos now so cast it to this;
l_prev = reinterpret_cast<BaseType<size_t>*>(l_pos);
}
//if we get here its in!
return{ l_ret, l_pos };
}
I am starting to insert like this:
auto firstElementPos = insertObject(value.MemberBegin()->value, nullptr).first;
While value.MemberBegin()->value is Object to be inserted and ->name holds the Name of the object. In the case below its Person and everything between {}.
The problem is, if i insert a JSON Object which has one Object inside like so:
"Person":
{
"age":25,
"double": 23.23,
"boolean": true,
"double2": 23.23,
"firstInnerObj":{
"innerDoub": 12.12
}
}
It works properly and i can reproduce the Object. But if i have more inner objects like so:
"Person":
{
"age":25,
"double": 23.23,
"boolean": true,
"double2": 23.23,
"firstInnerObj":{
"innerDoub": 12.12
},
"secondInnerObj":{
"secInnerDoub": 12.12
}
}
It fails and i lose data so i think that my recursion goes wrong but i dont see why. If you need any more informations let me know. Meight take a look here and the client here.
The test.json need to contain a json object like above. And the find only need to contain {"oid__":2} to get the second object that was inserted.
I could track the issue down to the Point where i recreate the Object recursively in the code. Some of the Nextpointers seem to be incorrect:
void* Page::buildObject(const size_t& hash, void* start, rapidjson::Value& l_obj,
rapidjson::MemoryPoolAllocator<>& aloc)
{
//get the meta information of the object type
//to build it
auto& l_metaIdx = meta::MetaIndex::getInstance();
//get the meta dataset
auto& l_meta = l_metaIdx[hash];
//now we are already in an object here with l_obj!
auto l_ptr = start;
for (auto it = l_meta->begin(); it != l_meta->end(); ++it)
{
//create the name value
rapidjson::Value l_name(it->name.c_str(), it->name.length(), aloc);
//create the value we are going to add
rapidjson::Value l_value;
//now start building it up again
switch (it->type)
{
case meta::OBJECT:
{
auto l_data = static_cast<BaseType<size_t>*>(l_ptr);
//get the hash to optain the metadata
auto l_hash = l_data->getData();
//set to object and create the inner object
l_value.SetObject();
//get the start pointer which is the "next" element
//and call recursive
l_ptr = static_cast<BaseType<size_t>*>(buildObject(l_hash,
(reinterpret_cast<char*>(l_data) + l_data->getNext()), l_value, aloc));
}
break;
case meta::ARRAY:
{
l_value.SetArray();
auto l_data = static_cast<ArrayType*>(l_ptr);
//get the hash to optain the metadata
auto l_size = l_data->size();
l_ptr = buildArray(l_size, static_cast<char*>(l_ptr) + l_data->getNext(), l_value, aloc);
}
break;
case meta::INT:
{
//create the data
auto l_data = static_cast<BaseType<long long>*>(l_ptr);
//with length attribute it's faster ;)
l_value = l_data->getData();
}
break;
case meta::DOUBLE:
{
//create the data
auto l_data = static_cast<BaseType<double>*>(l_ptr);
//with length attribute it's faster ;)
l_value = l_data->getData();
}
break;
case meta::STRING:
{
//create the data
auto l_data = static_cast<StringType*>(l_ptr);
//with length attribute it's faster
l_value.SetString(l_data->getString()->c_str(), l_data->getString()->length(), aloc);
}
break;
case meta::BOOL:
{
//create the data
auto l_data = static_cast<BaseType<bool>*>(l_ptr);
l_value = l_data->getData();
}
break;
default:
break;
}
l_obj.AddMember(l_name, l_value, aloc);
//update the lptr
l_ptr = static_cast<char*>(l_ptr) + static_cast<BaseType<size_t>*>(l_ptr)->getNext();
}
//return the l_ptr which current shows to the next lement. //see line above
return l_ptr;
}
After houers and houres of debugging i found the small issue which causes this. The method which builds up the Object after it was inserted returns a pointer to the actuall last element->next which was inserted and after the switch case i did call the ->next again which causes a loss of data because it scipped one element in the single chained list.
The Fix to this is to put the line
l_ptr = static_cast<char*>(l_ptr) + static_cast<BaseType<size_t>*>(l_ptr)->getNext();
Only into the switch cases where it is not an Object or Array. Fix Commit This actually also gave me the fix for an Issue with inserting Array.
Of cause the real issue could not know someone here who did not took a deep look into the code but i still want to show the fix here. Thanks to #sehe who helped alot with figuring out whats going wrong here.

ConcurrentHashMap in JDK7 code explanation (scanAndLockForPut)

The source codes of the method scanAndLockForPut in ConcurrentHashMap in JDK7 says:
private HashEntry<K,V> scanAndLockForPut(K key, int hash, V value) {
HashEntry<K,V> first = entryForHash(this, hash);
HashEntry<K,V> e = first;
HashEntry<K,V> node = null;
int retries = -1; // negative while locating node
while (!tryLock()) {
HashEntry<K,V> f; // to recheck first below
if (retries < 0) {
if (e == null) {
if (node == null) // speculatively create node
node = new HashEntry<K,V>(hash, key, value, null);
retries = 0;
}
else if (key.equals(e.key))
retries = 0;
else
e = e.next;
}
else if (++retries > MAX_SCAN_RETRIES) {
lock();
break;
}
else if ((retries & 1) == 0 &&
(f = entryForHash(this, hash)) != first) {
e = first = f; // re-traverse if entry changed
retries = -1;
}
}
return node;
}
I understand what the codes mean, but what I don't is this else if entry:
else if ((retries & 1) == 0 && (f = entryForHash(this, hash)) != first)
My question is:
Why do we have to do "(retries & 1) == 0"?
EDIT:
I kind of figure it out. It's all because the constant MAX_SCAN_RETRIES:
static final int MAX_SCAN_RETRIES = Runtime.getRuntime().availableProcessors() > 1 ? 64 : 1;
In single core processor, MAX_SCAN_RETRIES = 1. So the second time the thread steps into the loop "while(tryLock)", it doesn't have to check whether the first node was changed.
However, in multi cores processor, this will behave like checking whether the first node is changed every 2 times in the while loop.
Is the above explanation correct?
Let's break this down:
1:
(retries & 1) == 0
This returns 1 for odd numbers, 0 for even numbers. Basically, to get past, there's a 1 in 2 chance, if the number is even.
2:
f = entryForHash(this, hash)
f is a temporary variable used to store the value of the latest entry in the segment.
3:
(/* ... */) != first
Checks if the value changed. If it did, it would move the current entry to the start, and re-iterate the linked nodes again in attempt to acquire the lock.
I've asked this question on the concurrency-interest mailing list, and the author(Doug Lea) himself replied:
Yes. We need only ensure that staleness is eventually detected.
Alternating the head-checks works fine, and simplifies use of
the same code for both uni- and multi- processors.
link
So I think this is the end of this question.
I think there are some bugs for the method!
first let us see the put method:
final V put(K key, int hash, V value, boolean onlyIfAbsent) {
HashEntry<K,V> node = tryLock() ? null :
scanAndLockForPut(key, hash, value);//1. scanAndLockForPut only return
// null or a new Entry
V oldValue;
try {
HashEntry<K,V>[] tab = table;
int index = (tab.length - 1) & hash;
HashEntry<K,V> first = entryAt(tab, index);
for (HashEntry<K,V> e = first;;) {
if (e != null) {
K k;
if ((k = e.key) == key ||
(e.hash == hash && key.equals(k))) {
oldValue = e.value;
if (!onlyIfAbsent) {
e.value = value;
++modCount;
}
break;
}
e = e.next;
}
else {
// 2. here the node is null or a new Entry
// and the node.next is the origin head node
if (node != null)
node.setNext(first);
else
node = new HashEntry<K,V>(hash, key, value, first);
int c = count + 1;
if (c > threshold && tab.length < MAXIMUM_CAPACITY)
rehash(node);
else
setEntryAt(tab, index, node);//3. finally, the node become
// the new head,so eventually
// every thing we put will be
// the head of the entry list
// and it may appears two equals
// entry in the same entry list.
++modCount;
count = c;
oldValue = null;
break;
}
}
} finally {
unlock();
}
return oldValue;
}
step: 1. scanAndLockForPut only return null or a new Entry.
step: 2. the node eventualy a new Entry, and the node.next is the origin head node
step: 3. finally, the node become the new head,so eventually every thing we put will be the head of the entry list and it may appears two equals entry in the same entry list when the concurrentHashMap works in a concurrent environment.
That is my opinion, and I am not exactly sure about whether it is right or not. So I hope you all give me some advice,thanks a lot!!