cpp RapidJSON - Resolve key conflicts without information loss - c++

Question

I want to parse a text file which is similar to JSON. After some character conversions, it still has some objects, which have key conflicts. So my JSON looked like this:
{
"key1": {
"a": "asdf",
"a": "foo",
"a": "bar",
"a": "fdas"
}
}
And i wanted to resolve it into this:
{
"key1": {
"a": [
"asdf",
"foo",
"bar",
"fdas"
]
}
}
I tried to achieve this with JsonCpp, but it can't handle the key conflicts. So i chose to use RapidJSON, especially because it CAN keep all the key-conflict-members when parsing.
To then resolve the key conflicts without loosing information, i wrote the following recursive RapidJSON cpp code:
void resolveKeyConflicts(rj::Value& value) {
if (value.IsObject()) {
std::map<std::string, unsigned int> nameCount;
for (rj::Value::MemberIterator vMIt = value.MemberBegin();
vMIt != value.MemberEnd(); vMIt++) {
std::string name(vMIt->name.GetString());
if (nameCount.find(name) == nameCount.end()) {
nameCount[name] = 1;
} else {
nameCount[name] += 1;
}
}
for (std::map<std::string, unsigned int>::iterator nCIt =
nameCount.begin(); nCIt != nameCount.end(); nCIt++) {
if (nCIt->second > 1) {
rj::Value newArray(rj::kArrayType);
for (rj::Value::MemberIterator vFMIt = value.FindMember(
nCIt->first.c_str()); vFMIt != value.MemberEnd();
vFMIt++) {
if (vFMIt->name.GetString() == nCIt->first) {
rj::Value value(vFMIt->value, this->GetAllocator());
newArray.PushBack(value, this->GetAllocator());
}
}
value.EraseMember(value.FindMember(nCIt->first.c_str()),
value.MemberEnd());
rj::Value key(nCIt->first.c_str(), nCIt->first.length(),
this->GetAllocator());
value.AddMember(key, newArray, this->GetAllocator());
}
}
for (rj::Value::MemberIterator vMIt = value.MemberBegin();
vMIt != value.MemberEnd(); vMIt++) {
if (vMIt->value.IsObject() || vMIt->value.IsArray()) {
resolveKeyConflicts(vMIt->value);
}
}
} else if (value.IsArray()) {
for (rj::Value::ValueIterator vVIt = value.Begin(); vVIt != value.End();
vVIt++) {
resolveKeyConflicts(*vVIt);
}
}
}
This works pretty good as long as the conflicting key-members are the only members in that object. This can, i think, be archived with simpler code, but i additionally tried to be able to resolve arbitrary key conflicts like this:
{
"key2": {
"a": "asdf",
"b": "foo",
"b": "bar",
"c": "fdas"
}
}
Into this:
{
"key2": {
"a": "asdf",
"b": [
"foo",
"bar"
],
"c": "fdas"
}
}
Turns out FindMember does not, as i thought, gives back an iterator over all members with the same key name, but just the position of the first member with that key. I think my python way of thinking may have messed with my expectations on FindMember. So like this, the code is going to lose the "c": "fdas" member.
I relied on MemberIterator EraseMember(MemberIterator first, MemberIterator last) because all of the other methods to remove a member mentioned in http://rapidjson.org/md_doc_tutorial.html#ModifyObject seem to have problems removing the last member in the key1 case. But EraseMember like this is definitely the wrong choice for the key2 case.
So I'm kind of lost here. Can please somebody point me into the right direction to resolve the key conflicts without information loss, which can handle both the key1 and the key2 case?
edit: I'm using RapidJSON from https://github.com/miloyip/rapidjson/tree/v1.0.2 which is at the v1.0.2 tag.

Answer 1

I think the tricky part is to memorize whether the key has already expanded to an array (because the value may be originally an array).
So, another way is firstly convert all key: value into key:[value], do the merge, and then convert back to key: value if there is only one element in the array.
This is my attempt:
static void MergeDuplicateKey(Value& v, Value::AllocatorType& a) {
if (v.IsObject()) {
// Convert all key:value into key:[value]
for (Value::MemberIterator itr = v.MemberBegin(); itr != v.MemberEnd(); ++itr)
itr->value = Value(kArrayType).Move().PushBack(itr->value, a);
// Merge arrays if key is duplicated
for (Value::MemberIterator itr = v.MemberBegin(); itr != v.MemberEnd();) {
Value::MemberIterator itr2 = v.FindMember(itr->name);
if (itr != itr2) {
itr2->value.PushBack(itr->value[0], a);
itr = v.EraseMember(itr);
}
else
++itr;
}
// Convert key:[values] back to key:value if there is only one value
for (Value::MemberIterator itr = v.MemberBegin(); itr != v.MemberEnd(); ++itr) {
if (itr->value.Size() == 1)
itr->value = itr->value[0];
MergeDuplicateKey(itr->value, a); // Recursion on the value
}
}
else if (v.IsArray())
for (Value::ValueIterator itr = v.Begin(); itr != v.End(); ++itr)
MergeDuplicateKey(*itr, a);
}
I tested it in this commit.

Answer 2

I completely rewrote that part, trying (again) another approach. I think i found a pretty elegant solution:
void resolveKeyConflicts(rj::Value& value) {
if (value.IsObject()) {
std::vector<std::string> resolvedConflicts;
rj::Value newValue(rj::kObjectType);
for (rj::Value::MemberIterator vMIt = value.MemberBegin();
vMIt != value.MemberEnd(); vMIt++) {
rj::Value::MemberIterator nVFMIt = newValue.FindMember(vMIt->name);
if (nVFMIt == newValue.MemberEnd()) {
rj::Value newKey(vMIt->name, this->GetAllocator());
newValue.AddMember(newKey, vMIt->value, this->GetAllocator());
} else {
std::string conflict(vMIt->name.GetString(),
vMIt->name.GetStringLength());
if (std::find(resolvedConflicts.begin(),
resolvedConflicts.end(), conflict)
== resolvedConflicts.end()) {
rj::Value newArray(rj::kArrayType);
nVFMIt->value.Swap(newArray);
nVFMIt->value.PushBack(newArray, this->GetAllocator());
nVFMIt->value.PushBack(vMIt->value, this->GetAllocator());
resolvedConflicts.push_back(conflict);
} else {
nVFMIt->value.PushBack(vMIt->value, this->GetAllocator());
}
}
}
value.SetNull().SetObject();
for (rj::Value::MemberIterator nVMIt = newValue.MemberBegin();
nVMIt != newValue.MemberEnd(); nVMIt++) {
if (nVMIt->value.IsObject() || nVMIt->value.IsArray()) {
this->resolveKeyConflicts(nVMIt->value);
}
value.AddMember(nVMIt->name, nVMIt->value, this->GetAllocator());
}
} else if (value.IsArray()) {
for (rj::Value::ValueIterator vVIt = value.Begin(); vVIt != value.End();
vVIt++) {
if (vVIt->IsObject() || vVIt->IsArray()) {
this->resolveKeyConflicts(*vVIt);
}
}
}
}
I'm not so sure about the value.SetNull().SetObject() part for emptying value, but it works.
If you think there is room for improvement, just let me know where. Thanks.