Poco C++ building nested JSON objects - c++

I have a nested JSON object. I'm trying to build it in a function and add the inner object to the original, but I can't extract the result.
void build_object (Poco::JSON::Object * const result)
{
/* Construct some int/bool/string fields here */
Poco::JSON::Object inner;
inner.set("some_number", 5);
inner.set("some_string", "xyz");
/* This is where it breaks down */
std::string key = "new_object";
result->set("new_object", inner);
/* Then some debugging and testing */
// The new object is printed inside the first -> seems like it's working
result->stringify(std::cout);
printf("result has(key): %i\n", result->has(key)); // true
printf("isObject: %i\n", result->isObject(key)); // false - huh?
printf("isNull: %i\n", result->isNull(key)); // false
printf("isArray: %i\n", result->isArray(key)); // false
Poco::JSON::Object::Ptr ptr = result->getObject(key);
// unsurpisingly fails since the above indicates it's not an object
printf("ptr isNull: %i\n", ptr.isNull()); // true
// ptr->has("some_number"); // throws NullPointerException
// if it's not an object/null/array, it must be a value
Poco::Dynamic::Var v = result->get(key);
// at least one of these things should be true, otherwise what is it?
printf("var isString: %i\n", v.isString()); // false
printf("var isStuct: %i\n", v.isStruct()); // false
printf("var isEmpty: %i\n", v.isEmpty()); // false
printf("var isArray: %i\n", v.isArray()); // false
printf("var isSigned: %i\n", v.isSigned()); // false
printf("var isNumeric: %i\n", v.isNumeric());// false
}
So, I have an inner object that is correctly put into the result, it is being printed via stringify with all the correct values and result->has() is successful. But, according to the result, it is not an object, array, or null, so you should be able to get it with var. But, once it's gotten from var, it's not a string, struct, array, or number, and it's also not empty. The inner object seems to exist and not exist at the same time.
So, how do I put this object into my result? And how do I get it out?
Thanks
note: I've seen this thread Correct usage of Poco C++ JSON for parsing data, but it's building the JSON object from string and then parsing it. I suppose I could build everything as a string and convert to the Poco Object at the last step, but I'm still curious why the above is happening. Also, using result->set() and result->get() are a cleaner, less hack-y solution than going through a string.
References: Poco JSON Doc,
Poco Dynamic Var Doc

Poco::JSON Objects and Arrays are held as shared pointers internally by default (optimization to avoid values copying) and everything is Dynamic::Var, so it works for both pointers and values. When you insert an Object as value it works because Dynamic::Var will hold pretty much anything, but the problem you experience when inspecting it comes from the fact that internal comparison does not return true for Object values because it compares only with default type - Poco::SharedPtr<Poco::JSON::Object>.
Here's a workaround:
void build_object (Poco::JSON::Object * const result)
{
// smart pointer, so don't worry about cleaning up
Poco::JSON::Object::Ptr inner = new Poco::JSON::Object;
inner->set("some_number", 5);
inner->set("some_string", "xyz");
std::string key = "new_object";
result->set(key, inner);
printf("isObject: %i\n", result->isObject(key)); // true
}
I have opened a github issue to alleviate this caveat.

I have been trying to create json file having nested object using poco library. Finally able to do with Poco::Json::Array.
Please find the posted code sinippet. Hope it will help. Json output attached with post.
#include "Poco\JSON\JSON.h"
#include "Poco\JSON\Stringifier.h"
#include "Poco\JSON\Object.h"
#include "Poco\Dynamic\Var.h"
using namespace std;
using Poco::JSON::Stringifier;
using Poco::JSON::Object;
using Poco::JSON::Array;
void makeJsonNestedObject()
{
Object RootObj(true);
Array FLArray;
for(int i=0; i<3; i++)
{
Object::Ptr FirstLevelArrayNode = new Poco::JSON::Object(true);
TCHAR strNameBuff[15];
_stprintf(strNameBuff, _T("%s_%d"),_T("Servername"),i);
std::basic_string<TCHAR> strName = strNameBuff;
FirstLevelArrayNode->set("HostName", strName);
FirstLevelArrayNode->set("Overall Impact", "Dummy Data");
Array SLArray;
for(int j=0; j<3;j++)
{
Object::Ptr SecondLevelArrayNode = new Poco::JSON::Object(true);
TCHAR attr1NameBuff[15];
TCHAR attr2NameBuff[15];
_stprintf(attr1NameBuff, _T("%s_%d"),_T("AttrOne"),j);
_stprintf(attr2NameBuff, _T("%s_%d"),_T("AttrTwo"),j);
std::basic_string<TCHAR> attr1Name = attr1NameBuff;
std::basic_string<TCHAR> attr2Name = attr2NameBuff;
SecondLevelArrayNode->set("Attribute", attr1Name);
SecondLevelArrayNode->set("SubAttribute", attr2Name);
Poco::Dynamic::Var obj(SecondLevelArrayNode);
SLArray.add(obj);
}
FirstLevelArrayNode->set("Attribute_Details",SLArray);
Poco::Dynamic::Var FLArrayNodeobj(FirstLevelArrayNode);
FLArray.add(FLArrayNodeobj);
}
std::ostringstream os;
std::cout <<"before stringlify.." << std::endl;
FLArray.stringify(os, 2);
std::cout << os.str() << std::endl;
}
Json output:
[
{
"HostName" : "Servername_0",
"Overall Impact" : "Dummy Data",
"Attribute_Details" : [
{
"Attribute" : "AttrOne_0",
"SubAttribute" : "AttrTwo_0"
},
{
"Attribute" : "AttrOne_1",
"SubAttribute" : "AttrTwo_1"
},
{
"Attribute" : "AttrOne_2",
"SubAttribute" : "AttrTwo_2"
}
]
},
{
"HostName" : "Servername_1",
"Overall Impact" : "Dummy Data",
"Attribute_Details" : [
{
"Attribute" : "AttrOne_0",
"SubAttribute" : "AttrTwo_0"
},
{
"Attribute" : "AttrOne_1",
"SubAttribute" : "AttrTwo_1"
},
{
"Attribute" : "AttrOne_2",
"SubAttribute" : "AttrTwo_2"
}
]
},
{
"HostName" : "Servername_2",
"Overall Impact" : "Dummy Data",
"Attribute_Details" : [
{
"Attribute" : "AttrOne_0",
"SubAttribute" : "AttrTwo_0"
},
{
"Attribute" : "AttrOne_1",
"SubAttribute" : "AttrTwo_1"
},
{
"Attribute" : "AttrOne_2",
"SubAttribute" : "AttrTwo_2"
}
]
}
]

Related

nlohmann json insert value array partially to already existing data

I have a particular case which I am trying to solve with minimal changes if possible.
one of the data is
js["key1"]["subkey2"]["subsubkey3"].push_back({1,2,3,{4,5}});
[ 1,2,3,[[4,5]] ]
Later at some stage I want to insert
{1,2,3,{4,6}}
Then it should become
[ 1,2,3,[[4,5],[4,6]] ]
How can I make this possible without making 1,2,3 value as key?
I did some playing. I didn't get the results you were looking for. Here's my code and results so far.
#include <iostream>
#include <json.hpp>
using namespace std;
using JSON = nlohmann::json;
int main() {
JSON json = JSON::object();
JSON key1JSON = JSON::object();
JSON key2JSON = JSON::object();
JSON key3JSON = JSON::array();
key3JSON.push_back( {1,2,3, {4,5} } );
key3JSON.push_back( {6} );
key2JSON["subsubkey3"] = key3JSON;
key1JSON["subkey2"] = key2JSON;
json["key1"] = key1JSON;
cout << json.dump(2) << endl;
}
Output:
{
"key1": {
"subkey2": {
"subsubkey3": [
[
1,
2,
3,
[
4,
5
]
],
[
6
]
]
}
}
}
You'll see that the first push_back pushed an array inside an array, which is probably one level deeper than you wanted, and the second one added a second array, which is also not what you want.
Which means you're probably going to have to write your own method, especially as you want to also handle uniqueness. I personally never free-format data that way you have in your example. But maybe your method would look something like:
bool contains(const JSON &json, const JSON &value) {
... this looks like fun to write.
}
void appendUnique(JSON &json, const JSON &array) {
for (JSON & thisJson: array) {
if (!contains(json, thisJson)) {
json.push_back(thisJson);
}
}
}
I modified my code like this:
void appendUnique(JSON &json, const JSON & array) {
for (const JSON & thisJSON: array) {
json.push_back(thisJSON);
}
}
...
appendUnique(key3JSON, {1,2,3, {4,5} } );
appendUnique(key3JSON, {6} );
And got this:
{
"key1": {
"subkey2": {
"subsubkey3": [
1,
2,
3,
[
4,
5
],
6
]
}
}
}
I'm not going to write the isUnique method. But I think you may have to take this to conclusion.

How to convert any value to an object and add members with boost::property_tree json

I have a program that modifies a JSON document if necessary. The program has to add a child to another value whether or not it's an already an object. The program should behave like so:
If the object with key "x" does not exist, create object with key "x" and add value y as a child.
If the object with key "x" DOES exist, set value y as a child.
If the key "x" exists and is ANY OTHER type, delete it, create an object with the key "x" and then add value y as a child.
I see ways to test if property tree values exist or whether they are specified types, but none to test if it's an object or not an object.
Here's a simple program I made illustrating what I mean:
#include <boost/property_tree/ptree.hpp>
#include <boost/property_tree/json_parser.hpp>
#include <sstream>
#include <iostream>
const char *json = "{"
"\"object\" : { \"mighty\" : \"wind\" },"
"\"boolean\" : true"
"}";
void printTree( std::string name, boost::property_tree::ptree tree )
{
std::cout << "Pass '" << name << "'" << std::endl;
try
{
std::stringstream ss;
boost::property_tree::write_json( ss, tree );
std::cout << ss.str() << std::endl;
}
catch( std::exception &e )
{
std::cout << "Could not make create json: " << e.what() << std::endl;
}
}
int main( int argc, char *argv[] )
{
boost::property_tree::ptree tree;
// Load it
std::istringstream ss_json( json );
boost::property_tree::read_json( ss_json, tree );
// Add a value to an object that doesn't exist
tree.put( "none.value", "hello!" );
// Print to see
printTree( "Nonexistent value test", tree );
// Add a value to the object
tree.put( "object.value", "bello!" );
// Print this one
printTree( "Adding value test", tree );
// Convert boolean to an object and add a value
tree.put( "boolean.value", "mello!" );
// Print it
printTree( "Converting value test", tree );
}
The output will be:
Pass 'Nonexistent value test'
{
"object": {
"mighty": "wind"
},
"boolean": "true",
"none": {
"value": "hello!"
}
}
Pass 'Adding value test'
{
"object": {
"mighty": "wind",
"value": "bello!"
},
"boolean": "true",
"none": {
"value": "hello!"
}
}
Pass 'Converting value test'
Could not make create json: <unspecified file>: ptree contains data that cannot be represented in JSON format
You can see in the output, the last step fails to convert to JSON (doesn't throw when I try to set it).
How can I achieve scenario 3 in my list above?
If the key "x" exists and is ANY OTHER type, delete it, create an object with the key "x" and then add value y as a child. Also, they don't observe any of the JSON data types.
Your plan is pretty doomed. Property Tree is not a JSON library. Property Trees can have data and child nodes at the same node. E.g.
ptree p;
auto& x = p.put_child("x", {});
x.put_value("hello");
write_json(std::cout, p);
Prints
{
"x": "hello"
}
But adding
/*auto& a = */ p.put_child("x.a", {});
write_json(std::cout, p);
Fails with Live On Coliru
terminate called after throwing an instance of 'boost::wrapexcept<boost::property_tree::json_parser::json_parser_error>'
what(): <unspecified file>: ptree contains data that cannot be represented in JSON format
A workaround would be to remove any value prior to or when adding properties:
x.put_value("");
auto& a = p.put_child("x.a", {});
a.add("prop1", 123);
a.add("prop2", "one two three");
a.add("b.prop1", "nesting");
write_json(std::cout, p);
Would print Live On Coliru
Finer notes
It might seem more efficient to check the presence of a value before clearing it:
if (x.get_value_optional<std::string>()) {
x.put_value("");
}
But due the the stringly typed nature of Property Tree storage there's no difference as the condition will just always be true for std::string. (Similarly there's no way to retrieve a value by reference.)
Note ALSO that when setting the n.prop1 nested property, you MAY have to also check that b has no value if you don't control the source data, because otherwise it would fail again.
Assuming that your object graph structure is reasonably predictable (or even static), I'd suggest getting it over with ahead of time:
for (auto key : { "x", "x.a", "x.a.b" }) {
if (auto child = p.get_child_optional(key)) {
std::cout << "clearing " << key << std::endl;
child->put_value("");
}
}
Which can be generalized with a helper:
clear_values("x.a.b", p);
Which could be implemented as
void clear_values(ptree::path_type path, ptree& p) {
if (path.empty())
return;
auto head = path.reduce();
auto child = p.get_child_optional(head);
if (child) {
child->put_value("");
clear_values(path, *child);
}
}
Bonus
In fact with such a helper it might become opportune to also create the expected hierarchy on the fly:
void clear_values(ptree::path_type path, ptree& p, bool create = false) {
if (path.empty())
return;
auto head = path.reduce();
auto child = p.get_child_optional(head);
if (!child && create) {
child = p.put_child(head, {});
}
if (child) {
child->put_value("");
clear_values(path, *child, create);
}
}
Now it would even work well without any pre-existing data:
Live On Coliru
#include <boost/property_tree/json_parser.hpp>
#include <iostream>
using boost::property_tree::ptree;
void clear_values(ptree::path_type path, ptree& p, bool create = false) {
if (path.empty())
return;
auto head = path.reduce();
auto child = p.get_child_optional(head);
if (!child && create) {
child = p.put_child(head, {});
}
if (child) {
child->put_value("");
clear_values(path, *child, create);
}
}
int main() {
ptree p;
clear_values("x.a.b", p, true);
auto& a = p.get_child("x.a");
a.add("prop1", 123);
a.add("prop2", "one two three");
a.add("b.prop1", "nesting");
write_json(std::cout, p);
}
Prints
{
"x": {
"a": {
"b": {
"prop1": "nesting"
},
"prop1": "123",
"prop2": "one two three"
}
}
}

mongodb query with c++ legacy driver results in BSONElement: bad type -64

I am trying to a process an array in a mongodb collection using the c++ mongo driver. However I always end up with the error message:
terminate called after throwing an instance of mongo::MsgAssertionException'
what(): BSONElement: bad type -64
Aborted (core dumped)
I tried to find it on google, but the only answers if found told me to repair my database. I did this but the error stay.
I am really confused by this error. The collection printed by the mongo-shell seems not to be corrupted and the shell also does not claim any error (see below). So I would assume that the vector rawDataArray generated in my code (see also below) is holding valid data. However, when I try to print the values I get the mentioned error message.
What am I doing wrong?
Thanks a lot
Michael
PS:
I am using the mongodb C++ legacy driver and the error is generated by the following piece of code:
mongo::DBClientConnection connection;
connection.connect("localhost");
mongo::BSONObj resultObj;
try
{
std::auto_ptr<mongo::DBClientCursor> cursor =
connection.query("mydb.results",
MONGO_QUERY( "_id" << mongo::OID(resultID) ) );
while (cursor->more())
{
resultObj = cursor->next();
}
}
catch( const mongo::DBException &e )
{
// error handling
}
// extract raw_data array
std::vector<mongo::BSONElement> rawDataArray;
if( resultObj.hasField("raw_data") )
{
rawDataArray = resultObj["raw_data"].Array();
}
else
{
// error handling
}
for(auto const & data : rawDataArray)
{
std::cout << data << std::endl;
}
The collection is looking like:
db.getCollection('results').find({}).pretty()
{
"_id" : ObjectId("56cf1315f7e0583e2c4ec702"),
"experiment_id" : ObjectId("56c5b8e7e1fa370a1de9d06f"),
"module_id" : ObjectId("56c5b8e7e1fa370a1de9d06e"),
"raw_data" : [
{
"id_number" : "0accb65f4fc311",
"box" : "0accb65f4fc3",
"paper" : 1,
"seed" : 1,
"length" : 0,
"time" : ISODate("2015-09-15T20:00:00.000Z")
},
{
"id_number" : "0accb65f4fc312",
"box" : "0accb65f4fc3",
"paper" : 1,
"seed" : 2,
"length" : 0,
"time" : ISODate("2015-09-15T20:00:00.000Z")
},
{
"id_number" : "0accb65f4fc313",
"box" : "0accb65f4fc3",
"paper" : 1,
"seed" : 3,
"length" : 0,
"time" : ISODate("2015-09-15T20:00:00.000Z")
},
... skipped some data here, there are 204 nearly identical elements
{
"id_number" : "0accb65f4fc3451",
"box" : "0accb65f4fc3",
"paper" : 4,
"seed" : 51,
"length" : 0,
"time" : ISODate("2015-09-15T20:00:00.000Z")
}
],
"processed_data" : []
}
You are accessing resultObj after the cursor object has been destroyed, but resultObj is merely viewing data owned by the cursor. Call getOwned on the BSONObj to get an owning copy if you need its lifetime to extend beyond the lifetime of the cursor which returned it.

Remove nested object in JSON with rapidjson

I'm trying to remove object nested in object in JSON file. However, I can not find any examples on the internet or on the official rapidjson page. My code is written on C++.
I have tried with the following code:
const Value& rootObject= document["root"];
const Value& settingsObject = extensionsObject;
settingsObject.RemoveMember();
But I am not sure what parameter to pass or how to initialize MemberIterator for exact element (as I already know the name of the object I want to remove).
Here is example of the JSON structure:
{
"root": {
"settings": {
"item1": {
"someInfo": 123
},
"item2": {
"someInfo": "string"
}
}
}
}
please chek my code.
Value Child_Obj(kObjectType); // creat object to but it inside another object
Child_Obj.SetObject(); // set it
Child_Obj.AddMember("Child Number", Value(15), Document->GetAllocator()); // add to child key and its value
Value Parent_Obj(kObjectType); // creat parent object that will have inside him the child object
Parent_Obj.SetObject(); // set it
Parent_Obj.AddMember("Parent Number", Value(10), Document->GetAllocator()); // add to parent key and its value
Parent_Obj.AddMember("Child", Child_Obj, Document->GetAllocator()); // add child object to parent object , "Child" as key and Child_Obj as value
// now the file looks like this :
/*
{
"Parent":
{
"Parent Number":10,
"Child":
{
"Child Number":15
}
}
}
*/
// let delete this child
Parent_Obj.RemoveMember("Child"); // here you will give it the key for the object you need to delete
// now its look like this :
/*
{
"Parent":
{
"Parent Number":10,
}
}
*/
// and for fun , if you want to iterate through object , you can do this :
Value::MemberIterator it = Parent_Obj.MemberBegin();
for (; it != Parent_Obj.MemberEnd(); ++it)
{
std::string str = it->name.GetString(); // this will give you the key for current child
}

Writing in order to jsoncpp (c++)

Consider the following example for which my source is
Json::Value root;
root["id"]=0;
Json::Value text;
text["first"]="i";
text["second"]="love";
text["third"]="you";
root["text"]=text;
root["type"]="test";
root["begin"]=1;
root["end"]=1;
Json::StyledWriter writer;
string strJson=writer.write(root);
cout<<"JSON WriteTest" << endl << strJson <<endl;
I thought I'd write the json fields in the order of the lines. Instead the result is:
JSON WriteTest
{
"begin" : 1,
"end" : 1,
"id" : 0,
"text" : {
"first" : "i",
"second" : "love",
"third" : "you"
},
"type" : "test"
}
I want json format is
JSON WriteTest
{
"id" : 0,
"text" : {
"first" : "i",
"second" : "love",
"third" : "you"
},
"type" : "test"
"begin" : 1,
"end" : 1,
}
How can I write a Json order?
No, I don't think you can. JsonCpp keeps its values in a std::map<CZString, Value>, which is always sorted by the CZString comparison. So it doesn't know the original order you added items.
This is my workaround to a get an ordered json output from jsoncpp
Json::Value root;
root["*1*id"]=0;
Json::Value text;
text["*1*first"]="i";
text["*2*second"]="love";
text["*3*third"]="you";
root["*2*text"]=text;
root["*3*type"]="test";
root["*4*begin"]=1;
root["*5*end"]=1;
Json::StyledWriter writer;
string resultString=writer.write(root);
resultString=ReplaceAll(resultString,"*1*", "");
resultString=ReplaceAll(resultString,"*2*", "");
resultString=ReplaceAll(resultString,"*3*", "");
resultString=ReplaceAll(resultString,"*4*", "");
resultString=ReplaceAll(resultString,"*5*", "");
cout<<"JSON WriteTest" << endl << resultString <<endl;
with RepleceAll function defined as this
std::string ReplaceAll(std::string str, const std::string& from, const std::string& to) {
size_t start_pos = 0;
while((start_pos = str.find(from, start_pos)) != std::string::npos) {
str.replace(start_pos, from.length(), to);
start_pos += to.length(); // Handles case where 'to' is a substring of 'from'
}
return str;
}
I have a way can solve your problem. Would you like to try? My solution is that you use boost/property_tree/json_parser.hpp, the output is what format you want! About There is my code:
#include <boost/property_tree/json_parser.hpp>
#include <sstream>
#include <iostream>
using namespace std;
int main()
{
boost::property_tree::ptree parser, child;
parser.put("id", 0);
child.put("first", "i");
child.put("second", "love");
child.put("third", "you");
parser.put_child("text", child);
parser.put("type", "test");
parser.put("begin", 1);
parser.put("end", 1);
stringstream ss;
boost::property_tree::json_parser::write_json(ss, parser);
cout << ss.str() << endl;
return 0;
}
Before run the codes, you should install boost 1.57. The codes run well in gcc 4.7, boost 1.57.The output is { "id" : 0, "text" : { "first" : "i", "second" : "love", "third" : "you" }, "type" : "test" "begin" : 1, "end" : 1, }. About boost::property_tree::ptree, you can click here. It used list<pair<key, ptree>> for saving data. So it saved unordered data, unless you called list.sort(). I hope this can help you.
As mentioned by The Dark, JsonCpp keeps its values in a std::map<CZString, Value>, which is always sorted by the CZString comparison, without keeping track neither of the original order in which you added the items nor the desired order in the output.
But you can use this "hidden feature" in your benefit. I mean, you just need that the keys in the desired order follow the "natural" order of CZString. I have a method in my JSONCPP wrapper classes that do this. The quick'n'dirty code, converted to simple function, would be something like this:
std::string sortedStr(Json::Value & value, std::vector<std::string> sortKeys)
{
Json::Value sortedValue; // The JSON object to store the new (sorted) hash
char newKey[60]; // I use C expressions, modify to C++ if you like
// Build new sortedValue
int i = 0;
for (auto & key : sortKeys) {
sprintf(newKey, "SORTEDKEY:%03d-%s", i++, key.c_str());
sortedValue[newKey] = value[key];
}
// Write to string, should be sorted on primary keys
Json::StyledWriter w;
std::string result = w.write(sortedValue);
// Remove aux. tags from primary keys
std::size_t pos = 0;
while ((pos = result.find("SORTEDKEY:", pos)) != std::string::npos) {
result.erase(pos, 14);
}
return result;
}
To use it, just call:
std::string sortedObjStr = sortedValue(myValue, {"first", "second", "third", "fourth"});
Note that:
I use this for relatively small objects (configuration data).
I use the "tag" SORTEDKEY, since this is not going to appear anywhere in my data. Modify according to your needs.
I do not check that the keys used do exist. You can add this check.
You can use this also to generate a restricted, ordered subset of your original object.
The key-value pairs in an object will always be sorted. Json arrays are not sorted, they consists of a series of values without keys.
Objects, as named collections (arrays) of key-value pairs within brackets, in an array, will retain their positions, e.g.
{
"adressen" : [
{
"start" : {
"ID" : 1,
"key" : "2352KJ25",
"lat" : 52.157225922529967,
"lon" : 4.5298663828345527
}
},
{
"eind" : {
"ID" : 2,
"key" : "2352KJ25",
"lat" : 52.157225922529967,
"lon" : 4.5298663828345527
}
}
}
ID, key, lat, lon are sorted, but start and eind are in their original positions.
So, at least your first, second, third could have been
Json::Value text(Json::arrayValue);
text.append("I");
text.append("love");
text.append("you");
No need for the tags first, second and third!
Maybe this helps you to find a workaround.