Related
I have a program that modifies a JSON document if necessary. The program has to add a child to another value whether or not it's an already an object. The program should behave like so:
If the object with key "x" does not exist, create object with key "x" and add value y as a child.
If the object with key "x" DOES exist, set value y as a child.
If the key "x" exists and is ANY OTHER type, delete it, create an object with the key "x" and then add value y as a child.
I see ways to test if property tree values exist or whether they are specified types, but none to test if it's an object or not an object.
Here's a simple program I made illustrating what I mean:
#include <boost/property_tree/ptree.hpp>
#include <boost/property_tree/json_parser.hpp>
#include <sstream>
#include <iostream>
const char *json = "{"
"\"object\" : { \"mighty\" : \"wind\" },"
"\"boolean\" : true"
"}";
void printTree( std::string name, boost::property_tree::ptree tree )
{
std::cout << "Pass '" << name << "'" << std::endl;
try
{
std::stringstream ss;
boost::property_tree::write_json( ss, tree );
std::cout << ss.str() << std::endl;
}
catch( std::exception &e )
{
std::cout << "Could not make create json: " << e.what() << std::endl;
}
}
int main( int argc, char *argv[] )
{
boost::property_tree::ptree tree;
// Load it
std::istringstream ss_json( json );
boost::property_tree::read_json( ss_json, tree );
// Add a value to an object that doesn't exist
tree.put( "none.value", "hello!" );
// Print to see
printTree( "Nonexistent value test", tree );
// Add a value to the object
tree.put( "object.value", "bello!" );
// Print this one
printTree( "Adding value test", tree );
// Convert boolean to an object and add a value
tree.put( "boolean.value", "mello!" );
// Print it
printTree( "Converting value test", tree );
}
The output will be:
Pass 'Nonexistent value test'
{
"object": {
"mighty": "wind"
},
"boolean": "true",
"none": {
"value": "hello!"
}
}
Pass 'Adding value test'
{
"object": {
"mighty": "wind",
"value": "bello!"
},
"boolean": "true",
"none": {
"value": "hello!"
}
}
Pass 'Converting value test'
Could not make create json: <unspecified file>: ptree contains data that cannot be represented in JSON format
You can see in the output, the last step fails to convert to JSON (doesn't throw when I try to set it).
How can I achieve scenario 3 in my list above?
If the key "x" exists and is ANY OTHER type, delete it, create an object with the key "x" and then add value y as a child. Also, they don't observe any of the JSON data types.
Your plan is pretty doomed. Property Tree is not a JSON library. Property Trees can have data and child nodes at the same node. E.g.
ptree p;
auto& x = p.put_child("x", {});
x.put_value("hello");
write_json(std::cout, p);
Prints
{
"x": "hello"
}
But adding
/*auto& a = */ p.put_child("x.a", {});
write_json(std::cout, p);
Fails with Live On Coliru
terminate called after throwing an instance of 'boost::wrapexcept<boost::property_tree::json_parser::json_parser_error>'
what(): <unspecified file>: ptree contains data that cannot be represented in JSON format
A workaround would be to remove any value prior to or when adding properties:
x.put_value("");
auto& a = p.put_child("x.a", {});
a.add("prop1", 123);
a.add("prop2", "one two three");
a.add("b.prop1", "nesting");
write_json(std::cout, p);
Would print Live On Coliru
Finer notes
It might seem more efficient to check the presence of a value before clearing it:
if (x.get_value_optional<std::string>()) {
x.put_value("");
}
But due the the stringly typed nature of Property Tree storage there's no difference as the condition will just always be true for std::string. (Similarly there's no way to retrieve a value by reference.)
Note ALSO that when setting the n.prop1 nested property, you MAY have to also check that b has no value if you don't control the source data, because otherwise it would fail again.
Assuming that your object graph structure is reasonably predictable (or even static), I'd suggest getting it over with ahead of time:
for (auto key : { "x", "x.a", "x.a.b" }) {
if (auto child = p.get_child_optional(key)) {
std::cout << "clearing " << key << std::endl;
child->put_value("");
}
}
Which can be generalized with a helper:
clear_values("x.a.b", p);
Which could be implemented as
void clear_values(ptree::path_type path, ptree& p) {
if (path.empty())
return;
auto head = path.reduce();
auto child = p.get_child_optional(head);
if (child) {
child->put_value("");
clear_values(path, *child);
}
}
Bonus
In fact with such a helper it might become opportune to also create the expected hierarchy on the fly:
void clear_values(ptree::path_type path, ptree& p, bool create = false) {
if (path.empty())
return;
auto head = path.reduce();
auto child = p.get_child_optional(head);
if (!child && create) {
child = p.put_child(head, {});
}
if (child) {
child->put_value("");
clear_values(path, *child, create);
}
}
Now it would even work well without any pre-existing data:
Live On Coliru
#include <boost/property_tree/json_parser.hpp>
#include <iostream>
using boost::property_tree::ptree;
void clear_values(ptree::path_type path, ptree& p, bool create = false) {
if (path.empty())
return;
auto head = path.reduce();
auto child = p.get_child_optional(head);
if (!child && create) {
child = p.put_child(head, {});
}
if (child) {
child->put_value("");
clear_values(path, *child, create);
}
}
int main() {
ptree p;
clear_values("x.a.b", p, true);
auto& a = p.get_child("x.a");
a.add("prop1", 123);
a.add("prop2", "one two three");
a.add("b.prop1", "nesting");
write_json(std::cout, p);
}
Prints
{
"x": {
"a": {
"b": {
"prop1": "nesting"
},
"prop1": "123",
"prop2": "one two three"
}
}
}
I do not understand why the below code doesn't work as intended to copy the one element from doc1 to doc2:
void test_xml(){
using namespace tinyxml2;
XMLDocument doc1, doc2;
XMLPrinter printer;
doc1.LoadFile("./res/xml/basic.xml");
if(doc1.Error())
std::cout << doc1.ErrorName();
doc1.Print(&printer);
std::cout << printer.CStr(); // prints "</atag>" without quotes
printer.ClearBuffer();
doc2.InsertFirstChild(doc1.RootElement());
if(doc2.Error())
std::cout << doc2.ErrorName(); // doesn't run, there's no error
doc2.Print(&printer);
std::cout << printer.CStr(); // prints nothing, no child got inserted to doc2
std::cout << doc2.NoChildren(); //prints "1" meaning we didn't insert anything
}
Can someone point out how this can be improved?
From the TinyXml2 documentation:
InsertFirstChild ... Returns the addThis argument or 0 if the node does not belong to the same document.
Basically you can only add a node to a document if the node was manufactured by that document (with NewElement, NewText etc).
You have to walk through doc1 creating corresponding nodes as you go (with ShallowClone, and adding them to doc2. It appears there is no DeepClone to do it all for you.
At http://sourceforge.net/p/tinyxml/discussion/42748/thread/820b0377/, "practisevoodoo" suggests:
XMLNode *deepCopy( XMLNode *src, XMLDocument *destDoc )
{
XMLNode *current = src->ShallowClone( destDoc );
for( XMLNode *child=src->FirstChild(); child; child=child->NextSibling() )
{
current->InsertEndChild( deepCopy( child, destDoc ) );
}
return current;
}
Consider the following example for which my source is
Json::Value root;
root["id"]=0;
Json::Value text;
text["first"]="i";
text["second"]="love";
text["third"]="you";
root["text"]=text;
root["type"]="test";
root["begin"]=1;
root["end"]=1;
Json::StyledWriter writer;
string strJson=writer.write(root);
cout<<"JSON WriteTest" << endl << strJson <<endl;
I thought I'd write the json fields in the order of the lines. Instead the result is:
JSON WriteTest
{
"begin" : 1,
"end" : 1,
"id" : 0,
"text" : {
"first" : "i",
"second" : "love",
"third" : "you"
},
"type" : "test"
}
I want json format is
JSON WriteTest
{
"id" : 0,
"text" : {
"first" : "i",
"second" : "love",
"third" : "you"
},
"type" : "test"
"begin" : 1,
"end" : 1,
}
How can I write a Json order?
No, I don't think you can. JsonCpp keeps its values in a std::map<CZString, Value>, which is always sorted by the CZString comparison. So it doesn't know the original order you added items.
This is my workaround to a get an ordered json output from jsoncpp
Json::Value root;
root["*1*id"]=0;
Json::Value text;
text["*1*first"]="i";
text["*2*second"]="love";
text["*3*third"]="you";
root["*2*text"]=text;
root["*3*type"]="test";
root["*4*begin"]=1;
root["*5*end"]=1;
Json::StyledWriter writer;
string resultString=writer.write(root);
resultString=ReplaceAll(resultString,"*1*", "");
resultString=ReplaceAll(resultString,"*2*", "");
resultString=ReplaceAll(resultString,"*3*", "");
resultString=ReplaceAll(resultString,"*4*", "");
resultString=ReplaceAll(resultString,"*5*", "");
cout<<"JSON WriteTest" << endl << resultString <<endl;
with RepleceAll function defined as this
std::string ReplaceAll(std::string str, const std::string& from, const std::string& to) {
size_t start_pos = 0;
while((start_pos = str.find(from, start_pos)) != std::string::npos) {
str.replace(start_pos, from.length(), to);
start_pos += to.length(); // Handles case where 'to' is a substring of 'from'
}
return str;
}
I have a way can solve your problem. Would you like to try? My solution is that you use boost/property_tree/json_parser.hpp, the output is what format you want! About There is my code:
#include <boost/property_tree/json_parser.hpp>
#include <sstream>
#include <iostream>
using namespace std;
int main()
{
boost::property_tree::ptree parser, child;
parser.put("id", 0);
child.put("first", "i");
child.put("second", "love");
child.put("third", "you");
parser.put_child("text", child);
parser.put("type", "test");
parser.put("begin", 1);
parser.put("end", 1);
stringstream ss;
boost::property_tree::json_parser::write_json(ss, parser);
cout << ss.str() << endl;
return 0;
}
Before run the codes, you should install boost 1.57. The codes run well in gcc 4.7, boost 1.57.The output is { "id" : 0, "text" : { "first" : "i", "second" : "love", "third" : "you" }, "type" : "test" "begin" : 1, "end" : 1, }. About boost::property_tree::ptree, you can click here. It used list<pair<key, ptree>> for saving data. So it saved unordered data, unless you called list.sort(). I hope this can help you.
As mentioned by The Dark, JsonCpp keeps its values in a std::map<CZString, Value>, which is always sorted by the CZString comparison, without keeping track neither of the original order in which you added the items nor the desired order in the output.
But you can use this "hidden feature" in your benefit. I mean, you just need that the keys in the desired order follow the "natural" order of CZString. I have a method in my JSONCPP wrapper classes that do this. The quick'n'dirty code, converted to simple function, would be something like this:
std::string sortedStr(Json::Value & value, std::vector<std::string> sortKeys)
{
Json::Value sortedValue; // The JSON object to store the new (sorted) hash
char newKey[60]; // I use C expressions, modify to C++ if you like
// Build new sortedValue
int i = 0;
for (auto & key : sortKeys) {
sprintf(newKey, "SORTEDKEY:%03d-%s", i++, key.c_str());
sortedValue[newKey] = value[key];
}
// Write to string, should be sorted on primary keys
Json::StyledWriter w;
std::string result = w.write(sortedValue);
// Remove aux. tags from primary keys
std::size_t pos = 0;
while ((pos = result.find("SORTEDKEY:", pos)) != std::string::npos) {
result.erase(pos, 14);
}
return result;
}
To use it, just call:
std::string sortedObjStr = sortedValue(myValue, {"first", "second", "third", "fourth"});
Note that:
I use this for relatively small objects (configuration data).
I use the "tag" SORTEDKEY, since this is not going to appear anywhere in my data. Modify according to your needs.
I do not check that the keys used do exist. You can add this check.
You can use this also to generate a restricted, ordered subset of your original object.
The key-value pairs in an object will always be sorted. Json arrays are not sorted, they consists of a series of values without keys.
Objects, as named collections (arrays) of key-value pairs within brackets, in an array, will retain their positions, e.g.
{
"adressen" : [
{
"start" : {
"ID" : 1,
"key" : "2352KJ25",
"lat" : 52.157225922529967,
"lon" : 4.5298663828345527
}
},
{
"eind" : {
"ID" : 2,
"key" : "2352KJ25",
"lat" : 52.157225922529967,
"lon" : 4.5298663828345527
}
}
}
ID, key, lat, lon are sorted, but start and eind are in their original positions.
So, at least your first, second, third could have been
Json::Value text(Json::arrayValue);
text.append("I");
text.append("love");
text.append("you");
No need for the tags first, second and third!
Maybe this helps you to find a workaround.
I have a very simple problem parsing a yaml file of this form:
- Foo
- Bar:
b1: 5
I would like to parse the top level keys as strings namely "Foo" and "Bar".
As you can see the first entry in the sequence is a scalar and the second is a map containing one key/value pair. Let's say I've loaded this YAML text into a node called config. I iterate over config in the following way:
YAML::Node::const_iterator n_it = config.begin();
for (; n_it != config.end(); n_it++) {
std::string name;
if (n_it->Type() == YAML::NodeType::Scalar)
name = n_it->as<std::string>();
else if (n_it->Type() == YAML::NodeType::Map) {
name = n_it->first.as<std::string>();
}
}
The problem is parsing the second "Bar" entry. I get the following yaml-cpp exception telling me I'm trying to access the key from a sequence iterator n_it.
YAML::InvalidNode: yaml-cpp: error at line 0, column 0: invalid node; this may result from using a map iterator as a sequence iterator, or vice-versa
If I change the access to this:
name = n_it->as<std::string>();
I get a different yaml-cpp exception which I guess is due to the fact that I'm trying to access the whole map as a std::string
YAML::TypedBadConversion<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >: yaml-cpp: error at line 0, column 0: bad conversion
Can somebody please explain to me what's going wrong?
Edit: new problems
I'm still having problems with this api's handling of maps vs sequences. Now say I have the following structure:
foo_map["f1"] = "one";
foo_map["f2"] = "two";
bar_map["b1"] = "one";
bar_map["b2"] = "two";
I want this to be converted to the following YAML file:
Node:
- Foo:
f1 : one
f2 : two
- Bar:
b1 : one
b2 : two
I would do so by doing:
node.push_back("Foo");
node["Foo"]["b1"] = "one";
...
node.push_back("Bar");
However at the last line node has now been converted from a sequence to a map and I get an exception. The only way I can do this is by outputting a map of maps:
Node:
Foo:
f1 : one
f2 : two
Bar:
b1 : one
b2 : two
The problem with this is if I cannot read back such files. If I iterate over Node, I'm unable to even get the type of the node iterator without getting an exception.
YAML::Node::const_iterator n_it = node.begin();
for (; n_it != config.end(); n_it++) {
if (n_it->Type() == YAML::NodeType::Scalar) {
// throws exception
}
}
This should be very simple to handle but has been driving me crazy!
In your expression
name = n_it->first.as<std::string>();
n_it is a sequence iterator (since it's an iterator for your top-level node), which you've just established points to a map. That is,
YAML::Node n = *n_it;
is a map node. This map node (in your example) looks like:
Bar:
b1: 5
In other words, it has a single key/value pair, with the key a string, and the value a map node. It sounds like you want the string key. So:
assert(n.size() == 1); // Verify that there is, in fact, only one key/value pair
YAML::Node::const_iterator sub_it = n.begin(); // This iterator points to
// the single key/value pair
name = sub_it->first.as<std::string>();
Sample.yaml
config:
key1: "SCALER_VAL" # SCALER ITEM
key2: ["val1", "val2"] #SEQUENCE ITEM
key3: # MAP ITEM
nested_key1: "nested_val"
#SAMPLE CODE for Iterate Yaml Node;
YAML::Node internalconfig_yaml = YAML::LoadFile(configFileName);
const YAML::Node &node = internalconfig_yaml["config"];
for(const auto& it : node )
{
std::cout << "\nnested Key: " << it.first.as<std::string>() << "\n";
if (it.second.Type() == YAML::NodeType::Scalar)
{
std::cout << "\nnested value: " << std::to_string(it.second.as<int>()) << "\n";
}
if (it.second.Type() == YAML::NodeType::Sequence)
{
std::vector<std::string> temp_vect;
const YAML::Node &nestd_node2 = it.second;
for(const auto& it2 : nestd_node2)
{
if (*it2)
{
std::cout << "\nnested sequence value: " << it2.as<std::string>() << "\n";
temp_vect.push_back(it2.as<std::string>());
}
}
std::ostringstream oss;
std::copy(temp_vect.begin(), temp_vect.end(),
std::ostream_iterator<std::string>(oss, ","));
std::cout << "\nnested sequence as string: " <<oss.str() << "\n";
}
if (it2.second.Type() == YAML::NodeType::Map)
{
// Iterate Recursively again !!
}
}
Refer here for more details;
This can also be done with the new C++ loop:
std::string name;
for (const auto &entry: node_x) {
assert(name.empty());
name = entry.first.as<std::string>();
}
The assertion will trigger if the node_x is something else than you think. It should be only one entry in this map.
Try something like this:
- Foo: {}
- Bar:
b1: 15
I'm trying to create a JSON array using boost property trees.
The documentation says: "JSON arrays are mapped to nodes. Each element is a child node with an empty name."
So I'd like to create a property tree with empty names, then call write_json(...) to get the array out. However, the documentation doesn't tell me how to create unnamed child nodes. I tried ptree.add_child("", value), but this yields:
Assertion `!p.empty() && "Empty path not allowed for put_child."' failed
The documentation doesn't seem to address this point, at least not in any way I can figure out. Can anyone help?
Simple Array:
#include <boost/property_tree/ptree.hpp>
using boost::property_tree::ptree;
ptree pt;
ptree children;
ptree child1, child2, child3;
child1.put("", 1);
child2.put("", 2);
child3.put("", 3);
children.push_back(std::make_pair("", child1));
children.push_back(std::make_pair("", child2));
children.push_back(std::make_pair("", child3));
pt.add_child("MyArray", children);
write_json("test1.json", pt);
results in:
{
"MyArray":
[
"1",
"2",
"3"
]
}
Array over Objects:
ptree pt;
ptree children;
ptree child1, child2, child3;
child1.put("childkeyA", 1);
child1.put("childkeyB", 2);
child2.put("childkeyA", 3);
child2.put("childkeyB", 4);
child3.put("childkeyA", 5);
child3.put("childkeyB", 6);
children.push_back(std::make_pair("", child1));
children.push_back(std::make_pair("", child2));
children.push_back(std::make_pair("", child3));
pt.put("testkey", "testvalue");
pt.add_child("MyArray", children);
write_json("test2.json", pt);
results in:
{
"testkey": "testvalue",
"MyArray":
[
{
"childkeyA": "1",
"childkeyB": "2"
},
{
"childkeyA": "3",
"childkeyB": "4"
},
{
"childkeyA": "5",
"childkeyB": "6"
}
]
}
What you need to do is this piece of fun. This is from memory, but something like this works for me.
boost::property_tree::ptree root;
boost::property_tree::ptree child1;
boost::property_tree::ptree child2;
// .. fill in children here with what you want
// ...
ptree.push_back( std::make_pair("", child1 ) );
ptree.push_back( std::make_pair("", child2 ) );
But watch out there's several bugs in the json parsing and writing. Several of which I've submitted bug reports for - with no response :(
EDIT: to address concern about it serializing incorrectly as {"":"","":""}
This only happens when the array is the root element. The boost ptree writer treats all root elements as objects - never arrays or values. This is caused by the following line in boost/propert_tree/detail/json_parser_writer.hpp
else if (indent > 0 && pt.count(Str()) == pt.size())
Getting rid of the "indent > 0 &&" will allow it to write arrays correctly.
If you don't like how much space is produced you can use the patch I've provided here
When starting to use Property Tree to represent a JSON structure I encountered similar problems which I did not resolve. Also note that from the documentation, the property tree does not fully support type information:
JSON values are mapped to nodes containing the value. However, all type information is lost; numbers, as well as the literals "null", "true" and "false" are simply mapped to their string form.
After learning this, I switched to the more complete JSON implementation JSON Spirit. This library uses Boost Spirit for the JSON grammar implementation and fully supports JSON including arrays.
I suggest you use an alternative C++ JSON implementation.
In my case I wanted to add an array to a more or less arbitrary location, so, like Michael's answer, create a child tree and populate it with array elements:
using boost::property_tree::ptree;
ptree targetTree;
ptree arrayChild;
ptree arrayElement;
//add array elements as desired, loop, whatever, for example
for(int i = 0; i < 3; i++)
{
arrayElement.put_value(i);
arrayChild.push_back(std::make_pair("",arrayElement))
}
When the child has been populated, use the put_child() or add_child() function to add the entire child tree to the target tree, like this...
targetTree.put_child(ptree::path_type("target.path.to.array"),arrayChild)
the put_child function takes a path and a tree for an argument and will "graft" arrayChild into targetTree
As of boost 1.60.0, problem persists.
Offering a Python 3 workaround (Gist), which can be syscalled just after boost::property_tree::write_json.
#!/usr/bin/env python3
def lex_leaf(lf: str):
if lf.isdecimal():
return int(lf)
elif lf in ['True', 'true']:
return True
elif lf in ['False', 'false']:
return False
else:
try:
return float(lf)
except ValueError:
return lf
def lex_tree(j):
tj = type(j)
if tj == dict:
for k, v in j.items():
j[k] = lex_tree(v)
elif tj == list:
j = [lex_tree(l) for l in j]
elif tj == str:
j = lex_leaf(j)
else:
j = lex_leaf(j)
return j
def lex_file(fn: str):
import json
with open(fn, "r") as fp:
ji = json.load(fp)
jo = lex_tree(ji)
with open(fn, 'w') as fp:
json.dump(jo, fp)
if __name__ == '__main__':
import sys
lex_file(sys.argv[1])
If you want JSON in C++, there's no need for Boost. With this library you can get JSON as a first class data type that behaves like an STL container.
// Create JSON on the fly.
json j2 = {
{"pi", 3.141},
{"happy", true},
{"name", "Niels"},
{"nothing", nullptr},
{"answer", {
{"everything", 42}
}},
{"list", {1, 0, 2}},
{"object", {
{"currency", "USD"},
{"value", 42.99}
}}
};
// Or treat is as an STL container; create an array using push_back
json j;
j.push_back("foo");
j.push_back(1);
j.push_back(true);
// also use emplace_back
j.emplace_back(1.78);
// iterate the array
for (json::iterator it = j.begin(); it != j.end(); ++it) {
std::cout << *it << '\n';
}
Confused with the official document and the above answers.
Below is what I understand.
Property Tree consists of nodes.
Each node is like below
struct ptree
{
map<key_name,value> data;
vector<pair<key_name,ptree>> children;
};
To put 'value' into data with 'put'
To put 'node' into children with 'push_back'\
// Write
bt::ptree root;
bt::ptree active;
bt::ptree requested;
bt::ptree n1, n2, n3;
n1.put("name", "Mark");
n1.put("age", 20);
n1.put("job", "aaa");
n2.put("name", "Rosie");
n2.put("age", "19");
n2.put("job", "bbb");
n3.put("name", "sunwoo");
n3.put("age", "10");
n3.put("job", "ccc");
active.push_back ({ "",l1 });
active.push_back ({ "",l2 });
requested.push_back({ "",l3 });
root.push_back ({"active", active});
root.push_back ({"requested", requested});
bt::write_json("E:\\1.json", root);
// READ
bt::ptree root2;
bt::ptree active2;
bt::ptree requested2;
bt::ptree r1, r2, r3;
bt::read_json("E:\\1.json", root2);
// loop children
for (auto& [k,n] : root.get_child("active"))
{
cout << n.get<string>("name", "unknown");
cout << n.get<int> ("age" , 11);
cout << n.get<string>("job" , "man");
cout << endl << flush;
}