How to parse nested arrays inside json using C++ - c++

I know how to parse "normal" looking JSON data in C++. Usually, I do this, using boost::property_tree and read_json method. It may look like so:
BOOST_FOREAH(ptree::value_type &v, pt.get_child("rows"){
vec.push_back(v.second.get<std::string>("key"));
}
and the code above corresponds to this JSON file:
{
"rows":[{
"key":"1"
},{
"key":"2"
}]
}
However, the Neo4j result-set that I get, looks like:
{
"columns":{...},
"data":[[["object 1"]], [["object 2"]], [["object 3"]]]
}
I'm interested and want to parse "data" node. I tried to do it like so:
BOOST_FOREAH(ptree::value_type &v, pt.get_child("data"){
vec.push_back(v.second.data());
}
but this does not work. I do not get an error, but my vector vec remains empty, or to be more precise it is populated with empty values. So, that when I iterate through this vec I see a number of elements, but they do not have any value. Whereas, I want to have values "object 1", "object 2", "object 3".

The solution looks like this:
using boost::property::ptree;
ptree pt;
//... populate ptree pt with data from some source
BOOST_FOREACH(ptree::value_type &v, pt.get_child('data')){
ptree subtree1 = v.second;
BOOST_FOREACH(ptree::value_type &vs, subtree1){
ptree subtree2 = vs.second;
BOOST_FOREACH(ptree::value_type &vs2, subtree2){
do_something(vs2.second.data());
}
}
}
This code makes it possible to parse such JSON structure:
{
"data":[[["object 1"]], [["object 2"]], [["object 3"]]]
}
So, contrary to what some people are saying, actually, there is no need to use other third-party libraries. Use just boost and you are done.

This is an example of how I do it. You have to know the JSON structure ahead of time.
#include <boost/lexical_cast.hpp>
#include <boost/property_tree/ptree.hpp>
#include <boost/property_tree/json_parser.hpp>
boost::property_tree::ptree pt, sub_pt;
std::string json_str, key, sub_key;
std::stringstream ss;
int value = 0, bus_num = 0;
json_str = "{\"arduino_1\": {\"bus_1\": 17425,\"bus_2\": 1025,\"bus_3\": 0,\"bus_4\": 0,\"bus_5\": 0,\"bus_6\": 0,\"bus_7\": 0,\"bus_8\": 0}}";
ss << json_str; // put string into stringstream
boost::property_tree::read_json(ss, pt); // put stringstream into property tree
for (boost::property_tree::ptree::iterator iter = pt.begin(); iter != pt.end(); iter++)
{
// get data
key = boost::lexical_cast <std::string>(iter->first.data());
sub_pt = iter->second;
// iterate over subtree
for (boost::property_tree::ptree::iterator sub_iter = sub_pt.begin(); sub_iter != sub_pt.end(); sub_iter++)
{
// get data
sub_key = boost::lexical_cast <std::string>(sub_iter->first.data());
value = boost::lexical_cast <int>(sub_iter->second.data());
}
}

Related

Write in JSON file - Data not inserted in the correct order

I am creating a desktop app using QT C++ that take a text file and convert it to JSON File like this example:
{
"102": {
"NEUTRAL": {
"blend": "100"
},
"AE": {
"blend": "100"
}
},
"105": {
"AE": {
"blend": "100"
},
"NEUTRAL": {
"blend": "100"
}
}
}
This is the code I am using:
for (int i = 0; i < output_list1.size(); i++) {
if (output_list1[i] == "-") {
c_frame++;
continue;
}
if (output_list1[i] != "NEUTRAL") {
QJsonObject neutralBlendObject;
neutralBlendObject.insert("blend", "100");
QJsonObject phonemeObject;
phonemeObject.insert("NEUTRAL", neutralBlendObject);
QJsonObject keyBlendObject;
keyBlendObject.insert("blend", output_list1[i].split(' ')[1]);
phonemeObject.insert(output_list1[i].split(' ')[0], keyBlendObject);
mainObject.insert(QString::number(c_frame), phonemeObject);
}
c_frame++;
}
jsonDoc.setObject(mainObject);
file.write(jsonDoc.toJson());
file.close();
As you can see, I am inserting the NEUTRAL object first but I am getting data not in the correct order, somtimes NEUTRAL is the the first following with the next object and somtimes not.
How can I correct this issue?
In JSON there are two structures you can use for saving data:
1.) JSON Objects: A collection of key/value pairs. This collection is NOT ordered. In various languages its realized via associative data structures like dictionaries, hash tables etc. Internally qt saves JSON objects via QHash<QString, QVariant> containers.
2.) JSON Arrays: An ordered list of values (which can be JSON Objects). In programming languages this is realized as an array/vector/whatever. Qt uses a QVariantList (QList<QVariant) internally.
I did resolve it using rapidjson library instead of QJsonObject
Based on this example
#include <rapidjson/document.h>
#include <rapidjson/writer.h>
#include <rapidjson/stringbuffer.h>
#include "rapidjson/filewritestream.h"
#include <string>
#include "RapidJson/prettywriter.h"
using namespace rapidjson;
using namespace std;
FILE* fp = fopen("output.json", "wb"); // non-Windows use "w"
char writeBuffer[65536];
FileWriteStream os(fp, writeBuffer, sizeof(writeBuffer));
Document d;
d.SetObject();
rapidjson::Document::AllocatorType& allocator = d.GetAllocator();
size_t sz = allocator.Size();
d.AddMember("version", 1, allocator);
d.AddMember("testId", 2, allocator);
d.AddMember("group", 3, allocator);
d.AddMember("order", 4, allocator);
Value tests(kArrayType);
Value obj(kObjectType);
Value val(kObjectType);
string description = "a description";
//const char *description = "a description";
//val.SetString()
val.SetString(description.c_str(), static_cast<SizeType>(description.length()), allocator);
obj.AddMember("description", val, allocator);
string help = "some help";
val.SetString(help.c_str(), static_cast<SizeType>(help.length()), allocator);
obj.AddMember("help", val, allocator);
string workgroup = "a workgroup";
val.SetString(workgroup.c_str(), static_cast<SizeType>(workgroup.length()), allocator);
obj.AddMember("workgroup", val, allocator);
val.SetBool(true);
obj.AddMember("online", val, allocator);
tests.PushBack(obj, allocator);
d.AddMember("tests", tests, allocator);
// Convert JSON document to string
PrettyWriter<FileWriteStream> writer(os);
char indent = ' '; // single space x 4
writer.SetIndent(indent, 4);
d.Accept(writer);
Thank you all for your time

How to use boost property tree to parse elements from array in json string using boost?

I have a json string that looks like this:
[
"some text",
648547,
94.0,
111.0267520223,
10
]
so I need to assign a variable to each value like:
std::string value1 = "some text";
int value2 = 648547;
float value3 = 94.0;
float value4 = 111.0267520223;
int value5 = 10;
to read JSON, with Boost, I was doing something like this
std::stringstream jsonResponse;
boost::property_tree::ptree pt;
jsonResponse << "[\"some text\", 648547, 94.0, 111.0267520223, 10]";
std::istringstream is(jsonResponse);
boost::property_tree::read_json(is, pt);
But I don't know how to read array values from a property tree.
Does anyone have an idea how to do it?
thanks in advance!
Here my solution to iterate over no naming array:
boost::property_tree::basic_ptree<std::string,std::string>::const_iterator iter = pt.begin(),iterEnd = pt.end();
for(;iter != iterEnd;++iter)
{
//->first; // Key. Array elements have no names
//->second; // The object at each step
std::cout << "=> " << iter->second.get_value<std::string>() << std::endl;
}
You'll need to name the array so that it can be referenced:
{
"blah": [
"some text",
648547,
94.0,
111.0267520223,
10
]
}
This will validate on jsonlint.com, but it's still not simple to read using a property tree.
#include <boost/property_tree/json_parser.hpp>
#include <boost/property_tree/ptree.hpp>
#include <boost/property_tree/exceptions.hpp>
#include <boost/exception/diagnostic_information.hpp>
#include <boost/foreach.hpp>
typedef boost::property_tree::iptree ptree_t;
typedef ptree_t::value_type ptree_value_t;
typedef boost::optional<ptree_t &> optional_ptree_t;
void parseMyJson()
{
optional_ptree_t ptBlah = pt.get_child_optional("blah");
if (ptBlah)
{
BOOST_FOREACH (property_tree_t::value_type & field, pt.get_child("blah"))
{
}
}
}
With this kind of code you can iterate the fields in blah, but since they're different types, its not straightforward to parse.
I would suggest that you consider naming the fields so they can be directly referenced.
e.g.
field.second.get<string>("fieldname", "");
Please remember to wrap this code in a trycatch block, since boost property trees throw exceptions at the first sign of a problem (e.g. parse failure, or field not found etc.)
You might like to consider a more user friendly json library (https://github.com/nlohmann/json).

Boost and xml parsing

I have following xml data and i want to parse through boost xml parser.
<?xml version="1.0" encoding="UTF-8"?>
<applications>
<application>
<id>1</id>
<platform>linux-x64</platform>
<version>2.4</version>
</application>
<application>
<id>2</id>
<platform>windows</platform>
<version>2.5</version>
</application>
<application>
<id>3</id>
<platform>linux</platform>
<version>2.6</version>
</application>
</applications>
I have written below boost code but I read only first child of "applications" and not able to read other two childs. Everytime inner loop get the data of first child.
boost::property_tree::ptree pt;
boost::property_tree::read_xml(sModel, pt); // sModel is filename which contains above xml data
BOOST_FOREACH(boost::property_tree::ptree::value_type &v, pt.get_child("applications"))
{
std::string key = v.first.data();
std::string Id, platform, version;
if (key == std::string("application"))
{
BOOST_FOREACH(boost::property_tree::ptree::value_type &v_, pt.get_child("applications.application"))
{
std::string app_key = v_.first.data();
std::string app_value = v_.second.data();
if (app_key == std::string("id"))
pkgId = app_value;
else if (app_key == std::string("platform"))
platform = app_value;
else if (app_key == std::string("version"))
version = app_value;
}
}
}
Here, every time i get the platform as "linux-x64".
Can someone guide how to read all the child through this boost xml ?
Thanks in Advance.
get_child (and all the other path-based access functions) isn't very good at dealing with multiple identical keys. It will choose the first child with the given key and return that, ignoring all others.
But you don't need get_child, because you already hold the node you want in your hand.
pt.get_child("applications") gives you a ptree. Iterating over that gives you a ptree::value_type, which is a std::pair<std::string, ptree>.
The first weird thing, then, is this line:
std::string key = v.first.data();
The data() function you're calling here is std::string::data, not ptree::data. You could just write
std::string key = v.first;
The next strange thing is the comparison:
if (key == std::string("application"))
You don't need to explicitly construct a std::string here. In fact, doing so is a pessimization, because it has to allocate a string buffer and copy the string there, when std::string has comparison operators for C-style strings.
Then you iterator over pt.get_child("applications.application"), but you don't need to do this lookup - v.second is already the tree you want.
Furthermore, you don't need to iterate over the child at all, you can use its lookup functions to get what you need.
std::string pkgId = v.second.get("id", "");
So to sum up, this is the code I would write:
boost::property_tree::ptree pt;
boost::property_tree::read_xml(sModel, pt);
BOOST_FOREACH(boost::property_tree::ptree::value_type &v, pt.get_child("applications"))
{
// You can even omit this check if you can rely on all children
// being application nodes.
if (v.first == "application")
{
std::string pkgId = v.second.get("id", "");
std::string platform = v.second.get("platform", "");
std::string version = v.second.get("version", "");
}
}
Check this example:
#include <boost/property_tree/xml_parser.hpp>
#include <boost/property_tree/ptree.hpp>
#include <boost/foreach.hpp>
struct Application
{
int m_id
std::string m_platform;
float m_version;
};
typedef std::vector<Application> AppList;
AppList Read()
{
using boost::property_tree::ptree;
// Populate tree structure (pt):
ptree pt;
read_xml("applications.xml", pt); // For example.
// Traverse pt:
AppList List;
BOOST_FOREACH(ptree::value_type const& v, pt.get_child("applications"))
{
if (v.first == "application")
{
Application App;
App.id = v.second.get<int>("id");
App.platform = v.second.get<std::string>("platform");
App.version = v.second.get<float>("version");
List.push_back(App);
}
}
return List;
}

JSONCPP Writing to files

JSONCPP has a writer, but all it seems to do is get info from the parser and then output it into a string or a stream. How do I make it alter or create new objects, arrays, values, strings, et cetera and write them into the file?
#include<json/writer.h>
Code:
Json::Value event;
Json::Value vec(Json::arrayValue);
vec.append(Json::Value(1));
vec.append(Json::Value(2));
vec.append(Json::Value(3));
event["competitors"]["home"]["name"] = "Liverpool";
event["competitors"]["away"]["code"] = 89223;
event["competitors"]["away"]["name"] = "Aston Villa";
event["competitors"]["away"]["code"]=vec;
std::cout << event << std::endl;
Output:
{
"competitors" :
{
"away" :
{
"code" : [ 1, 2, 3 ],
"name" : "Aston Villa"
},
"home" :
{
"name" : "Liverpool"
}
}
}
#include <json.h>
#include <iostream>
#include <fstream>
void main()
{
std::ofstream file_id;
op_file_id.open("file.txt");
Json::Value value_obj;
//populate 'value_obj' with the objects, arrays etc.
Json::StyledWriter styledWriter;
file_id << styledWriter.write(value_obj);
file_id.close();
}
AFAICT, you create objects of type Json::Value, which caters for all the JSON data-types, and pass the result to a Json::Writer (one of its derived types, to be specific), or simply to a stream.
E.g.: to write an array of three integers to a file:
Json::Value vec(Json::arrayValue);
vec.append(Json::Value(1));
vec.append(Json::Value(2));
vec.append(Json::Value(3));
std::cout << vec;
Json::StyledWriter is deprecated, you can use Json::StreamWriterBuilder to write json into files.
Json::Value rootJsonValue;
rootJsonValue["foo"] = "bar";
Json::StreamWriterBuilder builder;
builder["commentStyle"] = "None";
builder["indentation"] = " ";
std::unique_ptr<Json::StreamWriter> writer(builder.newStreamWriter());
std::ofstream outputFileStream("/tmp/test.json");
writer -> write(rootJsonValue, &outputFileStream);
The json will be written into /tmp/test.json.
$ cat /tmp/test.json
{
"foo" : "bar"
}
First, you have to create the desired JSON::Value. You should look at all the constructors (first). To create the necessary hierarchies, see append and the operator[] overloads; there are overloads for both array indices and string keys for objects.
One way to write the JSON value back out is using StyledStreamWriter::write and ofstream.
See cegprakash's answer for how to write it.

Creating JSON arrays in Boost using Property Trees

I'm trying to create a JSON array using boost property trees.
The documentation says: "JSON arrays are mapped to nodes. Each element is a child node with an empty name."
So I'd like to create a property tree with empty names, then call write_json(...) to get the array out. However, the documentation doesn't tell me how to create unnamed child nodes. I tried ptree.add_child("", value), but this yields:
Assertion `!p.empty() && "Empty path not allowed for put_child."' failed
The documentation doesn't seem to address this point, at least not in any way I can figure out. Can anyone help?
Simple Array:
#include <boost/property_tree/ptree.hpp>
using boost::property_tree::ptree;
ptree pt;
ptree children;
ptree child1, child2, child3;
child1.put("", 1);
child2.put("", 2);
child3.put("", 3);
children.push_back(std::make_pair("", child1));
children.push_back(std::make_pair("", child2));
children.push_back(std::make_pair("", child3));
pt.add_child("MyArray", children);
write_json("test1.json", pt);
results in:
{
"MyArray":
[
"1",
"2",
"3"
]
}
Array over Objects:
ptree pt;
ptree children;
ptree child1, child2, child3;
child1.put("childkeyA", 1);
child1.put("childkeyB", 2);
child2.put("childkeyA", 3);
child2.put("childkeyB", 4);
child3.put("childkeyA", 5);
child3.put("childkeyB", 6);
children.push_back(std::make_pair("", child1));
children.push_back(std::make_pair("", child2));
children.push_back(std::make_pair("", child3));
pt.put("testkey", "testvalue");
pt.add_child("MyArray", children);
write_json("test2.json", pt);
results in:
{
"testkey": "testvalue",
"MyArray":
[
{
"childkeyA": "1",
"childkeyB": "2"
},
{
"childkeyA": "3",
"childkeyB": "4"
},
{
"childkeyA": "5",
"childkeyB": "6"
}
]
}
What you need to do is this piece of fun. This is from memory, but something like this works for me.
boost::property_tree::ptree root;
boost::property_tree::ptree child1;
boost::property_tree::ptree child2;
// .. fill in children here with what you want
// ...
ptree.push_back( std::make_pair("", child1 ) );
ptree.push_back( std::make_pair("", child2 ) );
But watch out there's several bugs in the json parsing and writing. Several of which I've submitted bug reports for - with no response :(
EDIT: to address concern about it serializing incorrectly as {"":"","":""}
This only happens when the array is the root element. The boost ptree writer treats all root elements as objects - never arrays or values. This is caused by the following line in boost/propert_tree/detail/json_parser_writer.hpp
else if (indent > 0 && pt.count(Str()) == pt.size())
Getting rid of the "indent > 0 &&" will allow it to write arrays correctly.
If you don't like how much space is produced you can use the patch I've provided here
When starting to use Property Tree to represent a JSON structure I encountered similar problems which I did not resolve. Also note that from the documentation, the property tree does not fully support type information:
JSON values are mapped to nodes containing the value. However, all type information is lost; numbers, as well as the literals "null", "true" and "false" are simply mapped to their string form.
After learning this, I switched to the more complete JSON implementation JSON Spirit. This library uses Boost Spirit for the JSON grammar implementation and fully supports JSON including arrays.
I suggest you use an alternative C++ JSON implementation.
In my case I wanted to add an array to a more or less arbitrary location, so, like Michael's answer, create a child tree and populate it with array elements:
using boost::property_tree::ptree;
ptree targetTree;
ptree arrayChild;
ptree arrayElement;
//add array elements as desired, loop, whatever, for example
for(int i = 0; i < 3; i++)
{
arrayElement.put_value(i);
arrayChild.push_back(std::make_pair("",arrayElement))
}
When the child has been populated, use the put_child() or add_child() function to add the entire child tree to the target tree, like this...
targetTree.put_child(ptree::path_type("target.path.to.array"),arrayChild)
the put_child function takes a path and a tree for an argument and will "graft" arrayChild into targetTree
As of boost 1.60.0, problem persists.
Offering a Python 3 workaround (Gist), which can be syscalled just after boost::property_tree::write_json.
#!/usr/bin/env python3
def lex_leaf(lf: str):
if lf.isdecimal():
return int(lf)
elif lf in ['True', 'true']:
return True
elif lf in ['False', 'false']:
return False
else:
try:
return float(lf)
except ValueError:
return lf
def lex_tree(j):
tj = type(j)
if tj == dict:
for k, v in j.items():
j[k] = lex_tree(v)
elif tj == list:
j = [lex_tree(l) for l in j]
elif tj == str:
j = lex_leaf(j)
else:
j = lex_leaf(j)
return j
def lex_file(fn: str):
import json
with open(fn, "r") as fp:
ji = json.load(fp)
jo = lex_tree(ji)
with open(fn, 'w') as fp:
json.dump(jo, fp)
if __name__ == '__main__':
import sys
lex_file(sys.argv[1])
If you want JSON in C++, there's no need for Boost. With this library you can get JSON as a first class data type that behaves like an STL container.
// Create JSON on the fly.
json j2 = {
{"pi", 3.141},
{"happy", true},
{"name", "Niels"},
{"nothing", nullptr},
{"answer", {
{"everything", 42}
}},
{"list", {1, 0, 2}},
{"object", {
{"currency", "USD"},
{"value", 42.99}
}}
};
// Or treat is as an STL container; create an array using push_back
json j;
j.push_back("foo");
j.push_back(1);
j.push_back(true);
// also use emplace_back
j.emplace_back(1.78);
// iterate the array
for (json::iterator it = j.begin(); it != j.end(); ++it) {
std::cout << *it << '\n';
}
Confused with the official document and the above answers.
Below is what I understand.
Property Tree consists of nodes.
Each node is like below
struct ptree
{
map<key_name,value> data;
vector<pair<key_name,ptree>> children;
};
To put 'value' into data with 'put'
To put 'node' into children with 'push_back'\
// Write
bt::ptree root;
bt::ptree active;
bt::ptree requested;
bt::ptree n1, n2, n3;
n1.put("name", "Mark");
n1.put("age", 20);
n1.put("job", "aaa");
n2.put("name", "Rosie");
n2.put("age", "19");
n2.put("job", "bbb");
n3.put("name", "sunwoo");
n3.put("age", "10");
n3.put("job", "ccc");
active.push_back ({ "",l1 });
active.push_back ({ "",l2 });
requested.push_back({ "",l3 });
root.push_back ({"active", active});
root.push_back ({"requested", requested});
bt::write_json("E:\\1.json", root);
// READ
bt::ptree root2;
bt::ptree active2;
bt::ptree requested2;
bt::ptree r1, r2, r3;
bt::read_json("E:\\1.json", root2);
// loop children
for (auto& [k,n] : root.get_child("active"))
{
cout << n.get<string>("name", "unknown");
cout << n.get<int> ("age" , 11);
cout << n.get<string>("job" , "man");
cout << endl << flush;
}