Processing array of generic BSON documents with MongoDB C++ driver - c++

I have the following document in my MongoDB test database:
> db.a.find().pretty()
{
"_id" : ObjectId("5113d680732fb764c4464fdf"),
"x" : [
{
"a" : 1,
"b" : 2
},
{
"a" : 3,
"b" : 4
}
]
}
I'm trying to access and process the elements in the "x" array. However, it seems that the Mongo driver is identifying it not as an array of JSON document, but as Date type, as shown in the following code:
auto_ptr<DBClientCursor> cursor = c.query("test.a", BSONObj());
while (cursor->more()) {
BSONObj r = cursor->next();
cout << r.toString() << std::endl;
}
which output is:
{ _id: ObjectId('51138456732fb764c4464fde'), x: new Date(1360233558334) }
I'm trying to follow the documentation in http://api.mongodb.org/cplusplus and http://docs.mongodb.org/ecosystem/drivers/cpp-bson-array-examples/, but it is quite poor. I have found other examples of processing arrays, but always with simple types (e.g. array of integer), but not when the elements in the array are BSON documents themselves.
Do you have some code example of procesing arrays which elements are generic BSON elements, please?

you could use the .Array() method or the getFieldDotted() method: as in the following:
Query query = Query();
auto_ptr<DBClientCursor> cursor = myConn.query("test.a", query);
while( cursor->more() ) {
BSONObj nextObject = cursor->next();
cout << nextObject["x"].Array()[0]["a"] << endl;
cout << nextObject.getFieldDotted("x.0.a") << endl;
}

At the end, it seems that embeddedObject() method was the key:
auto_ptr<DBClientCursor> cursor = c.query("test.a", BSONObj());
while (cursor->more()) {
BSONObj r = cursor->next();
cout << "Processing JSON document: " << r.toString() << std::endl;
std::vector<BSONElement> be = r.getField("x").Array();
for (unsigned int i = 0; i<be.size(); i++) {
cout << "Processing array element: " << be[i].toString() << std::endl;
cout << " of type: " << be[i].type() << std::endl;
BSONObj bo = be[i].embeddedObject();
cout << "Processing a field: " << bo.getField("a").toString() << std::endl;
cout << "Processing b field: " << bo.getField("b").toString() << std::endl;
}
}
I was wrongly retrieving a different ObjectID and a different type (Date instead of array) becuase I was looking to a different collection :$
Sorry for the noise. I hope that the fragment above helps others to figure out how to manipulate arrays using the MongoDB C++ driver.

Related

Parse nested JSON with QJsonDocument in Qt

I am interested in seeing how we can use Qt's QJsonDocument to parse all entries from a simple nested JSON (as I have just started studying this).
nested json example:
{
"city": "London",
"time": "16:42",
"unit_data":
[
{
"unit_data_id": "ABC123",
"unit_data_number": "21"
},
{
"unit_data_id": "DEF456",
"unit_data_number": "12"
}
]
}
I can parse the non-nested parts of it like so:
QJsonObject jObj;
QString city = jObj["city"].toString();
QString time = jObj["time"].toString();
I am not sure what you are asking, but perhaps this might help:
QJsonDocument doc;
doc = QJsonDocument::fromJson("{ "
" \"city\": \"London\", "
" \"time\": \"16:42\", "
" \"unit_data\": "
" [ "
" { "
" \"unit_data_id\": \"ABC123\", "
" \"unit_data_number\": \"21\" "
" }, "
" { "
" \"unit_data_id\": \"DEF456\", "
" \"unit_data_number\": \"12\" "
" } "
" ] "
" }");
// This part you have covered
QJsonObject jObj = doc.object();
qDebug() << "city" << jObj["city"].toString();
qDebug() << "time" << jObj["time"].toString();
// Since unit_data is an array, you need to get it as such
QJsonArray array = jObj["unit_data"].toArray();
// Then you can manually access the elements in the array
QJsonObject ar1 = array.at(0).toObject();
qDebug() << "" << ar1["unit_data_id"].toString();
// Or you can loop over the items in the array
int idx = 0;
for(const QJsonValue& val: array) {
QJsonObject loopObj = val.toObject();
qDebug() << "[" << idx << "] unit_data_id : " << loopObj["unit_data_id"].toString();
qDebug() << "[" << idx << "] unit_data_number: " << loopObj["unit_data_number"].toString();
++idx;
}
The output I get is:
city "London"
time "16:42"
"ABC123"
[ 0 ] unit_data_id : "ABC123"
[ 0 ] unit_data_number: "21"
[ 1 ] unit_data_id : "DEF456"
[ 1 ] unit_data_number: "12"
In JSON notation, everything should be formatted in key-value. Keys are always strings, but values could be string literals ("example"), number literals , arrays ([]) and objects ({}).
QJsonDocument::fromJson(...).object() returns the root object of a given JSON string. Recall that objects are written by {} notation. This method gives you a QJsonObject. This JSON object has 3 keys ("city", "name" and "unit_data") which value of these keys are of type string literal, string literal and array respectively.
So if you want to access the data stored in that array you should do:
QJsonArray array = rootObj["unit_data"].toArray();
Note that arrays don't have keys, they have only values which could be of the three types mentioned above. In this case, the array holds 2 objects which can be treated as other JSON objects. So,
QJsonObject obj = array.at(0).toObject();
Now the obj object points to the following object:
{
"unit_data_id": "ABC123",
"unit_data_number": "21"
}
So, you should now be able to do what you want. :)
It can happen that one of the elements inside your JSON has more elements inside. It can also happen that you don't know the characteristics of the file (or you want to have a general function).
Therefore you can use a function for any JSON:
void traversJson(QJsonObject json_obj){
foreach(const QString& key, json_obj.keys()) {
QJsonValue value = json_obj.value(key);
if(!value.isObject() ){
qDebug() << "Key = " << key << ", Value = " << value;
}
else{
qDebug() << "Nested Key = " << key;
traversJson(value.toObject());
}
}
};

How to save specific values in a list to txt using jsoncpp?

I have yahoo finance json file from which I want to isolate Date,Close and volume from the quote list and save it in the same order with a comma separtion in a single text file. This is my json script.
Json::Value root; // will contains the root value after parsing.
Json::Reader reader;
bool parsingSuccessful = reader.parse( YahooJson, root );
if(not parsingSuccessful)
{
// Report failures and their locations
// in the document.
std::cout<<"Failed to parse JSON"<<std::endl
<<reader.getFormatedErrorMessages()
<<std::endl;
return 1;
}else{
std::cout<<"\nSucess parsing json\n"<<std::endl;
std::cout << root<< std::endl;
std::cout <<"No of Days = "<< root["query"]["count"].asInt() << std::endl;
//below for loop returns an error
for (auto itr : root["query"]["result"]["quote"]) {
std::string val = itr.asString();
}
}
I was able to succed in fetching the json values and print root["query"]["count"].asInt() but when I go to the list values(quote) I dont know how to iterate through quote (query->result->quote) to get Date,close and volume values?
EDIT
Also tried this method
const Json::Value& quotes = root["query"]["results"]["quote"];
for (int i = 0; i < quotes.size(); i++){
std::cout << " Date: " << quotes[i]["Date"].asString();
std::cout << " Close: " << quotes[i]["Close"].asFloat();
std::cout << " Volume: " << quotes[i]["Volume"].asFloat();
std::cout << std::endl;
}
It works only when output was Date. For close and volume output it exits with a runtime error message and also this error
what() type is not convertible to string
You haven't specified which JSON library you are using, and I don't know the Yahoo finance data well enough to know the exact field names, but if you are using the JsonCpp library, which has documentation here, and you are asking about how to iterate over a JSON array, then one way to do it using iterators would look something like this
const Json::Value quote = root["query"]["results"]["quote"];
for (Json::ValueConstIterator itr = quote.begin(); itr != quote.end(); ++itr)
{
const Json::Value date = (*itr)["Date"];
const Json::Value close = (*itr)["Close"];
const Json::Value volume = (*itr)["Volume"];
std::cout << "Date: " << date.asString() << std::endl;
std::cout << "Close: " << close.asString() << std::endl;
std::cout << "Volume: " << volume.asString() << std::endl;
}

accessing elements from nlohmann json

My JSON file resembles this
{
"active" : false,
"list1" : ["A", "B", "C"],
"objList" : [
{
"key1" : "value1",
"key2" : [ 0, 1 ]
}
]
}
Using nlohmann json now, I've managed to store it and when I do a dump jsonRootNode.dump(), the contents are represented properly.
However I can't find a way to access the contents.
I've tried jsonRootNode["active"], jsonRootNode.get() and using the json::iterator but still can't figure out how to retrieve my contents.
I'm trying to retrieve "active", the array from "list1" and object array from "objList"
The following link explains the ways to access elements in the JSON. In case the link goes out of scope here is the code
#include <json.hpp>
using namespace nlohmann;
int main()
{
// create JSON object
json object =
{
{"the good", "il buono"},
{"the bad", "il cativo"},
{"the ugly", "il brutto"}
};
// output element with key "the ugly"
std::cout << object.at("the ugly") << '\n';
// change element with key "the bad"
object.at("the bad") = "il cattivo";
// output changed array
std::cout << object << '\n';
// try to write at a nonexisting key
try
{
object.at("the fast") = "il rapido";
}
catch (std::out_of_range& e)
{
std::cout << "out of range: " << e.what() << '\n';
}
}
In case anybody else is still looking for the answer.. You can simply access the contents using the same method as for writing to an nlohmann::json object. For example to get values from
json in the question:
{
"active" : false,
"list1" : ["A", "B", "C"],
"objList" : [
{
"key1" : "value1",
"key2" : [ 0, 1 ]
}
]
}
just do:
nlohmann::json jsonData = nlohmann::json::parse(your_json);
std::cout << jsonData["active"] << std::endl; // returns boolean
std::cout << jsonData["list1"] << std::endl; // returns array
If the "objList" was just an object, you can retrieve its values just by:
std::cout << jsonData["objList"]["key1"] << std::endl; // returns string
std::cout << jsonData["objList"]["key2"] << std::endl; // returns array
But since "objList" is a list of key/value pairs, to access its values use:
for(auto &array : jsonData["objList"]) {
std::cout << array["key1"] << std::endl; // returns string
std::cout << array["key2"] << std::endl; // returns array
}
The loop runs only once considering "objList" is array of size 1.
Hope it helps someone
I really like to use this in C++:
for (auto& el : object["list1"].items())
{
std::cout << el.value() << '\n';
}
It will loop over the the array.

MongoDB 3.2 c++ drivers , using $exists

bsoncxx::builder::stream::document search_builder;
mongocxx::options::find img_find; // This speeds up the queries
search_builder_images.clear();
search_builder_images << "_id" << "abc" << "data" << open_document <<"$exists" << true << close_document ;
for (bsoncxx::document::view doc : cursor_cal) {
std::cout << bsoncxx::to_json(doc) << std::endl;
}
auto cursor_cal = dbMongo[collectionName].find(search_builder.view());
Here randomly 50-50% chances , I sometimes get the output I expect and sometimes I get segmentation fault error.
What am I doing wrong ? ( I am trying to create this search_builder to search in mongodb database and get documents where data exists ? )
This is a bit old but I was having a segfault issue for the construction of the document, not sure if its what you were facing. I had to break the query document construction into multiple lines, such as :
auto queryDoc = document{};
queryDoc << _id << "abc";
queryDoc << "data" << open_document;
queryDoc << "$exists" << true;
queryDoc << close_document;
auto query = queryDoc << finalize;
Hope this helps someone else.

Accessing elements in maps of maps with vectors of shared pointers

I have a somewhat unique situation that I can't quite get working.
I've followed a lot of examples of using maps of maps but the vector of shared pointers seems to throw me off a bit.
Suppose I have the following:
typedef boost::shared_ptr<RecCounts> RecCountsPtr;
typedef std::vector<RecCountsPtr> RecCountsPtrVec;
typedef std::map<std::string, RecCountsPtrVec> InnerActivityMap;
typedef std::map< std::string, InnerActivityMap > ActivityMap;
Where RecCounts is a simple structure.
Now, I think I've figured out how to populate my ActivityMap properly.
RecCountsPtr recCountsPtr(new RecCounts());
config.actType = "M";
config.mapDate = "2010/07";
recCountsPtr->iHousehold = "50";
recCountsPtr->iZero = "150";
config.actMap[config.actType][config.mapDate].push_back(recCountsPtr);
Yes? I don't get any compile/runtime errors for this...but since I haven't figured out how to access all the different elements of the map I can't confirm this!
This is the config structure:
struct Config
{
std::string actType;
std::string mapDate;
// Map
ActivityMap actMap;
InnerActivityMap innerActMap;
//Iterator
ActivityMap::iterator actMapIter;
InnerActivityMap::iterator innerActMapIter;
};
Now, suppose I want to access each element of the ActivityMap. How would I get the following elements?
The outer map Key?
for (config.actMapIter= config.actMap.begin();
config.actMapIter != config.actMap.end();
++config.actMapIter)
{
std::cout << "Outer Key = "
<< (*config.actMapIter).first << std::endl;
}
This seemed to do the trick.
The inner map Key?
I can't figure this out.
The inner map vector elements?
I can do it like this if I know the two keys:
config.actMap[config.actType][config.mapDate][0]->iHouehold
config.actMap[config.actType][config.mapDate][0]->iZero
...but can't seem to figure out how to iterate through them. :(
This is what I've done to try and iterate through all elements.
for (config.actMapIter= config.actMap.begin();
config.actMapIter != config.actMap.end();
++config.actMapIter)
{
std::cout << "Outer Key = " << (*config.actMapIter).first << std::endl;
for (config.innerActMapIter = config.innerActMap.begin();
config.innerActMapIter != config.innerActMap.end();
++config.innerActMapIter)
{
std::cout << "Inner Key = "
<< (*config.innerActMapIter).first << std::endl;
for (size_t i = 0;
i < config.actMap[(*config.actMapIter).first]
[(*config.innerActMapIter).first].size();
++i)
{
std::cout << "iHousehold = "
<< config.actMap[(*config.actMapIter).first]
[(*config.innerActMapIter).first]
[i]->iHousehold << std::endl;
std::cout << "iZero = "
<< config.actMap[(*config.actMapIter).first]
[(*config.innerActMapIter).first]
[i]->iZero << std::endl;
}
}
}
I don't get any errors but I only get the outer key print to the screen:
Outer Key = M
I suspect something is amiss with my inner iterator...in that it's not associated the ActivityMap. Even if I am correct I don't know how to make such an association.
Any suggestions?
ANSWER (thanks to crashmstr):
Here is a verbose version of the answer suggested by crashmstr.
for (config.actMapIter= config.actMap.begin();
config.actMapIter != config.actMap.end();
++config.actMapIter)
{
std::cout << "Outer Key = " << (*config.actMapIter).first << std::endl;
InnerActivityMap innerActMap = (*config.actMapIter).second;
InnerActivityMap::iterator innerActMapIter;
for (innerActMapIter = innerActMap.begin();
innerActMapIter != innerActMap.end();
++innerActMapIter)
{
std::cout << "Inner Key = " << (*innerActMapIter).first << std::endl;
for (size_t i = 0;
i < config.actMap[(*config.actMapIter).first][(*innerActMapIter).first].size();
++i)
{
std::cout << "iHousehold = "
<< config.actMap[(*config.actMapIter).first]
[(*innerActMapIter).first]
[i]->iHousehold << std::endl;
std::cout << "iZero = "
<< config.actMap[(*config.actMapIter).first]
[(*innerActMapIter).first]
[i]->iZero << std::endl;
}
}
}
I get the following printed to the screen:
Outer Key = M
Inner Key = 2010/07
iHousehold = 50
iZero = 150
When iterating over a map, .first is the "key", and .second is the data that belongs to that key.
So in your case:
for (config.actMapIter= config.actMap.begin();
config.actMapIter != config.actMap.end();
++config.actMapIter)
{
std::cout << "Outer Key = " << (*config.actMapIter).first << std::endl;
//(*config.actMapIter).second is a std::map<std::string, RecCountsPtrVec>
//create a new for loop to iterate over .second
}