Storing BSONObj object in std::vector crashes with big documents - c++

Question

I have the following fragment of code, that accumulates BSON documents from a query cursor, to be processed afterwards:
// Accumulate
std::vector<BSONObj> results;
while (cursor->more()) {
BSONObj r = cursor->nextSafe();
results.push_back(r);
}
...
// Process it (example)
for (unsigned int ix = 0; ix < results.size(); ix++) {
BSONElement be = results[ix].getField("_id");
// Do somtething with 'be'
...
}
This code has worked well from a time (months) but we have recently found that with large documents in DB (around 1.1MB) the results[ix].getField("_id") statement crashes with segfault. This is the top of backtrace:
(gdb) bt
#0 readNative<int> (offset=0, t=<synthetic pointer>, this=<optimized out>) at src/mongo/base/data_view.h:46
#1 readNative<int> (offset=0, this=<optimized out>) at src/mongo/base/data_view.h:53
#2 readLE<int> (offset=0, this=<optimized out>) at src/mongo/base/data_view.h:59
#3 objsize (this=0x7f74340022e0) at src/mongo/bson/bsonobj.h:309
#4 BSONObjIterator (jso=..., this=<synthetic pointer>) at src/mongo/bson/bsonobjiterator.h:42
#5 mongo::BSONObj::getField (this=0x7f74340022e0, name=...) at src/mongo/bson/bsonobj.cpp:635
...
I have solved the problem using results.push_back(r.copy()) instead of results.push_back(r). Thus, probably the error was caused when the r object is destroyed at the end of the while block scope, leaving the copy pushed back in the vector in an unstable state. Pushing back a copy of r without hitting the block scope as a new variable seems to solve the problem.
So, I have the following questions:
What is the best way of storing BSONObj got from a query result in a std::vector? I think I have found a reasonable solution, but not sure if this is the best one.
Why the code using push_back(r) works with small documents? If the right way is to use r.copy() to avoid problems destroying r at the end of while block scope, I understand it should fail always, not only in the case of objects of around 1.1MB.
I'm using MongoDB C++ driver legacy-1.0.7 (in the case it may help or the problem could be related with specific versions of MongoDB C++ driver).

Answer 1

The BSONObj objects returned by nextSafe do not own their data, and are invalidated by subsequent calls to nextSafe.
So, your vector becomes populated with invalid BSONObj objects.
Instead, call BSONObj::getOwned() on the cursor result before pushing back in the vector.
If you run your program under AddressSanitizer or valgrind, you will almost certainly see use-after-free type errors.