rapidjson producing inconsistent results when converting to string - c++

I currently have a JSON serializer class that I'm using. It's for some experimental code that I'm working on. This code is using cpprestsdk. The serialization is setup to use either rapidjson or cpprestsdk's json.
So for example, the virtual functions for the class look like:
virtual void toJson(rapidjson::Document& json) const =0;
virtual void toJson(web::json::value& json) const =0;
virtual void fromJson(const rapidjson::Value& json) =0;
virtual void fromJson(const web::json::value& json) =0;
I can convert from JSON no problem. I'm currently doing the conversion of a class object to JSON, and then exporting it as a string to a file. I have found that with rapidjson, I get variable results.
On some exports, I see a snippet like this:
"base": {
"name\u0000refere": "base",
On other runs, I see a snippet like this:
"base": {
"name": "base",
This is for successive runs, with no changes to the code.
The fields are actually globally defined const char * like so:
const char *kSymbolKeyName = "name";
const char *kSymbolKeyReferenceName = "referenceName";
The code to generate the JSON object that has the issue looks like:
void Object::toJson(rapidjson::Document& json) const {
using namespace rapidjson;
json.SetObject(); // Reset and clear any existing
auto& allocator = json.GetAllocator();
json.AddMember(StringRef(kObjectKeyName), Value(name.c_str(), allocator), allocator);
json.AddMember(StringRef(kObjectKeyPrioritizeTable), Value(prioritizeTable), allocator);
json.AddMember(StringRef(kObjectKeyPrioritizeGreaterOn), Value(prioritizeGreaterOn), allocator);
}
Note that kObjectKeyNameis defined as const char *kObjectKeyName = "name";.
And the caller to this class's toJson would look like:
using namespace rapidjson;
json.SetObject(); // Reset and clear any existing
auto& allocator = json.GetAllocator();
for (const auto& it : tables) {
Document iJson;
it.second->toJson(iJson);
json.AddMember(Value(it.first.c_str(), allocator), iJson, allocator);
}
Part of the problem may stem from the way I am using rapidjson::Documents and allocators. I believe toJsoncall will end up with it's own allocator once I make the SetObjectcall.
My plan is to revamp the code to use Value instead of Document in toJson and then pass the allocator in as an argument. I ideally didn't want to do that mainly because I was being lazy and wanted the signature to the same so it was easy to flip between rapidjson or cppsrestsdk's son.
Oh yeah, the code to output the file as a string is the following
std::ofstream out("output.json");
rapidjson::Document outDoc;
dataSet.toJson(outDoc);
rapidjson::StringBuffer buffer;
buffer.Clear();
rapidjson::PrettyWriter<rapidjson::StringBuffer> writer(buffer);
outDoc.Accept(writer);
out << buffer.GetString();
out.close();
There is no doubt something that I am doing odd/dumb as just recently started using rapidjson. I'm just trying to narrow down my issues and better understand the error of my ways.

It appears if modifying the process to pass in an allocator works.
I modified my toJson function to be
rapidjson::Value toJson(rapidjson::Document::AllocatorType& allocator);
In my usage, it means all generated Values, when needed, will use the base Document's allocator.

Related

gmock save argument string

I hope there is an easier way to do this... I need to capture the string which is passed as an argument to a mock.
The mock
class web_api_mock : public iweb_api
{
public:
MOCK_METHOD(
(bool),
http_post,
(const etl_normal_string &, const char*),
(override));
};
I want to capture the char * passed to the mock as second argument. I need to construct some json structure from it, and I want to check if a certain element has a certain value.
I had to spend a lot of time to get it to work, eventually copying the trick from here. This brilliant mind figured out you can rely on the gmock's Invoke.
The EXPECT_CALL
http_post_args args;
EXPECT_CALL(_web_api_mock, http_post(etl_string_equals(url), _))
.WillOnce(
DoAll(
Invoke(&args, &http_post_args::capture),
Return(true)));
Here I am invoking all arguments of the mock to a struct which I defined as follows
struct http_post_args
{
void capture(etl_normal_string url, const char * p)
{
payload = std::string(p);
}
std::string payload;
};
And finally, I get my hands on the char * and do whatever I want afterwards.
It seems awfully complicated to save an argument when it's of the type char *.
My first attempt was the obvious mistake I guess many before (and after) me made: using the SaveArgPointee which will copy only the first element of the string and gives me with a string where the first character is correct, but the remaining string is filled with random mem.
My second attempt was to define an ACTION_P. This "almost" worked. In the callstack I could see the string I am interested in until the very last stackframe, where the args simply seem not to be passed to the actual implementation of my custom ACTION_P.
ACTION_P2(capture_string, url, payload)
{
/* if I break in the debugger, and go 1 stackframe up,
I can see that gmock holds my string in varargs as second element
But I couldn't find a way to access it here*/
}
I also tried the ACTION_TEMPLATE but I am not c++ enough to understand what they are trying to explain me on gmock cookbook.
So my final question: is the above working trick with http_post_args struct really "the only way" to capture a const char * being passed as an argument to a mock?
If it SHOULD be possible using ACTION_P or ACTION_TEMPLATE would somebody be so kind to provide an actual working example with a const char *?
You could simply use a lambda, like so (live example):
TEST(SomeTest, Foo)
{
std::string payload;
web_api_mock m;
EXPECT_CALL(m, http_post(Eq("url"), _))
.WillOnce([&](const std::string &, const char* p){
payload = p;
return true;
});
m.http_post("url", "foo string");
EXPECT_THAT(payload, Eq("foo string"));
}
No additional http_post_args or actions etc required.
Of course, you could also change the payload to a const char* if you want to "capture" the raw char pointer. But be careful with the lifetime of the pointed to characters, in this case.
You hinted that your real code will need to parse the payload string as json and check for a certain element. It might lead to more readable tests when you create a dedicated matcher that does this. To show a rough draft (live example):
MATCHER_P(ContainsJsonElement, expectedElement, "")
{
const char * payload = arg;
// Parse payload as json, check for element, etc.
const bool foundElement = std::string(payload) == expectedElement;
return foundElement;
}
TEST(SomeTest, Foo)
{
web_api_mock m;
EXPECT_CALL(m, http_post(Eq("url"), ContainsJsonElement("foo string")));
m.http_post("url", "foo string");
}

Cereal - unable to deserialize a json string - unhandled exception

This is my json string
{
"connectionString" : "MyConnectionString",
"value" :"MyVal"
}
This is my class
struct Settings
{
std::string connectionString;
std::string value;
template<class Archive>
void serialize(Archive& ar)
{
ar(CEREAL_NVP(connectionString),
CEREAL_NVP(value)
);
}
};
And this is what I am doing:
std::ifstream ifs("Settings.json");
std::string content((std::istreambuf_iterator<char>(ifs)),(std::istreambuf_iterator<char>())); // Yes this is valid - The content gets populated
Settings settings;
{
cereal::JSONInputArchive archive_in(ifs);
archive_in(settings); //<<-----Exception here - Unhandled exception
}
The above solution would works only if my json string was this (i.e) if all the json string was an object of another key.
{
"SomeThing" :{
"connectionString" : "MyConnectionString",
"value" :"MyVal"
}
}
My question is how can I make my actual json string work (without wrapping it in an object)? I currently have this in a json file
{
"connectionString" : "MyConnectionString",
"value" :"MyVal"
}
and I wanted to know the best approach of deserializing this into an object ?
The code expecting that outer object is put there by the default JSON "prologue" and "epilogue" behavior:
https://github.com/USCiLab/cereal/blob/10d9a29c225fe9a843d043bfe9f13c5f958844dc/include/cereal/archives/json.hpp#L837
From the documentation:
These functions are given a reference to the active archive and a constant reference to the type being serialized. Careful specialization of prologue and epilogue allows for archives to exercise special behaviors without having to re-implement serialization functions for types. This lets us make the type support fairly generic. For example, cereal::XMLOutputArchive (<cereal/archives/xml.hpp>) makes use of these functions to start and finish nodes in its in-memory tree.
If you add in your code an overload for your type that does nothing:
void epilogue(cereal::JSONInputArchive&, const Settings&) { }
void prologue(cereal::JSONInputArchive&, const Settings&) { }
It won't try to parse an outer object

Cannot use std::string variable in RapidJSON function call

I am all new to C++ and am running into an issue. I am using rapidJSON to create JSON documents.
void setKeyValue() {
Value obj(kObjectType);
Value key("key");
Value val(42);;
obj.AddMember(key, val, d.GetAllocator());
}
Works as expected. But when I try to replace the call to key to make it use a passed in param, like so:
void setKeyValue(string myKey) {
Value obj(kObjectType);
Value key(myKey);
Value val(42);;
obj.AddMember(key, val, d.GetAllocator());
}
The myKey in Value key(myKey) get a red curly underling in Visual Studio saying the following:
What is causing this and how can I solve it?
You don't get support for std::string by default. rapidJSON requires you to specify you want std::string support.
#define RAPIDJSON_HAS_STDSTRING 1
Only then is this constructor you're using valid:
GenericValue (const std::basic_string< Ch > &s, Allocator &allocator)
JSON library you are using seems that doesn't work with string objects from standard library, but it works with const char*.
So you must convert string object to char* with the method c_str():
void setKeyValue(string myKey) {
Value obj(kObjectType);
Value key((char*)myKey.c_str());
Value val(42);;
obj.AddMember(key, val, d.GetAllocator());
}

C++ nlohmann/json how to use runtime provided json_pointers to read json values

I am using the json parser Json for Modern C++ (https://github.com/nlohmann/json). I know that I can get the value of a JSON value with a JSON_Pointer:
auto v1 = j["/a/b/c"_json_pointer];
But how would I go about getting the value if the JSON Pointer is defined at runtime (passed into my function)?
std:string s1 = "/a/b/c";
auto v1 = j[s1]; // doesn't work
You can't append "json_pointer" to either the std::string assignment or to the s1 variable. Is there a function that will convert a std::string to a json_pointer? The caller knows nothing about json and can't have access to the "json.hpp" header. I've also tried
std::string s1 = "/a/b/c";
json_pointer p1(s1);
but "json_pointer" class is undefined. Other than this issue this is a great library that does everything else I need. TIA.
Look at the source code:
inline nlohmann::json::json_pointer operator "" _json_pointer(const char* s, std::size_t)
{
return nlohmann::json::json_pointer(s);
}
If json_pointer is undefined, then you aren't using the right namespaces. Try
using nlohmann::json::json_pointer;
std::string s1 = "/a/b/c";
json_pointer p1(s1);

Accept null values for strings with rapidjson inside cereal, make them ""

For deserializing JSON into a c++ class, I'm using Cereal, which uses RapidJSON. As expected, c++ std::string can't have a null value. But other platforms do support null for strings (.NET SQL etc) and I get JSON from them with null values for strings. I need to tolerate this, and just make an empty string for nulls. What's the best way to do that?
I default to string substitute on the JSON changing nulls to "" like the following, but it is not a clean solution.
#include <cereal/archives/json.hpp>
#include <boost/algorithm/string.hpp>
// i.e. substitue all ":null with ":"" Like {"key":null} into {"key":""}
boost::replace_all(json, "\":null", "\":\"\"");
auto r = std::make_shared<R>();
std::stringstream ss(json);
{
cereal::JSONInputArchive archive(ss);
r->serialize(archive);
}
In case someone looks for this answer based on the exception generated by Cereal, it is: "rapidjson internal assertion failure: IsString()"
in cereal-1.1.2\include\cereal\external\rapidjson\document.h
change this
const Ch* GetString() const { RAPIDJSON_ASSERT(IsString()); return data_.s.str; }
to this
const Ch* GetString() const { if (IsNull_()) return ""; RAPIDJSON_ASSERT(IsString()); return data_.s.str; }
I'm not proposing this should be changed in the cereal source because some people may want strict type checking like the original