Handling of const char* on ESP32 - c++

I'm working on making some Spotify API calls on an ESP32. I'm fairly new to C++ and while I seem to got it working how I wanted it to, I would like to know if it is the right way/best practice or if I was just lucky. The whole thing with chars and pointers is still quite confusing for me, no matter how much I read into it.
I'm calling the Spotify API, get a json response and parse that with the ArduinoJson library. The library returns all keys and values as const char*
The library I use to display it on a screen takes const char* as well. I got it working before with converting it to String, returning the String with the getTitle() function and converting it back to display it on screen. After I read that Strings are inefficient and best to avoid, I try to cut out the converting steps.
void getTitle()
{
// I cut out the HTTP request and stuff
DynamicJsonDocument doc(1024);
DeserializationError error = deserializeJson(doc, http.getStream(), );
JsonObject item = doc["item"];
title = item["name"]; //This is a const char*
}
const char* title = nullptr;
void loop(void) {
getTitle();
u8g2.clearBuffer();
u8g2.setDrawColor(1);
u8g2.setFont(u8g2_font_6x12_tf);
u8g2.drawStr(1, 10, title);
u8g2.sendBuffer();
}
Is it okay to do it like that?

This is not fine.
When seeing something like this, you should immediately become suspicious.
This is because in getTitle, you are asking a local object (item) for a pointer-- but you use the pointer later, when the item object no longer exists.
That means your pointer might be meaningless once you need it-- it might no longer reference your data, but some arbitrary other bytes instead (or even lead to crashes).
This problem is independent of what exact library you use, and you can often find relevant, more specific information by searching your library documentation for "lifetime" or "object ownership".
FIX
Make sure that item (and also DynamicJsonDocument, because the documentation tells you so!) both still exist when you use the data, e.g. like this:
void setTitle(const char *title)
{
u8g2.clearBuffer();
u8g2.setDrawColor(1);
u8g2.setFont(u8g2_font_6x12_tf);
u8g2.drawStr(1, 10, title);
u8g2.sendBuffer();
}
void updateTitle()
{
DynamicJsonDocument doc(1024);
DeserializationError error = deserializeJson(doc, http.getStream(), );
JsonObject item = doc["item"];
setTitle(item["name"]);
}
See also: https://arduinojson.org/v6/how-to/reuse-a-json-document/#the-best-way-to-use-arduinojson
Edit: If you want to keep parsing/display update decoupled
You could keep the JSON document "alive" for when the parsed data is needed:
/* "static" visibility, so that other c/cpp files ("translation units") can't
* mess mess with our JSON doc directly
*/
static DynamicJsonDocument doc(1024);
static const char *title;
void parseJson()
{
[...]
// super important to avoid leaking memory!!
doc.clear();
DeserializationError error = deserializeJson(doc, http.getStream(), );
// TODO: robustness/error handling (e.g. inbound JSON is missing "item")
title = doc["item"]["name"];
}
// may be nullptr when called before valid JSON was parsed
const char* getTitle()
{
return title;
}

Related

gmock save argument string

I hope there is an easier way to do this... I need to capture the string which is passed as an argument to a mock.
The mock
class web_api_mock : public iweb_api
{
public:
MOCK_METHOD(
(bool),
http_post,
(const etl_normal_string &, const char*),
(override));
};
I want to capture the char * passed to the mock as second argument. I need to construct some json structure from it, and I want to check if a certain element has a certain value.
I had to spend a lot of time to get it to work, eventually copying the trick from here. This brilliant mind figured out you can rely on the gmock's Invoke.
The EXPECT_CALL
http_post_args args;
EXPECT_CALL(_web_api_mock, http_post(etl_string_equals(url), _))
.WillOnce(
DoAll(
Invoke(&args, &http_post_args::capture),
Return(true)));
Here I am invoking all arguments of the mock to a struct which I defined as follows
struct http_post_args
{
void capture(etl_normal_string url, const char * p)
{
payload = std::string(p);
}
std::string payload;
};
And finally, I get my hands on the char * and do whatever I want afterwards.
It seems awfully complicated to save an argument when it's of the type char *.
My first attempt was the obvious mistake I guess many before (and after) me made: using the SaveArgPointee which will copy only the first element of the string and gives me with a string where the first character is correct, but the remaining string is filled with random mem.
My second attempt was to define an ACTION_P. This "almost" worked. In the callstack I could see the string I am interested in until the very last stackframe, where the args simply seem not to be passed to the actual implementation of my custom ACTION_P.
ACTION_P2(capture_string, url, payload)
{
/* if I break in the debugger, and go 1 stackframe up,
I can see that gmock holds my string in varargs as second element
But I couldn't find a way to access it here*/
}
I also tried the ACTION_TEMPLATE but I am not c++ enough to understand what they are trying to explain me on gmock cookbook.
So my final question: is the above working trick with http_post_args struct really "the only way" to capture a const char * being passed as an argument to a mock?
If it SHOULD be possible using ACTION_P or ACTION_TEMPLATE would somebody be so kind to provide an actual working example with a const char *?
You could simply use a lambda, like so (live example):
TEST(SomeTest, Foo)
{
std::string payload;
web_api_mock m;
EXPECT_CALL(m, http_post(Eq("url"), _))
.WillOnce([&](const std::string &, const char* p){
payload = p;
return true;
});
m.http_post("url", "foo string");
EXPECT_THAT(payload, Eq("foo string"));
}
No additional http_post_args or actions etc required.
Of course, you could also change the payload to a const char* if you want to "capture" the raw char pointer. But be careful with the lifetime of the pointed to characters, in this case.
You hinted that your real code will need to parse the payload string as json and check for a certain element. It might lead to more readable tests when you create a dedicated matcher that does this. To show a rough draft (live example):
MATCHER_P(ContainsJsonElement, expectedElement, "")
{
const char * payload = arg;
// Parse payload as json, check for element, etc.
const bool foundElement = std::string(payload) == expectedElement;
return foundElement;
}
TEST(SomeTest, Foo)
{
web_api_mock m;
EXPECT_CALL(m, http_post(Eq("url"), ContainsJsonElement("foo string")));
m.http_post("url", "foo string");
}

Buffer gets overwritten

I'm facing this issue on an ESP8266 (Arduino like board), but this problem is regarding c/c++, so I'm asking this here.
I have not that much experience with native languages like c/c++ and I'm facing a strange issue, which drives me crazy. So I'm using an Wemos D1 mini (ESP8266) which uses a class calles ConfigManager to read a configuration file from eeprom. The config file is formatted as json, so I'm using ArduinoJson to parse the content. I have declared a StaticJsonBuffer and pointer to a JsonObject in the header, like you can see in the code example:
//FILE: ConfigManager.h
#ifndef ConfigManager_H
#define ConfigManager_H
#include <Arduino.h>
#include <ArduinoJson.h>
#include <FS.h>
#include "Logger.h"
class ConfigManager {
public:
Settings *settings;
ConfigManager();
void read_from_eeprom();
private:
File configFile;
JsonObject *json;
StaticJsonBuffer<200> jsonBuffer;
void open_file(const char *permission);
void read_json();
void recreate_file();
void create_json();
void check_success();
void populate_settings();
void clean_up();
};
#endif
When the function read_from_eeprom is invoked, it opens the file and invokes the functionread_json:
void ConfigManager::read_json() {
size_t size = configFile.size();
Log.verbose("[ConfigManager] Config file size: %d", size);
std::unique_ptr<char[]> buf(new char[size]);
configFile.readBytes(buf.get(), size);
Log.verbose("[ConfigManager] File content: %s", buf.get());
Log.verbose("[ConfigManager] Parsing json");
json = &jsonBuffer.parseObject(buf.get());
Log.notice("[ConfigManager] Json is:");
json->printTo(Serial);
Serial.println();
}
Which is followed by a call to check_success()
void ConfigManager::check_success() {
Log.notice("[ConfigManager] Json is:");
json->printTo(Serial);
Serial.println();
bool should_recreate = true;
if (json->success()) {
Log.notice("[ConfigManager] Parsed json successfully");
auto version = json->get<const char*>("version");
if (version) {
if (strcmp(version, Settings::current_version) == 0) {
Log.notice("[ConfigManager] Config version is up2date");
should_recreate = false;
} else {
Log.warning("[ConfigManager] Config version outdated");
}
} else {
Log.warning("[ConfigManager] Invalid config file");
}
} else {
Log.warning("[ConfigManager] Config file is not valid json");
}
if (should_recreate) {
Log.notice("[ConfigManager] Recreating config file");
recreate_file();
create_json();
}
Log.notice("JSON IS: ");
json->prettyPrintTo(Serial);
Log.notice("[ConfigManager] Sucessfully read json");
}
So what I noticed is that the file content is fine. E.g. {"version":"0.2","led_count":"64"}.
Then the json is parsed, which succeeds and logs the json object, which is again {"version":"0.2","led_count":"64"}.
Afterwards the function returns, and calls check_success, which again prints the content of the json object to the log, but this time it seems that something has overwritten the JsonBuffer, which causes the json object to be corrupted. This time the logged content is {"v␂":"0.2","led_count":"64"} (with some strange unicorn characters that change as the source code changes). I'm trying to figure out whats going on for many hours now, but I'm stuck. Can someone please point me in the right direction to solve this problem? Thank you!
The full Log can be found HERE, as well as ConfigManager.h and ConfigManager.cpp
*I'd prefer write that in comments, because I don't have arduino and can't verify that my advice 100% helpful. But I can't use comments with "my reputation" :). So please don't press "minus button" if my answer didn't help... *
According to that it seems you need to keep original json string while you using json buffer.
Keep the JSON string in memory long enough
The library never make memory duplication. This has an important implication on string
values, it means that the library will return pointer to chunks of the
string.
For instance, let’s imagine that you parse ["hello","world"], like
this:
char[] json = "[\"hello\",\"world\"]";
StaticJsonBuffer<32> buffer;
JsonArray& array = buffer.parseArray(json);
const char* first = array[0];
const char* second = array[1];
In that
case, both first and second are pointers to the content of the
original string json. So this will only work if json is still in
memory.
So, it make sense try make std::unique_ptr buf a class member (same as StaticJsonBuffer) and check how it works.
BTW, IMO std::vector will be more suitable there... And I'm not sure that unique_ptr deletes arrays properly.

Qt - passing reference of QJsonObject or QJsonArray

I'm making Json format data editor with Qt treeview and Qt Json support.
I wanna pass QJsonObject or QJsonArray reference parameter to function.
This works:
void makeJsonData(QJsonObject &obj) {
obj.insert("key", 1234);
}
//call makeJsonData()
QJsonObject jobj;
makeJsonData(jobj);
int keysize = jobj.keys().size(); //1, OK.
But not with this:
//QJsonValue, because it can handle both QJsonObject and QJsonArray
void makeJsonData(QJsonValue &obj) {
obj.toObject().insert("key", 1234); //obj is QJsonObject
}
//call makeJsonData()
QJsonObject jobj;
makeJsonData(QJsonValue::fromVariant(jobj)); //fromVariant() to cast QJsonObject to QJsonValue
int keysize = jobj.keys().size(); //0, Fail.
It looks like QJsonValue::toObject() just copies parameter..
How can I use reference of both QJsonObject and QJsonArray with one parameter type?
There are a couple ways I see to solve your problem:
Option 1 (as mentioned in my comment)
A dynamic cast can be used like so:
bool makeJsonData(void* obj) {
QJsonObject* asObj = dynamic_cast<QJsonObject*>(obj);
QJsonArray* asArray = dynamic_cast<QJsonArray*>(obj);
if (asObj) {
//do what you would if it were an object
}
else if (asArray) {
//do what you would if it were an array
}
else {
//cast fail. Returning false to tell the caller that they passed bad data
//an alternate (probably better) would be to throw an exception
return false;
}
}
Option 2
I honestly feel that this business with void* is the wrong way to do it. Doing void* stuff is almost always a code smell (it removes compile-time checks that save us from stepping on their own feet) and in this case I think that the way you are doing this needs work. Also, dynamic_cast requires RTTI which may not always be turned on (compiler support, performance issues, etc).
I took a look at the Qt headers on my machine and as far as I can tell, QJsonObject and QJsonArray don't really inherit from anything, so going down the route of changing the void* to a base type in order to keep a semblance of type checking won't quite work.
What I would do would be this:
Make two separate methods. One for handling arrays and one for handling objects. They have different methods and different things you can do, so this makes sense to me. You could even keep the same name so that they are overloaded.
Have another method with your common stuff in it. I assume that your function is trying to add some data to either the array or object that is passed. Make a method that creates the data (i.e. QJsonObject createJsonData()) and call it inside both of your methods mentioned above.
The idea is to keep code repetition down while still preserving type checking. The time you spend making the one extra method to handle both cases could be far less than the time you will spend debugging code after accidentally passing in something to a void* pointer that you never meant to pass.
Option 3
Alternately, you could use QJsonValue, change the return type of the function to QJsonValue, and make it return the new object without modifying the original. Further, the QJsonValue class has those fun isArray/isObject methods that you could use to do something like mentioned earlier. An example:
QJsonValue makeJsonData(const QJsonValue& val) {
if (val.isObject()) {
QJsonObject obj = val.toObject();
//do your stuff, modifying obj as you please (perhaps calling another method so that this can have less repetition
return QJsonValue(obj);
}
else if (val.isArray()) {
QJsonArray arr = val.toArray();
//do your stuff, modifying arr as you please (perhaps calling another method so that this can have less repetition
return QJsonValue(arr);
}
else {
throw "Invalid Value Type";
}
}
I honestly prefer this pattern, but I know there are reasons for going the way you have mentioned such as avoiding gratuitous memory allocations.
You may need to add this:
#include <QJsonArray>

How can I get more details about errors generated during protobuf parsing? (C++)

I am new to protobuf (C++) and my code fails during parse of my messages. How can I get more details about the errors that occurred?
Example
The following snippet illustrates the problem:
const bool ok=my_message.ParseFromCodedStream(&stream);
if(ok){
std::cout<< "message parsed. evidence:\n"<< my_message.DebugString();
}
else{
std::cerr<< "error parsing protobuf\n";
//HOW CAN I GET A REASON FOR THE FAILURE HERE?
}
If you look inside protobuf code, you will find it's using its own logging system - based on macros. By default all these messages goes to stderr, but you can capture them in your program with SetLogHandler():
typedef void LogHandler(LogLevel level, const char* filename, int line,
const std::string& message);
The possible solution is to make your own errno-like mechanism (sorry for C++11-ishness):
typedef LogMessage std::tuple<LogLevel, std::string, int, std::string>; // C++11
typedef LogStack std::list<LogMessage>;
namespace {
LogStack stack;
bool my_errno;
} // namespace
void MyLogHandler(LogLevel level, const char* filename, int line,
const std::string& message) {
stack.push_back({level, filename, line, message}); // C++11.
my_errno = true;
}
protobuf::SetLogHandler(MyLogHandler);
bool GetError(LogStack* my_stack) {
if (my_errno && my_stack) {
// Dump collected logs.
my_stack->assign(stack.begin(), stack.end());
}
stack.clear();
bool old_errno = my_errno;
my_errno = false;
return old_errno;
}
And use it in your code:
...
else {
std::cerr<< "error parsing protobuf" << std::endl;
LogStack my_stack;
if (GetError(&my_stack) {
// Handle your errors here.
}
}
The main drawback of my sample code - it doesn't work well with multiple threads. But that can be fixed on your own.
Sometimes error information will be printed to the console, but that's it. There's no way to get extra error info through the API.
That said, there are only two kinds of errors anyway:
A required field was missing. (Information should be printed to the console in this case.)
The data is corrupt. It was not generated by a valid protobuf implementation at all -- it's not even a different type of protobuf, it's simply not a protobuf.
If you are seeing the latter case, you need to compare your data on the sending and receiving side and figure out why it's different. Remember that the data you feed to the protobuf parser not only must be the same bytes, but it must end at the same place -- the protobuf parser does not know where the message ends except by receiving EOF. This means that if you are writing multiple messages to a stream, you need to write the size before the data, and make sure to read only that many bytes on the receiving end before passing on to the protobuf parser.

Is it possible to *safely* return a TCHAR* from a function?

I've created a function that will convert all the event notification codes to strings. Pretty simple stuff really.
I've got a bunch of consts like
const _bstr_t DIRECTSHOW_MSG_EC_ACTIVATE("A video window is being activated or deactivated.");
const _bstr_t DIRECTSHOW_MSG_EC_BUFFERING_DATA("The graph is buffering data, or has stopped buffering data.");
const _bstr_t DIRECTSHOW_MSG_EC_BUILT("Send by the Video Control when a graph has been built. Not forwarded to applications.");
.... etc....
and my function
TCHAR* GetDirectShowMessageDisplayText( int messageNumber )
{
switch( messageNumber )
{
case EC_ACTIVATE: return DIRECTSHOW_MSG_EC_ACTIVATE;
case EC_BUFFERING_DATA: return DIRECTSHOW_MSG_EC_BUFFERING_DATA;
case EC_BUILT: return DIRECTSHOW_MSG_EC_BUILT;
... etc ...
No big deal. Took me 5 minutes to throw together.
... but I simply don't trust that I've got all the possible values, so I want to have a default to return something like "Unexpected notification code (7410)" if no matches are found.
Unfortunately, I can't think of anyway to return a valid pointer, without forcing the caller to delete the string's memory ... which is not only nasty, but also conflicts with the simplicity of the other return values.
So I can't think of any way to do this without changing the return value to a parameter where the user passes in a buffer and a string length. Which would make my function look like
BOOL GetDirectShowMessageDisplayText( int messageNumber, TCHAR* outBuffer, int bufferLength )
{
... etc ...
I really don't want to do that. There must be a better way.
Is there?
I'm coming back to C++ after a 10 year hiatus, so if it's something obvious, don't discount that I've overlooked it for a reason.
C++? std::string. It's not going to destroy the performance on any modern computer.
However if you have some need to over-optimize this, you have three options:
Go with the buffer your example has.
Have the users delete the string afterwards. Many APIs like this provide their own delete function for deleting each kind of dynamically allocated return data.
Return a pointer to a static buffer which you fill in with the return string on each call. This does have some drawbacks, though, in that it's not thread safe, and it can be confusing because the returned pointer's value will change the next time someone calls the function. If non-thread-safety is acceptable and you document the limitations, it should be all right though.
If you are returning a point to a string constant, the caller will not have to delete the string - they'll only have to if you are new-ing the memory used by the string every time. If you're just returning a pointer to a string entry in a table of error messages, I would change the return type to TCHAR const * const and you should be OK.
Of course this will not prevent users of your code to attempt to delete the memory referenced by the pointer but there is only so much you can do to prevent abuse.
Just declare use a static string as a default result:
TCHAR* GetDirectShowMessageDisplayText( int messageNumber )
{
switch( messageNumber )
{
// ...
default:
static TCHAR[] default_value = "This is a default result...";
return default_value;
}
}
You may also declare "default_value" outside of the function.
UPDATE:
If you want to insert a message number in that string then it won't be thread-safe (if you are using multiple threads). However, the solution for that problem is to use thread-specific string. Here is an example using Boost.Thread:
#include <cstdio>
#include <boost/thread/tss.hpp>
#define TCHAR char // This is just because I don't have TCHAR...
static void errorMessageCleanup (TCHAR *msg)
{
delete []msg;
}
static boost::thread_specific_ptr<TCHAR> errorMsg (errorMessageCleanup);
static TCHAR *
formatErrorMessage (int number)
{
static const size_t MSG_MAX_SIZE = 256;
if (errorMsg.get () == NULL)
errorMsg.reset (new TCHAR [MSG_MAX_SIZE]);
snprintf (errorMsg.get (), MSG_MAX_SIZE, "Unexpected notification code (%d)", number);
return errorMsg.get ();
}
int
main ()
{
printf ("Message: %s\n", formatErrorMessage (1));
}
The only limitation of this solution is that returned string cannot be passed by the client to the other thread.
Perhaps have a static string buffer you return a pointer to:
std::ostringstream ss;
ss << "Unexpected notification code (" << messageNumber << ")";
static string temp = ss.str(); // static string always has a buffer
return temp.c_str(); // return pointer to buffer
This is not thread safe, and if you persistently hold the returned pointer and call it twice with different messageNumbers, they all point to the same buffer in temp - so both pointers now point to the same message. The solution? Return a std::string from the function - that's modern C++ style, try to avoid C style pointers and buffers. (It looks like you might want to invent a tstring which would be std::string in ANSI and std::wstring in unicode, although I'd recommend just going unicode-only... do you really have any reason to support non-unicode builds?)
You return some sort of self-releasing smart pointer or your own custom string class. You should follow the interface as it's defined in std::string for easiest use.
class bstr_string {
_bstr_t contents;
public:
bool operator==(const bstr_string& eq);
...
~bstr_string() {
// free _bstr_t
}
};
In C++, you never deal with raw pointers unless you have an important reason, you always use self-managing classes. Usually, Microsoft use raw pointers because they want their interfaces to be C-compatible, but if you don't care, then don't use raw pointers.
The simple solution does seem to be to just return a std::string. It does imply one dynamic memory allocation, but you'd probably get that in any case (as either the user or your function would have to make the allocation explicitly)
An alternative might be to allow the user to pass in an output iterator which you write the string into. Then the user is given complete control over how and when to allocate and store the string.
On the first go-round I missed that this was a C++ question rather than a plain C question. Having C++ to hand opens up another possibility: a self-managing pointer class that can be told whether or not to delete.
class MsgText : public boost::noncopyable
{
const char* msg;
bool shouldDelete;
public:
MsgText(const char *msg, bool shouldDelete = false)
: msg(msg), shouldDelete(shouldDelete)
{}
~MsgText()
{
if (shouldDelete)
free(msg);
}
operator const char*() const
{
return msg;
}
};
const MsgText GetDirectShowMessageDisplayText(int messageNumber)
{
switch(messageNumber)
{
case EC_ACTIVATE:
return MsgText("A video window is being activated or deactivated.");
// etc
default: {
char *msg = asprintf("Undocumented message (%u)", messageNumber);
return MsgText(msg, true);
}
}
}
(I don't remember if Windows CRT has asprintf, but it's easy enough to rewrite the above on top of std::string if it doesn't.)
Note the use of boost::noncopyable, though - if you copy this kind of object you risk double frees. Unfortunately, that may cause problems with returning it from your message-pretty-printer function. I'm not sure what the right way to deal with that is, I'm not actually much of a C++ guru.
You already use _bstr_t, so if you can just return those directly:
_bstr_t GetDirectShowMessageDisplayText(int messageNumber);
If you need to build a different message at runtime you can pack it into a _bstr_t too. Now the ownership is clear and the use is still simple thanks to RAII.
The overhead is negligible (_bstr_t uses ref-counting) and the calling code can still use _bstr_ts conversion to wchar_t* and char* if needed.
There's no good answer here, but this kludge might suffice.
const char *GetDirectShowMessageDisplayText(int messageNumber)
{
switch(messageNumber)
{
// ...
default: {
static char defaultMessage[] = "Unexpected notification code #4294967296";
char *pos = defaultMessage + sizeof "Unexpected notification code #" - 1;
snprintf(pos, sizeof "4294967296" - 1, "%u", messageNumber);
return defaultMessage;
}
}
}
If you do this, callers must be aware that the string they get back from GetDirectShowMessageText might be clobbered by a subsequent call to the function. And it's not thread safe, obviously. But those might be acceptable limitations for your application.