Use path as key in FileStorage - c++

I am trying to write a file (json/yaml/xml, the format does not matter) that stores a filepath with a specific cv::Mat. As I ended up realizing here, the key combination does not allow for a path (/home/user, for example).
void
persist_histograms(const QString& filepath, const QHash<QString, Mat>& histograms)
{
cv::FileStorage storage(filepath.toStdString(), cv::FileStorage::WRITE);
QHashIterator<QString, Mat> it(histograms);
while (it.hasNext()) {
it.next();
storage << it.key().toStdString() << it.value();
}
}
Is this a real limitation of the cv::FileStorageclass? Or is there a way of going around it?
The end result should be something like:
{
"/my/path/to/file.png": "<my cv::Mat serialization here>",
"/my/path/to/another/file.png": "<my cv::Mat serialization here>",
...
}
Observation
The error occurs independently of the format. If I pass a filepath that
ends with either yaml/json/xml the errors are the same:
This is the error if a try adding a letter at the beginning of the key:
This is the error I get when trying the code above:
OpenCV Error: Unspecified error (Incorrect element name /home/user/dtd/images/freckled/freckled_0055.jpg) in operator<<, file /build/opencv/src/opencv-3.2.0/modules/core/src/persistence.cpp, line 6877
terminate called after throwing an instance of 'cv::Exception'
what(): /build/opencv/src/opencv 3.2.0/modules/core/src/persistence.cpp:6877: error: (-2) Incorrect element name /home/user/dtd/images/freckled/freckled_0055.jpg in function operator<<
This is the error I get if I try to add a letter to the beginning of the key.
OpenCV Error: Bad argument (Key names may only contain alphanumeric characters [a-zA-Z0-9], '-', '_' and ' ') in icvJSONWrite, file /build/opencv/src/opencv-3.2.0/modules/core/src/persistence.cpp, line 3896

It seems that I have encountered a similar error. If I try to store a matrix in YAML format like this
std::string fName("/path/to/cameraParams.yml");
cv::FileStorage fs(fName, cv::FileStorage::WRITE);
fs << "camera matrix" << cameraMatrix;
everything works fine. But as soon as I change the file format from YAML to XML
std::string fName("/path/to/cameraParams.xml");
I get an error
terminate called after throwing an instance of 'cv::Exception'
what(): OpenCV(4.0.0-pre) /home/shura/software.downloads/libraries/opencv/opencv/modules/core/src/persistence_xml.cpp:83: error: (-5:Bad argument) Key name may only contain alphanumeric characters [a-zA-Z0-9], '-' and '_' in function 'writeTag'
As I have realized, the reason is that XML format does not allow spaces in a tag, while YAML does. So for YAML the file below is completely valid
%YAML:1.0
---
camera matrix: !!opencv-matrix
while for XML something like this is not allowed
<?xml version="1.0"?>
<opencv_storage>
<camera matrix type_id="opencv-matrix">
As soon as I replace the space in "camera matrix" by underscore, everything works fine.
fs << "camera_matrix" << cameraMatrix;
As I have found out here XML format has some restrictions on symbols you can have in a tag name. In particular, slashes, which you have in names, are not allowed. Dots seem to be allowed in tags, but for some reason OpenCV refuses to work with tags, which contain dots.

Related

How to enforce double quotes on all template values in input transformer

I have a JSON input that I would like to transform to another JSON output. I defined my list of input JSONPaths, and trying to create a simple JSON output in the template like so:
{
"fax": \"<userFax>\"
}
This was one of the formats given in the example from AWS themselves:
{
"instance": \"<instance>\",
"state": [9, \"<state>\", true],
"Transformed": "Yes"
}
However, when I try to update the changes, I get the following error:
Invalid InputTemplate for target ... : [Source: (String)"{ "fax": \"null\" }"; line: 2, column: 13].
Basically, I'd like all incoming values in the input to be converted to strings as an output via the template. This is to prevent values like zip codes from being converted into an integer and having it's leading zero stripped away. But it's confusing that even following the simple example from AWS is failing.

CJson - Insert missing quotes around strings

I have a text file of format given below (this is just sample file, but actually file is big one with key, values). Some of the values for the key will have format in file (look for value of Port key in below file). This is input file for me.
I am converting this file into Json using CJson Library in C++. Since value corresponding to Port key is not a valid string or value, CJson gives error message saying "Insert missing quotes around strings". So I tried to convert Port value to string using below code. But I am unable to insert missing quotes around. Could you please help me how to "Insert missing quotes around strings".
Code I tried:
ifstream infile { "conf_file.conf" };
string file_contents{ istreambuf_iterator<char>(infile), istreambuf_iterator<char>() };
pParentJson = cJSON_Parse( (char*)file_contents.c_str() );
cJSON* pJsonObj = cJSON_GetObjectItem(pParentJson,"port");
char* pDataBuffer = cJSON_Print(pJsonObj);
cout<<pDataBuffer<<endl;
Text File
{
"port": <CbTcpPortVariable>
"IP" : "127.0.0.1"
"Message": "Hello"
}

How to determine the position of an error in the input document with Poco Json Parser?

I am using the Poco C++ Json Parser. If the input document contains a Json syntax error, the parser throws an exception of type Poco::JSON::JSONException. My problem is that this exception type does not hold any information on the location of the error in the input document.
However the users need some hint on where the problem is so they can fix it. Is there any way I can get it?
EDIT:
For example, if the JSON input looks like this:
{
"objects": [
{
"timestamp: "2019-09-13T01:30:00.000Z",
...
The displayText() of the JSONException is:
error: 4: expected ':' after property name
No mention of a line number oder column (in the input document) whatsoever.
I am using Poco 1.9.0.
I don't know much about POCO, but a glance at the code suggests that the exception has a "message" field that contains the line number in which the json error occurred at - along with other useful information about the parse error.
From ParserImpl.cpp:
case JSON_ERROR:
{
const char* pErr = json_get_error(_pJSON);
std::string err(pErr ? pErr : "JSON parser error.");
throw JSONException(err);
}
Where that error string originates from this macro in pdjson.c:
#define json_error(json, format, ...) \
if (!(json->flags & JSON_FLAG_ERROR)) { \
json->flags |= JSON_FLAG_ERROR; \
snprintf(json->errmsg, sizeof(json->errmsg), \
"error: %lu: " format, \
(unsigned long) json->lineno, <<=== THERE'S YOUR LINE NUMBER
__VA_ARGS__); \
}
So the exception has everything the parser is capable of suggesting back to the user. Catch the JSONExcepton and literally print exception.message() back to the user. But that might require a modification if you want to support non-English. In any case, it doesn't look like it would be difficult to include the json->lineno formally into the JSONException class with a few modifications to the code. But not all errors have a line number - some are "unexpected end of file".

How to delete all lines and words from a text file up until a certain point

The problem I have is that I have a text file which contains a whole load of rubbish and just at the bottom are a few lines which holds the information I need. I want to be able to delete all the rubbish and just keep the useful information.
Iam hoping i can attach the text file which can explain it better.
There is a specific post which is relevant to this post that I am doing, however I have followed it and it hasn’t solved the problem I have. I have tried arrays and many other things, it just hasn’t worked.
it has information like this in it
|83Ïú¯–ŸªÖ!
just at the bottom of the text file it has this
000---------
i want to keep everything after the 000 and the rest of it i want to remove.
Open the original file for reading.
std::ifstream fin(orig_filename, std::ios::binary);
Open a new temporary file for writing in the same directory
std::ofstream fout(temp_filename, std::ios::binary);
Search for the marker pattern in the original file:
std::string marker = "000---------";
auto it =
std::search(std::istreambuf_iterator<char>(fin),
std::istreambuf_iterator<char>(), marker.begin(), marker.end());
Copy the rest from fin to fout (if the pattern was found):
if(it == std::istreambuf_iterator<char>()) {
std::cout << "pattern " << marker << " not found\n";
} else {
// the iterator "it" is now at the last char in the marker pattern,
// step over it and copy the rest to fout
std::copy(++it, std::istreambuf_iterator<char>(),
std::ostreambuf_iterator<char>(fout));
}
Rename the temporary file to the same name as the original file:
std::rename(temp_filename, orig_filename);
Note: Opening the files in std::ios::binary mode is afaik only needed on Windows.

Groovy Regex matcher OR - either unique groups or duplicate group ids

I want to regex search a string representing a log file, locate multiple potential error message, and print the result.
Example Input (single string)
...
Unable to parse YAML file: [mapping values are not allowed in this context] at line 1234\r\n
worthless line\r\n
KitchenInventory.cs(285,118): error CS2345: 'ButteryBread' does not contain a definition for 'DropOnFloor'\r\n
another worthless line\r\n
...
Each error type has certain grouping categories that I'm attempting to capture, which may or may not be present in other error types.
My goal is to output error details after doing the regex search, filling in as many values as are present in each error regex:
Error: Unable to parse YAML file
Details: mapping values are not allowed in this context
Line: 1234
Error: CS2345
Details: 'ButteryBread' does not contain a definition for 'DropOnFloor'
Filename: KitchenInventory.cs
Line: 285,118
(Notice that filename is ommited for the YAML error, due to YAML errors not containing that piece of information.
I've found that I cannot do a logical OR to create a massive Regex string, because there are duplicate grouping names. I attempted to do each regex search individually, but I receive an exception when checking for the presence of a field:
def pattern = ~patternString
def matcher = pattern.matcher(lines)
while (matcher.find()) {
if (matcher.group('filename')) {
println "Filename: "+matcher.group('filename')
}
if (matcher.group('error')) {
println "Error: "+matcher.group('error')
}
if (matcher.group('message')) {
println "Message: "+matcher.group('message')
}
}
Caught: java.lang.IllegalArgumentException: No group with name <errorCode>
java.lang.IllegalArgumentException: No group with name <errorCode>
Is there a way for me to iterate through Groovy regex group names that COULD be present, without throwing exceptions? Alternatively, am I going about this the wrong way, and is there an easier way for me to achieve my goal?
I was looking for the same thing and was not able to find a "groovier" way to do it.
Only solution that I found was to verify if the pattern string contains the group name:
if (stringPattern.contains('message')) {
println "Message: "+matcher.group('message')
}