I am having trouble saving and reading a cv::Mat saved into a cv::FileNode through which I am iterating. I can write the matrix to a file using
// Data to save
std::vector<cv::Mat> masks;
masks.push_back(cv::Mat::ones(3,3,CV_8UC1));
masks.push_back(cv::Mat::zeros(2,2,CV_8UC1));
// Open file for writing
cv::FileStorage fs(file, cv::FileStorage::WRITE);
// Create node
fs << "data" << "[";
for (size_t i=0; i<masks.size(); ++i)
{
// Fill each node with data
fs << "{:";
// Write other data...
fs << "mask" << masks.at(i);
fs << "}";
}
fs << "]";
Writing a file in this way causes no problems. However, subsequently opening this file causes an unhandled exception which prints:
Reading OpenCV Error: Parsing error (testPR.yml(7): Missing , between
the elemen ts) in unknown function, file
......\modules\core\src\persistence.cpp, line 12 63
To read the matrix I attempt to do this:
// Open file for reading
cv::FileStorage fs(file, cv::FileStorage::READ); ///< This causes an exception
// Open node
cv::FileNode d = fs["data"];
cv::FileNodeIterator it = d.begin(), it_end = d.end();
for( ; it!= it_end; ++it)
{
// Read other data...
cv::Mat mask;
(*it)["mask"] >> mask;
}
Surely it's possible to save/read a matrix to a node in this manner, isn't it?
It turns out that the solution to this was to save the data in an .xml rather than a .yml file, although I am none the wiser as to why that might be.
When I look at http://docs.opencv.org/modules/core/doc/xml_yaml_persistence.html
In YAML (but not XML), mappings and sequences can be written in a compact Python-like inline form. In the sample above matrix elements, as well as each feature, including its lbp value, is stored in such inline form. To store a mapping/sequence in a compact form, put ”:” after the opening character, e.g. use “{:” instead of “{“ and “[:” instead of “[“. When the data is written to XML, those extra ”:” are ignored.
Maybe {: can't be used without [:, but I couldn't find another documentation on this features. But regarding to your solution to store it in XML: The extra ":" is ignored...
From my opinion, storing to YAML should also work when deleting the ":".
Related
I am trying to add elements into a .json file between [] as last.
How can I move the cursor to add elements between [...] with efficiently with std::ofstream?
I have tried several open modes but there are strange things. First I created this question about not able to use the file streaming for read and write because of the overwrite issue.
#include <iostream>
#include <fstream>
int main ()
{
char errmsg[2048];
std::ofstream ostream;
ostream.exceptions(std::ios_base::badbit);
try
{
ostream.open("LS22731.json", std::fstream::ate | std::fstream::in);
strerror_s(errmsg, 2048, errno);
std::cout << "Error (" << errno << "): " << errmsg << std::endl;
if (ostream && ostream.is_open())
{
auto ppos = ostream.tellp();
std::streampos sub = 1; //
std::cout << "Tellp: " << ppos << std::endl; // Always show zero but file has large data
if (ppos > 1)
ostream.seekp(ppos - sub) << "aa";
ppos = ostream.teelp();
std::cout << "New tellp: " << ppos << std::endl;
ostream.close();
}
}
catch (std::ios_base::failure& fb)
{
std::cout << "Failure: " << fb.what() << std::endl;
char errmsg[2048];
strerror_s(errmsg, 2048, errno);
std::cout << "Error (" << errno << "): " << errno << std::endl;
}
}
I searched about open modes then I found this but is it good to open file with both mode std::fstream::ate | std::fstream::in together for std::ofstream? And when I open the file with std::fstream::out mode it is rewriting so deleting whole document,
std::fstream::out: Delete all contents of the file (overwrite)
std::fstream::app: Cannot move the cursor with seekp
std::fstream::ate: Delete all contents of the file (overwrite)
std::fstream::binary: Delete all contents of the file (overwrite)
std::fstream::ate | std::fstream::app: Cannot move the cursor with seekp
std::fstream::ate | std::fstream::out: Delete all contents of the file (overwrite)
std::fstream::ate | std::fstream::in: Can move the cursor but not insert delete all after.
I don't want to use c FILE.
Well JSON files are err... sequential text files. That means that the file contains a stream of bytes representing the JSON content. And AFAIK, no filesystem has provision for inserting data in the middle of a sequential file. The foolproof way is:
copy up to the insertion point to a temp file
write the new data
add the remaining data from the original file
rename the old file to a backup name
rename the temp file with the original name
(optionaly) remove the backup file
The brave way is to move the second part up by chunks starting from the end to create an emply place to put the data write the new data in that place, and pray all along the operation for no problem in the middle because the file would be irremediably corrupted.
Those 2 ways can process files of arbitrary sizes. For small files, you could load everything in memory, write the new data at the insertion point and rewrite the remaining data after the new data. You just need to use a default fstream and use neither ate nor trunc. out does not mean deleting all the file content. You simply replace the original bytes at the place where you write.
So you should use:
ostream.open("LS22731.json", std::fstream::out | std::fstream::in);
Then you:
read up to your insertion point and discard the data
note the position with tellp
read the end of file and save it
go to the insertion point
write the new data
write the saved data
close the stream
Here is an adaptation of the previous algorithm. The cautious points as:
you must use a fstream with std::fstream::out | std::fstream::in mode to be able to read and write a file. The file must exist and you will be initially positioned at the beginning of the file
to reliably be able to compute positions, you must open the file in binary mode (std::fstream::binary)(should be possible in text mode but I could not find my way...)
Here is a close adaptation of your code: it opens the file, search for the first closing bracket (]), and inserts ,"h" before to simulate adding a value into a list.
...
std::fstream ostream;
ostream.exceptions(std::ios_base::badbit);
try
{
// use binary mode to ba able to relyably seek the file.
ostream.open("LS22731.json",
std::fstream::out | std::fstream::in | std::fstream::binary);
strerror_s(errmsg, 2048, errno);
std::cout << "Error (" << errno << "): " << errmsg << std::endl;
if (ostream && ostream.is_open())
{
std::streampos ppos;
// search the first ]
ostream.ignore(std::numeric_limits<std::streamsize>::max(), ']');
// we want to insert just before it
ppos = ostream.tellg() - std::streampos(1);
ostream.seekg(ppos); // prepare to read from the ]
std::string old = "", tmp;
// save end of file, starting at the ]
while (std::getline(ostream, tmp)) {
old += tmp + "\n";
}
ostream.clear(); // clear eof indicator
ostream.seekp(ppos, std::ios::beg); // go back to the insertion point
ostream << ",\"h\""; // add some data
ostream << old; // add the remaining of the original data
ostream.close();
}
...
Disclaimers:
DO NOT PRETEND I ADSISED YOU THIS WAY. If there is a problem in the middle of processing, the file will be irremediately corrupted.
it will fail miserabily if a text field contains a closing bracket, because it is not a JSON parser
If you open a file for reading, you cant set the write head of it.
You are using std::ofstream with ios::in mode which I'm not sure is effective. but std::ofstream must be opened with ios::out or ios::app. When you override the default you should give also the default.
If you need to open a file for both read and write, you should use std::fstream.
Another issue is that you trying to add some string in the middle of a text file, and it is not so good idea, it is not similar to paste some string in a text file when opened in Notepad. you must replace a section with another section with the same length, pushing some string won't move the rest of the data forward.
I think the easy way is to read the whole JSON to memory, process it by add or remove some data, and finally rewrite the whole JSON to the file.
I am trying to build an application, which get data from webcam or external device, saves Video Frames into text file, then read frames from created text file.
I don't know whether it is a good idea to save to text file, I'm open suggestions.
So far I've done to saving to a text file.
My problem is reading from text file. Basically I read text line by line, but I don't know how to convert this text into Mat object.
So far my code is:
ifstream read_storage(new_vid_frm_path);
if(!read_storage.is_open()) {
perror("\n\n\n\t\t\t(-)FAIL : Can't Open SavedVideoFrames.txt\n\n\n\t\t\t");
return -1;
}
VideoWriter *vid = new VideoWriter(new_vid_frm_path,CV_FOURCC('P', 'I', 'M', '1'),30,Size(vc.get(CV_CAP_PROP_FRAME_WIDTH),vc.get(CV_CAP_PROP_FRAME_HEIGHT)));
Mat line;
vector<Mat> vid_frms;
while ( getline (read_storage,line) ) {
cout << line << '\n';
}
read_storage.close();
if(vid_frms.size() == 0){
printf("\n\n\n\t\t\t(-)FAIL: Error In Frame\n\n\n\t\t\t");
return -1;
}
for(size_t i = 0; i<vid_frms.size(); i++)
(*vid).write(vid_frms[i]);
printf("\n\n\n\t\t\t(+)SUCCESS: Video Processing Complete \n\n\n\t\t\t ");
Do you have ay suggestions how can I cast or convert this line string to Mat obejct?
while ( getline (read_storage,line) ) {
cout << line << '\n';
}
Thanks.
By the way, I looked at this solution, but I couldn't understand.
Convert a string of bytes to cv::mat
I couldn't find the byte type in c++ and I think there might be a direct conversion between String to Mat object.
You can save anything (just about) in OpenCV to a .xml or .yml text file and then read it back in using the OpenCV XML/YAML FileStorage methods.
I highly recomend this over using native C++ methods for file stuff.
It's specifically designed to handle all this legwork for you.
I know how to load/save a cv::Mat instance into a XML-file (See this question).
But what I really need, is to parse a std::string (or char *) that contains the XML, and get the cv::Mat. Say I get the XML out of a database, and not from a file.
Is that possible?
You can do it since OpenCV 2.4.1.
Here is a code sample from release notes:
//==== storing data ====
FileStorage fs(".xml", FileStorage::WRITE + FileStorage::MEMORY);
fs << "date" << date_string << "mymatrix" << mymatrix;
string buf = fs.releaseAndGetString();
//==== reading it back ====
FileStorage fs(buf, FileStorage::READ + FileStorage::MEMORY);
fs["date"] >> date_string;
fs["mymatrix"] >> mymatrix;
I am maintaining a C++ method which one of my clients is hitting an issue with. The method is supposed to write out a series of identifiers to a file delimited by a new line. However on their machine somehow the method is writing a series of NULL's out to the file. Opening the file in a binary editor shows that it contains all zeros.
I can't understand why this is happening. I've tried assigning empty strings and strings with the first character set to 0. There is no problem creating the file, just writing the identifiers to it.
Here is the method:
void writeIdentifiers(std::vector<std::string> IDs, std::string filename)
{
std::ofstream out (filename.c_str(), std::ofstream::binary);
if (out.is_open())
{
for (std::vector<std::string>::iterator it = IDs.begin();
it != IDs.end();
it++)
{
out << *it << "\n";
}
}
out.close();
}
My questions: is there any possible input you can provide that method which will create a file which has NULL values in it?
Yeah, the following code quite clearly writes a series of NULL bytes:
std::vector<std::string> ids;
std::string nullstring;
nullstring.assign("\0\0\0\0\0\0\0\0\0\0", 10);
ids.push_back(nullstring);
writeIdentifiers(ids, "test.dat");
Because the std::string container stores the string length, it can't necessarily be used in the same way as an ordinary C (null-terminated) string. Here, I assign a string containing 10 NULL bytes. Those are then output because the string length is 10.
I am not a programmer, but am an engineer who needs to use C++ coding on this occasion, so sorry if this question is a little basic.
I need to use a look up table as I have some highly non-linear dynamics going on that I need to model. It consists of literally 1000 paired values, from a pair of (0.022815, 0.7) up to (6.9453, 21.85).
I don't want to have to type all these values out in my C code. The values are currently stored in Matlab. Can I read them from a .dat file or something similar?
I will have calculated a value and simply want the program to kick out the paired value.
Thanks,
Adam
You can't read something stored in Matlab directly, unless you want to
write a parser for whatever format Matlab stores its data in. I'm not
familiar with Matlab, but I would be very surprised if it didn't have a
function to output this data to a file, in some text format, which you
could read and parse.
Assuming this is constant data, if it could output something along the
lines of:
{ 0.022815, 0.7 },
...
{ 6.9453, 21.85 },
you could include it as the initializer of a table in C++. (It may look
strange to have a #include in the middle of a variable definition, but
it's perfectly legal, and in such cases, perfectly justified.) Or just
copy/paste it into your C++ program.
If you can't get exactly this format directly, it should be trivial to
write a small script that would convert whatever format you get into
this format.
this program defines a map, then reading from a.txt file, inserting to a map, iterating on map for any purposes you have, and finally writing the map into a file.
just a simple practice:
#include <fstream>
#include <iostream>
#include <map>
using namespace std;
int main(){
ifstream inFile("a.txt", ios::in);
if (! inFile ){
cout<<"unabl to open";
return 0;
}
//reading a file and inserting in a map
map<double,double> mymap;
double a,b;
while( ! inFile.eof() ){
inFile>>a>>b;
mymap.insert ( a,b );
}
inFile.close(); //be sure to close the file
//iterating on map
map<double,double>::iterator it;
for ( it=mymap.begin() ; it != mymap.end(); it++ ){
// (*it).first
// (*it).second
}
//writing the map into a file
ofstream outFile;
outFile.open ("a.txt", ios::out); // or ios::app if you want to append
for ( it=mymap.begin() ; it != mymap.end(); it++ ){
outFile << (*it).first << " - " << (*it).second << endl; //what ever!
}
outFile.close();
return 0;
}
What I would do for this is as follows as I think this is faster than file open and close. First of all create a header file which contains all the data in an array. You could you a "replace all" available in Notepad or so to replace the () braces to { } braces. Later on you could even write a script that makes the header file from the Matlab file
>> cat import_data.h
#define TBL_SIZE 4 // In your case it is 1000
const double table[TBL_SIZE][2] =
{
{ 0.022815, 0.7 },
{ 6.9453, 21.85 },
{ 4.666, 565.9},
{ 567.9, 34.6}
};
Now in the main program you include this header also for the data
>> cat lookup.c
#include <stdio.h>
#include "import_data.h"
double lookup(double key)
{
int i=0;
for(;i<TBL_SIZE; i++) {
if(table[i][0] == key)
return table[i][1];
}
return -1; //error
}
int main() {
printf("1. Value is %f\n", lookup(6.9453));
printf("2. Value is %f\n", lookup(4.666));
printf("3. Value is %f\n", lookup(4.6));
return 0;
}
Yes, you can read them from the dat file. The question is, what format is the dat file? Once you know that, you want to use:
fopen
fread
fclose
for C and
ifstream
for C++ (or something similar).
The program still has to get those pairs from the file and load them in memory. You can loop through the lines in the file, parse the pairs and shove them in a std::map.
Something like this:
#include<fstream>
#include<map>
...
ifstream infile("yourdatfile.dat");
std::string str;
std::map<double, double> m; //use appropriate type(s)
while(getline(infile, str)){
//split str by comma or some delimiter and get the key, value
//put key, value in m
}
//use m
For the signal processing toolbox you can export data to C header files
directly from Matlab(don't know if it's your particular case):
Matlab export to C header
Or maybe the following article could be of help:
Exporting/Importing Data To/From MATLAB
One of options is to generate the C++ lookup table in matlab. Just write to some text file (lookup.cpp), read table producing C++ source...