Please tell me that the following line writes the address of the structure variable to the file or it writes the values of members of the structure.
file_write.write((char*)&structure_data,sizeOfStructure);
Where file_write is fstream's object and "structure_data" is a struct having 3 members of integer datatypes. Thanks.
This line:
file_write.write((char*)&structure_data,sizeOfStructure);
takes whatever structure_data is, and just copies those blob of bytes that makes up structure_data to a file.
It doesn't figure out what the members are. Also, this is the cause for thousands of SO questions that erroneously do coding like this, where structure_data cannot be written to a file this way and have the file contents make sense. It is quickly discovered that the contents of the file are useless when an attempt to read back the data into a program
is unsuccessful.
Most of the time in those scenarios structure_data would contain pointers, or members that are not C-layout compatible, i.e. non-POD types such as std::string or std::vector, that basically renders this technique of writing to a file like this totally useless (and invalid).
Look up object serialization such as this link on the topic
Related
My plan is to build a fileContainer that consists mainly of a std::map that associate specific files to an ID. Each file has for attributes a ofstream, a path (string) and a few other information.
The problem is that an ofstream cannot be copied (Why copying stringstream is not allowed?) and cannot even be moved (Why can't std::ostream be moved?). One can therefore not create file objects to then insert them into the map of the fileContainer.
What I am trying to do is something like
file f(arguments...); // create a `file`
FC.insert(f); // Note the FC is a global singleton of class `fileContainer`
...
{
file& f = FC.getFile(fileID); // find a specific file in the file container
f.open();
f.write(s1);
f.write(s2);
f.write(s3);
f.close();
}
I fail to see how such functionality could be achieved without having to copy or move a stream. Can you help me out to build this type of functionality?
Response to #πάνταῥεῖ comment
My code can produce about 20 different types of files. The types of files (which I above call ID) that are actually being produce depends upon the user input.
My goal is to have a fileContainer in which, I insert file objects. Those file objects are created while I read the input. Each file object match to a file ID. For any given file object a single file is being produced but for others, several files are being produced (information that can be gathered only during the process).
During the process, I would just look at whether a given ID is present in the fileContainer and if it is, then I write to it the associated file.
I already have a working version of the code, the issue is that it does not compile on every machine (looks like some compilers are fine with moving streams while others aren't)
If you only need std::ofstream
No problem! These are moveable.
If you need any std::ostream
Since std::ostreams cannot be copied or moved, your only option is to go back in time and store pointers instead. Use dynamic allocation to get fine control over the lifetime of your stubborn stream objects.
Abstracting away your custom container, a basic example looks like this:
std::vector<std::unique_ptr<std::ostream>> container;
container.push_back(std::make_unique<std::ostringstream>());
container.push_back(std::make_unique<std::ofstream>("/tmp/lol"));
…and so forth.
Nice, huh? #c++
I have a C++ object which needs a huge amount of data to instantiate. For example:
class object {
public object() {
double a[] = { array with 1 million double element };
/* rest of code here*/};
private:
/* code here*/
}
Now the data (i.e 1 million double numbers) is in a separate text file. The question: How can I put it after "double a[]" in an efficient way and eventually compile the code? I do not want to read the data at run time from a file. I want it compiled with the object. What can be a solution? Ideally I would like the data to sit in the separate text file as it presently resides and somehow also have an assignment like double a[] =..... above.
Is this possible? Thanks in advance!
Something like:
class object
{
public
object(){ double a[] = {
#include "file.h"
};
/* rest of code here*/};
private:
/* code here*/
}
The file has to be formatted correctly though - i.e. contain something like:
//file.h
23, 24, 40,
5, 1.1,
In general, you can use #include directives to paste content into files. I've seen virtual methods being pasted like that, if they were common for most derived classes. I personally don't really like this technique.
One large problem with this design is that 1 million ints on the stack will probably blow the stack. What you probably want is to put the data on the data segment, or in some kind of resource that is stores in your binary file and can be loaded at run time. If you need more than one copy of the data, duplicate it into a std::vector at run time, so you know the data is on the free store (heap). Mayhap even use a shared_ptr to a std::array to reduce the chance of needless accidental duplication(or unique_ptr to reduce the chance of reference duplication).
4mb of data is not going to play all that well is all I am saying. And locality of reference on a 4mb array to your other variables is not going to be your biggest concern.
Depending in your compiled target platform and framework, there will be ways to stuff this kind of data into a binary resource. I've never done it for a multi-meg file, but here is the visual studio help on resource files: http://msdn.microsoft.com/en-us/library/7zxb70x7%28v=vs.80%29.aspx
Note that "the data being in the code" does not make it fundamentally faster to load (other than traversing the filesystem once to find it maybe). The OS still has to load the binary, and larger binaries take more time to load, and a big array of values will take up as much room in a binary as it does in a distinct file. The real advantage is that it isn't a file that can be "misplaced" relative to your executable, but resource fork/resource file/etc methods can deal with that.
As noted in the comments below, static const data (and global data) tends to be loaded into the data segment, which is distinct from both the heap (aka free store) and stack (aka automatic store). I forget what the standard calls it. I do know that a static local variable in a function will behave differently than a static or global non-local variable with regards to initialization order (global (static or not) data gets initialized fully prior to main starting, while static local is initialized the first time the function is called, if I remember correctly).
The answer of Luchian Grigore is quite correct. But compiler can have some limit on length of source code line. See for example https://stackoverflow.com/questions/10519738/source-line-length-limit
So try on your compiler. But I am afraid, more simple solution of your problem will be reading of huge data from file.
I have code that currently passes around a lot of (sometimes nested) C (or C++ Plain Old Data) structs and arrays.
I would like to convert these to/from google protobufs. I could manually write code that converts between these two formats, but it would be less error prone to auto-generate such code. What is the best way to do this? (This would be easy in a language with enough introspection to iterate over the names of member variables, but this is C++ code we're talking about)
One thing I'm considering is writing python code that parses the C structs and then spits out a .proto file, along with C code that copies from member to member (in either direction) for all of the types, but maybe there is a better way... or maybe there is another IDL that already can generate:
.h file containing all of nested types
.proto file containing equivalents
.c file with functions that copy either direction between the C++ structs that the .proto file generates and the structs defined in the .h file
I could not find a ready solution for this problem, if there is one, please let me know!
If you decide to roll your own in python, the python bindings for gdb might be useful. You could then read the symbol table, find all structs defined in specified file, and iterate all struct members.
Then use <gdbtype>.strip_typedefs() to get the primitive type of each member and translate it to appropriate protobuf type.
This is probably safer then a text parsers as it will handle types that depends on architecture, compiler flags, preprocessor macros, etc.
I guess the code to convert to and from protobuf also could be generated from the struct member to message field relation, but does not sound easy.
Protocol buffers can be built by parsing an ASCII representation using TextFormat. So one option would be to add a method dumpAsciiProtoBuf to each of your structs. The method would dump any simple fields (like strings, bools, etc) and call dumpAsciiProtoBuf recursively on nested structs fields. You would then have to make sure that the concatenated result is a valid ASCII protocol buffer which can be parsed using TextFormat.
Note though that this might have some performance implications (since parsing the ASCII representation could be expensive). However, this would save you the trouble of writing a converter in a different language, so it seems to be a convenient solution.
I would not parse the C source code myself, instead I would use the LibClang to parse C files into an AST and my own AST walker to generate the Protobuf and the transcoders as necessary. Googling for "libclang walk AST" should give something to start with, like ast-walker.cc and ast-dumper.cc from this github repository, for example.
The question brought up is the age old challenge with "C" (and C++) code - No easy (or standard) way to reflect on c "struct" (or classes). Just search stack overflow on C reflection, and you will see lot of unsuccessful attempts. My first advice will be NOT to try to build another solution (in python, etc.).
One simple approach: Consider using gdb ptype to get structured output for you structures, which you can use to create the .proto file. The advantage is that there is no need to handle the full syntax of the C language (#define, line breaks, ...). See How do I show what fields a struct has in GDB?
From the gdb ptype, it's a short trip to protobuf '.proto' file.
You can get similar result from libCLang (and I believe there is comparable gcc plugin, but I can not locate it). However, you will have to write some non-trivial "C" code.
Another approach - will be to use 'swig' (https://www.swig.org), and process the swig xml output (or the -xmlout option) to dump the parse tree into XML. While this approach will require a little bit of digging to locate the structure that are needed, the information in XML format is complete, easy to parse (using whatever XML parser you want - python, perl). If you are brave enough, you can use xslt to generate the output.
all. A new project of mine involves reading names from a file, and I realized, especially for somebody who likes to (attempt to, more like) makes games, reading/writing to store information is very useful. I looked into it and discovered the std library pulls through again. I subsequently realized that, at least for me, the libraries from ios, ios_base, iostream, fstream, etc etc, seem to be kind of complex.
I looked around, but am not quite sure why this specific approach does not work. It puzzles me because examples found online that seem to follow the library as they should - I will refer to one on cplusplus.com. Of interesting note is that one worked as expected, while another didn't. Far as I know, that is.
My problem is, I can create a file using ofstream, but not ifstream nor fstream. The way I understand it, there constructors, which I was using (which I beleive is identical paramaters as you would pass to open - that is, the filename and flags) are identical except in that they have different default perameters - ostream has ios::out, istream has ifstream has ios::in, and fstream has ios::in | ios::out - both flags, which is logical since it is the combination of both classes. Note this is not a problem of WHERE the file is created, rather, but the fact of its creation. I know this by two reasons. First, when using ofstream the file appears in expected directory, but not at all with ifstream nor fstream. Second, using the is_open function, (specifically, testing the expressions "while (is_open)" , the console never closed (I test most new concepts in the console because, well, it is simple), but with the others it did. If it had, then it never would have ended, since it can never go out of scope and thus the destructor spells the end for the open file.
My second problem was with using gcount - mostly how to access it. No matter how I tried (std::gcount, file::gcount, fstream::gcount...) it never recognized what it was. I was a bit befuddled.
Now comes a bit off off-topic ness to the specific issue, but more into the general reason of why I encountered this problem to begin with, which is sort of a discussion but I think it could be beneficial thing to learn, unless somehow I am the only one with this problem.
Firstly, the tuturial I had read over (learncpp.com) goes over the seekp/seekg functions, and said that it moves a relative amount in BYTES. It occured to me this is generally the same as a char, but is it possible that this is different on a different system (i.e, if a char was 2 bytes, you could figure this out, and apply this to the file searching), or will it always go by character, essentially, anyway?
And my main wonder... how to really use these tools to do things. Let's say I want to put a name on each line. I would be a bit unsure how to do that, - newline character perhaps? Or the other end, reading a name per line - how do you find the starting position of the next line? (my issue with gcount arrises because I figured you could use getline(), then gcount(), and voila, move that many characters... not sure if that works though)
Thanks.
To answer just one of your questions: you can't create a file with ifstream because, well, it doesn't create files; it's simply not intended to. This purpose of this class is to read data from existing files, and to open a file and read from it, it must exist first. Simple as that.
As far as fstream goes, you could make a case that it would make sense for it to create a file if none exists, but in fact, that's not what it does; the argument to fstream's constructor must be an existing file. If the named file doesn't exist, a failure bit is set in the object, and no file is created.
Is it possible in C++ to create a struct type based on file contents?
STRUCT_NAME
int var1;
int var2;
string v3;
STRUCT NAME *pointer;
The first line would be the name of the struct, the other lines would be the variables
Do you mean on the fly, when your application is running? No, you can't do that, all types must be known at compile time. Of course, you can always create some kind of container object that could be configured dynamically during run-time, but that is a much more advanced system.
On the other hand, if you mean to do this once when you build you application, you could write a simple tool that eats your text file and emits a C++ header file, that you later could use when compiling.
If you are reading from a file and trying to create this struct,
--> if in same program, i.e. runtime then NO; because you can not compile the source code which is presently executing
--> for some different source code which is yet to be compiled and you are manipulating using file operations then YES
No. The closest you can come is an std::map<std::string, boost::any> (or boost::variant if you can limit the set of types).
Whatever #lindydancer #iammilind #james are saying is correct.. You cannot do it in single go. You may get work around (Though thats not professinal way of doing) like this :
1. First read the file in which your structure is define and create new file(Cpp and h files).
2. Now compile new file and provide new exe file to your user, you can do this process in background so that for user its all dynamic.. (But atlease someone has to do this work...)