I am using the C++ streams to read in a bunch of files in a directory and then write them to another directory. Since these files may be of different types, I am using a the generic ios::binary flag when reading/writing these files.
Example code below:
std::fstream inf( "ex.txt", std::ios::in | std::ios::binary);
char c;
while( inf >> c ) {
// writing to another file in binary format
}
The issue I have is that in the case of files containing text, the end of line characters in these text files are not being written to the output file.
Edit: Or at least they do not appear to be as when the newly written file is opened, there is only a single continuous line of characters.
Edit again: The problem (of the continuous string) appears to persist even when the read / write is made in text mode.
Thus, I was wondering if there was a way to check if a file has text or binary and then read/write it appropriately. Else, is there any way to preserve the end of line characters even when opening the file in binary format?
Edit: I am using the g++ 4.8.2 compiler
When you want to manipulate bytes, you need to use read and write methods, not >> << operators.
You can get the intended behavior with inp.flags(inp.flags() & ~std::ios_base::skipws);, though.
Related
(My previous questions was closed as a duplicate of Are there binary memory streams in C++ which is ridiculous, since i can't change the implementation of the library I'm using)
I'm using a library (Poco) to create zip files. It takes ostream as an input and writes the data of the zip file into it. Something like:
std::ofstream ofs("file.zip", std::ios::binary);
Compress compress(ofs);
// add data to compress ...
compress.close();
// now file.zip contains added file
This works. But I want to be able to create a zip in memory without creating a file. I tried using stringstream instead of ofstream, i get additionl newline characters in the data in the zip file is corrupted. Is there any other stream i can use?
(If someone still thinks it's a duplicate, I'm gonna need an explanation, since I don't see how this other question is helpful for me)
Use a std::stringstream -- that will create an in-memory string that you can write to as an ostream, and WILL NOT add extra newlines. If you later copy the string to an fstream that was opened in text mode (such as std::cout), then that process may add extra CR characters that are not in the string (nor in the original output).
If you are seeing extra characters corrupting your stream, they are coming from somewhere else -- something besides you compress call/lib is writing to the stream, or something with how you are looking at your stream is doing something.
If you're on linux, how about creating an anonymous file using memfd_create? You can then open /proc/self/fd/<fd> and do your stuff. Some implementations of std::ofstream may even provide a constructor that takes a FILE*, you can check if that's the case on your system.
I want to download a file to memory using curl.
I am currently using this and it looks like it works to a certain extent but corrupts my file.
I have used sigbench and it is around 20% different (while comparing original and downloaded)
The file I want to download is a binary so it won't work after it's modified.
I am currently testing with the x86 version of this.
original binary
downloaded binary
This is the code I am using to write it to a file:
ofstream stream = ofstream("test.dll");
stream.write(chunk.memory, chunk.size);
Opening the file like this:
ofstream stream = ofstream("test.dll");
will cause line-end characters to be adjusted to match your target system.
You should instead open the file in binary mode:
ofstream stream = ofstream("test.dll", std::ios::binary);
This will leave the characters that could be interpreted as line-endings unchanged.
Further reading: https://en.cppreference.com/w/cpp/io/c#Binary_and_text_modes
I'm trying to write a Huffman encoder but I'm getting some compression errors. I identified the problem as mismatches between characters that were put() to the ofstream and the characters read() from the same file.
One specific instance of this problem :
The put() writes ASCII character 10 (Line feed)
The read() reads ASCII character 13 (Carriage return)
I thought read and put read and write raw data ( no character translations ) I'm not sure why this is happening. Can someone help me out?
Here is the ofstream instance for writing the compressed file:
std::ofstream compressedFileStream(getCompressedFileName(),std::ios::binary||std::ios::ate);
and the ifstream instance for reading the same
std::ifstream fileInput(getFileName()+".huf",std::ios::binary);
The code is running on Windows 7 and all streams in the program are opened in binary mode.
Not opening in binary mode due to a typo:
std::ofstream compressedFileStream(getCompressedFileName(),std::ios::binary||std::ios::ate)
should be:
std::ofstream compressedFileStream(getCompressedFileName(),std::ios::binary|std::ios::ate)
// ^
|, not ||.
The symptoms show that you are creating the ofsteam with text mode or you are creating it using a filedesc that is opened in text mode.
You will want to pass ios::binary to it at construction time or it may run in text mode on Windows.
After you added the code, the reason proves to be a typo;
std::ios::binary||std::ios::ate
should be
std::ios::binary|std::ios::ate
On Windows, if you are writing binary data, you need to open the file with the appropriate attributes.
Similarly, if you are reading binary data, you need to open the file with the appropriate attributes.
I have a problem with with my output when I write to I file I get squares when I put endl to change lines.
std::ofstream outfile (a_szFilename, std::ofstream::binary);
outfile<<"["<<TEST<<"]"<<std::endl;
I get something like this in my file plus the other outputs don't write on the next line but on the same one.
[TEST]square
apparently I can't write the square here, but is it something about the ofstream being binary or something?
You don't really want to open the file in binary mode in this case.
Try this instead:
std::ofstream outfile (a_szFilename);
outfile<<"["<<TEST<<"]"<<std::endl;
You're opening the file in binary mode. in this case the endl is written as \n while a newline on windows is supposed to be \r\n
To open you file in text mode just don't include the binary flag the translation will be done automatically
std::ofstream outfile(a_szFilename);
outfile<<"["<<TEST<<"]"<<std::endl;
It's probably because you're in binary mode and the line endings are wrong. std::endl will place '\n' on the stream before flushing. In text mode, this will be converted to the correct line ending for your platform. In binary mode, no such conversions take place.
If you're on Windows, your code will have a line feed (LF), but Windows also requires a carriage return (CF) first, which is '\r'. That is, it wants "\r\n", not just a newline.
Your fix is to open the file in text mode. Binary files are not suppose to be outputting newlines or formatted output, which is why you don't want to use the extraction and insertion operators.
If you really want to use binary, then treat your file like a binary file and don't expect it to display properly. Binary and formatted output do not go hand in hand. From your usage, it seems you should be opening in text mode.
When I do
fstream someFile("something.dat", ios::binary|ios::out);
someFile.seekp(someLocation, ios::beg);
someFile.write(someData, 100);
It seems to replace the entire file with those 100 bytes instead of replacing only the appropriate 100 bytes, as if I had specified ios::trunc. Is there a portable way to not have it truncate the file?
Edit: adding ios::in seems to do the trick, by why is this required, and is that standard behavior?
Edit #2: I am not trying to append to the existing file. I need to replace the 100 bytes while leaving the rest unaffected.
You want the append flag, ios::app, if you want to write at the end of the file.
To do it somewhere arbitrarily in the middle of the file, you need to seek to the right place. You CAN do this by opening the file for in and out, but if I were you I'd create a temp file, copy input up to mark, write your new data, copy the rest to EOF, close the files and replace the previous version with the temp file. This is called a "Master File update".
AFAIR ios::out only specifies the file is for output and ios:binary only specifies the files is binary. The default behaviour for a stream is to create a new file and overwrite the old file. If you want to modify an existing file you must open the file with the ios::app flag.
ATM I cannot check my references so be sure to double check, but I felieve that is accurate.
Since the file already exists open it in 'read+write' mode and then do seekp. I think it will work.
fstream someFile("something.dat", ios::binary|ios::out|ios::in);
someFile.seekp(someLocation, ios::beg);
someFile.write(someData, 100);
The ios:in mask tells the file pointer to position at the beginning of the file so it can start reading in from the start. But, you may want to use fseek to set the file pointer at the beginning of the file.