fwrite() writes garbage at the end - c++

I'm trying to write a function that execute a sql file with postgres. Postgres rise me an exception but without specificate the error. So I tryed to rewrite what it read, and I discovery that the file has some garbage at end
stat("treebase.sql",&buf);
dbschema= new (std::nothrow) char[buf.st_size+1];
if(!dbschema)
{
wxMessageBox(_("Not Enough memory"));
return;
}
if( !(fl=fopen("treebase.sql","r")))
{
wxMessageBox(_("Can not open treebase.sql"));
delete []dbschema;
return;
};
fo=fopen("tbout.sql","w");
fread(dbschema,sizeof(char),buf.st_size,fl);
fclose(fl);
dbschema[buf.st_size]='\0';
fwrite(dbschema,sizeof(char),buf.st_size+1,fo);
fflush(fo);
fclose(fo);
and the result is
![screen shot][1]
The input file 150473 length, the output is 156010. I really can not undersand where the 5000 bytes come from.
where is the bug?
[1]: https://i.stack.imgur.com/IXesz.png

You probably can't read buf.st_size much of data, because of the mode of fopen is "r" which defaults to text modes. In text mode fread and fwrite may do some conversions on what you read or write to match the environment special rules about text files such as end of lines. Use "rb" and "wb" modes for fopen for reading and writing binary files as is respectively.
Also, I would rather use fseek and ftell to get the size of file instead of stat.

Here's an example of how you could read the content of the file into memory and then write down an exact copy to another file. I added error checking too to make it clear if anything goes wrong. There's also no need to use stat etc. Plain standard C++ will do.
#include <cerrno>
#include <cstring>
#include <fstream>
#include <iostream>
#include <iterator>
#include <stdexcept>
#include <string>
std::string get_file_as_string(const std::string& filename) {
std::ifstream fl(filename, std::ios::binary); // binary mode
if(!fl) throw std::runtime_error(std::strerror(errno));
// return the content of the whole file as a std::string
return {std::istreambuf_iterator<char>(fl),
std::istreambuf_iterator<char>{}};
}
bool write_string_to_file(const std::string& str, const std::string& filename) {
std::ofstream fo(filename, std::ios::binary);
if(!fo) return false;
// return true or false depending on if it succeeded writing the file:
return static_cast<bool>(fo << str);
}
int main() {
auto dbschema = get_file_as_string("treebase.sql");
// use dbschema.c_str() if you need a `const char*`:
postgres_c_function(dbschema.c_str());
// use dbschema.data() if you need a `char*`:
postgres_c_function(dbschema.data());
if(write_string_to_file(dbschema, "tbout.sql")) {
std::cout << "success\n";
} else {
std::cout << "failure: " << std::strerror(errno) << '\n';
}
}

Related

ifstream keeps read even though the file does not exists anymore

I'm facing a problem using std::ifstream to read a file. I have a zip file called "nice_folder.zip" (71.6MB) and the following code that reproduces the issue:
#include <filesystem>
#include <fstream>
#include <iostream>
#include <memory>
#include <unistd.h>
int main() {
size_t read = 0;
size_t f_size = std::filesystem::file_size("nice_folder.zip");
std::shared_ptr<char[]> buffer{new char[4096]};
std::ifstream file{"nice_folder.zip"};
while (read < f_size) {
size_t to_read = (f_size - read) > 4096 ? 4096 : (f_size - read);
file.read(buffer.get(), to_read);
sleep(2);
std::cout << "read: " << std::to_string(to_read) << "\n";
}
}
The problem is the following: after some read cycles I delete the file from the folder but it keeps reading it anyway. How is it possible ? How can I catch an error if an user delete a file while reading using ifstream ? I guess that ifstream takes the content of the file into memory before start reading but i'm not sure.
If you're doing this on e.g. linux, then the OS will not actually delete the file until you've closed all file handles to it. So it might seem like the file is deleted, but it's still stored somewhere on disk. See e.g. What happens internally when deleting an opened file in linux.
So if you're trying to detect this file deletion to prevent wrong reads, then don't worry, the file won't actually be deleted.
If you close the stream, and then try and open it again for that file, you'll get an error. But that also means you won't be able to read from it...

Letting a `std::vector<unsigned char>` steal memory from a `std::string`

Suppose we have std::string s holding a raw data buffer, but we want std::vector<uint8_t> v instead. The buffer length is in the millions. Is there a simple way to let v steal s's memory and thereby avoid copying the buffer?
Like what std::vector<uint8_t>::vector(std::string&&) would have done, but doing it somehow from the outside of STL.
Alternatively, is it possible to get v from std::stringstream ss with an operation about as efficient as ss.str()?
OK, lots of comments there, let's try and put something together, because I need the practise and might maybe earn some points [update: didn't :(].
I'm fairly new to 'modern C++' so please take it as you find it. Might need as late as C++17, I haven't checked that too carefully. Any critique more than welcome but I would prefer to edit my own post. And please bear in mind when reading this that what the OP actually wants to do is read his bytes from a file. Thx.
Update: Tweaked to handle the case where the the file size changes between the call to stat() and the call to fread() as per #Deduplicator's comment below ... and subsequently replaced fread with std::ifstream, I think we're there now.
#include <string>
#include <vector>
#include <optional>
#include <iostream>
#include <fstream>
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
#include <stdio.h>
#include <errno.h>
using optional_vector_of_char = std::optional <std::vector <char>>;
// Read file into a std::optional <std::vector <char>>.
// Now handles the file size changing when we're not looking.
optional_vector_of_char SmarterReadFileIntoVector (std::string filename)
{
for ( ; ; )
{
struct stat stat_buf;
int err = stat (filename.c_str (), &stat_buf);
if (err)
{
// handle the error
return optional_vector_of_char (); // or maybe throw an exception
}
size_t filesize = stat_buf.st_size;
std::ifstream fs;
fs.open (filename, std::ios_base::in | std::ios_base::binary);
if (!fs.is_open ())
{
// handle the error
return optional_vector_of_char ();
}
optional_vector_of_char v (filesize + 1);
std::vector <char>& vecref = v.value ();
fs.read (vecref.data (), filesize + 1);
if (fs.rdstate () & std::ifstream::failbit)
{
// handle the error
return optional_vector_of_char ();
}
size_t bytes_read = fs.gcount ();
if (bytes_read <= filesize) // file same size or shrunk, this code handles both
{
vecref.resize (bytes_read);
vecref.shrink_to_fit ();
return v; // RVO
}
// File has grown, go round again
}
}
int main ()
{
optional_vector_of_char v = SmarterReadFileIntoVector ("abcde");
std::cout << std::boolalpha << v.has_value () << std::endl;
}
Live demo. No actual file available to read in of course, so...
Also: Have you considered writing your own simple container that maps a view of the file? Just a thought.

read and write a binary file in c++ with fstream

I'm trying to write simple c++ code to read and write a file.
The problem is my output file is smaller than the original file, and I'm stuck finding the cause.
I have a image with 6.6 kb and my output image is about 6.4 kb
#include <iostream>
#include <fstream>
using namespace std;
ofstream myOutpue;
ifstream mySource;
int main()
{
mySource.open("im1.jpg", ios_base::binary);
myOutpue.open("im2.jpg", ios_base::out);
char buffer;
if (mySource.is_open())
{
while (!mySource.eof())
{
mySource >> buffer;
myOutpue << buffer;
}
}
mySource.close();
myOutpue.close();
return 1;
}
Why not just:
#include <fstream>
int main()
{
std::ifstream mySource("im1.jpg", std::ios::binary);
std::ofstream myOutpue("im2.jpg", std::ios::binary);
myOutpue << mySource.rdbuf();
}
Or, less chattily:
int main()
{
std::ofstream("im2.jpg", std::ios::binary)
<< std::ifstream("im1.jpg", std::ios::binary).rdbuf();
}
Two things: You forget to open the output in binary mode, and you can't use the input/output operator >> and << for binary data, except if you use the output operator to write the input-streams basic_streambuf (which you can get using rdbuf).
For input use read and for output use write.
There are 3 problems in your code:
1- You have not opened your output file in Binary.
2- Your code return "1", normally you should return "0", if something went wrong then return an error code.
3- You should use "manipulators" and make c++ not to avoid whitespaces, so in order to read from file instead of:
mySource >> buffer;
you should use:
mySource >> std:noskipws >> buffer;
Well, its just because of padding at the end of the image. eof of any file do not include the padded bytes added at the end of file.
Try this
take img1.jpg contains 20 space charecter at the end not visible here (uegfuyregwfyugwrerycgerfcg6ygerbucykgeugcrgfrgeyf ) and run your program (do not include parenthesis in the file, these are used to show the data content)
you will see img2.jpg contains (uegfuyregwfyugwrerycgerfcg6ygerbucykgeugcrgfrgeyf)
So, its better option to read the file byte by byte using the filesize which you can get using stat, and run for loop till filesize. Hope this should resolve your problem you mentioned above

C++ write to file error with full PATH from GetEnvironmentVariable()

I'm noob in C++ but wanting to learn. I have a little program that writes some info to my \etc\hosts in Windows; I get the %WINDIR% variable via GetEnvironmentVariable(), if I put the full path manually everything is ok, but when I substitute with WINDIR variable my code isn't compiling. I know I don't do something right.
#include <windows.h>
#include <ios>
#include <fstream>
char buffer[1000];
int main() {
GetEnvironmentVariable("WINDIR",(char*)&buffer,sizeof(buffer));
std::ofstream log;
log.open("%s\\system32\\drivers\\etc\\hosts", buffer);
log << "127.0.0.1 domain.com\n" << std::endl;
return 0;
}
I get really ugly errors like:
C:\Documents and Settings\xtmtrx\Desktop\coding\windir.cpp no matching function for call to `std::basic_ofstream<char, std::char_traits<char> >::open(const char[30], char[1000])'
ofstream cannot format the path for you. You need to do that separately, eg:
#include <windows.h>
#include <ios>
#include <fstream>
char buffer[1000] = {0};
int main() {
GetEnvironmentVariable("WINDIR",buffer,sizeof(buffer));
strcat(buffer, "\\system32\\drivers\\etc\\hosts");
std::ofstream log;
log.open(buffer, ios_base::ate);
log << "127.0.0.1 domain.com\n" << std::endl;
return 0;
}
FYI, you should use GetWindowsDirectory(), GetSystemDirectory(), SHGetSpecialFolderPath() or SHGetKnownFolderPath() instead of GetEnvironmentVariable(). And you should use PathCombine() when concantenating paths together so it can ensure the slashes are correct.
open("%s\\system32\\drivers\\etc\\hosts", buffer); open doesn't understand format strings..you are using %s does not make sense. learn here
Try like this:
GetEnvironmentVariable("WINDIR",buffer,sizeof(buffer));
strcat(buffer, "\\system32\\drivers\\etc\\hosts");
std::ofstream log;
log.open(buffer.str().c_str(), ios_base::ate);
You need to concate the string together like this:
LPTSTR windir[MAX_PATH];
LPTSTR fullpath[MAX_PATH];
GetWindowsDirectory(windir, MAX_PATH);
if(PathCombine(fullpath, windir, _T("system32\\drivers\\etc\\hosts")) != NULL) {
std::ofstream log;
log.open(buffer, ios_base::ate);
log << "127.0.0.1 domain.com\n" << std::endl;
}
At first you need to concate the directory and the file part with PathCombine. Then you can open the file and write the content. You should also note that you need admin permissions to change this file and some antivirus programmes may reject the access of the hosts file.

When will ofstream::open fail?

I am trying out try, catch, throw statements in C++ for file handling, and I have written a dummy code to catch all errors. My question is in order to check if I have got these right, I need an error to occur. Now I can easily check infile.fail() by simply not creating a file of the required name in the directory. But how will I be able to check the same for outfile.fail() (outfile is ofstream where as infile is ifstream). In which case, will the value for outfile.fail() be true?
sample code [from comments on unapersson's answer, simplified to make issue clearer -zack]:
#include <fstream>
using std::ofstream;
int main()
{
ofstream outfile;
outfile.open("test.txt");
if (outfile.fail())
// do something......
else
// do something else.....
return 0;
}
The open(2) man page on Linux has about 30 conditions. Some intresting ones are:
If the file exists and you don't have permission to write it.
If the file doesn't exist, and you don't have permission (on the diretory) to create it.
If you don't have search permission on some parent directory.
If you pass in a bogus char* for the filename.
If, while opening a device file, you press CTRL-C.
If the kernel encountered too many symbolic links while resolving the name.
If you try to open a directory for writing.
If the pathname is too long.
If your process has too many files open already.
If the system has too many files open already.
If the pathname refers to a device file, and there is no such device in the system.
If the kernel has run out of memory.
If the filesystem is full.
If a component of the pathname is not a directory.
If the file is on a read-only filesystem.
If the file is an executable file which is currently being executed.
By default, and by design, C++ streams never throw exceptions on error. You should not try to write code that assumes they do, even though it is possible to get them to. Instead, in your application logic check every I/O operation for an error and deal with it, possibly throwing your own exception if that error cannot be dealt with at the specific place it occurs in your code.
The canonical way of testing streams and stream operations is not to test specific stream flags, unless you have to. Instead:
ifstream ifs( "foo.txt" );
if ( ifs ) {
// ifs is good
}
else {
// ifs is bad - deal with it
}
similarly for read operations:
int x;
while( cin >> x ) {
// do something with x
}
// at this point test the stream (if you must)
if ( cin.eof() ) {
// cool - what we expected
}
else {
// bad
}
To get ofstream::open to fail, you need to arrange for it to be impossible to create the named file. The easiest way to do this is to create a directory of the exact same name before running the program. Here's a nearly-complete demo program; arranging to reliably remove the test directory if and only if you created it, I leave as an exercise.
#include <iostream>
#include <fstream>
#include <sys/stat.h>
#include <cstring>
#include <cerrno>
using std::ofstream;
using std::strerror;
using std::cerr;
int main()
{
ofstream outfile;
// set up conditions so outfile.open will fail:
if (mkdir("test.txt", 0700)) {
cerr << "mkdir failed: " << strerror(errno) << '\n';
return 2;
}
outfile.open("test.txt");
if (outfile.fail()) {
cerr << "open failure as expected: " << strerror(errno) << '\n';
return 0;
} else {
cerr << "open success, not as expected\n";
return 1;
}
}
There is no good way to ensure that writing to an fstream fails. I would probably create a mock ostream that failed writes, if I needed to test that.