I'm trying to implement a FUSE driver.
What I've done, is taken the basic operations that you override with your own function, and I've added a cout << operation << path << endl; So I can see which operation gets called. Here is my open function:
int Router::open(const char *path, struct fuse_file_info *fi) {
std::cout << "open " << path << std::endl;
return 0;
}
This returns 0 so the open should succeed all the time.
Then, here is my read function:
int Router::read(const char *path, char *buf, size_t size, off_t offset, struct fuse_file_info *fi) {
std::cout << "read " << path << "Size: " << size << "Offset: " << offset << std::endl;
buf[0] = 'h';
buf[1] = '\0';
return 2;
}
This should copy over h\0 to the buffer.
But when I do: cat myfile it runs my open() function, which succeeds, then it runs my read() function, but the output of cat is nothing. It should output h but it doesn't. What am I doing wrong?
Also I find that under OSX, the exact same code only runs the open function, but never the read()
What could be the problem?
So my problem was that in the function: getattr(...) I did not specify the file size.
After doing: st->st_size = 4096; Everything was working fine :)
Related
IN IOS app, module written in C++ I am writing my data (map of basic strings and integers) to a text file. Using following method:
bool Recognizer::saveMap(const char * s)
{
if(trainingData.model && !trainingData.model.empty()) {
const string filename = string(s);
std::ofstream file(s, ios_base::trunc );
try{
if(! file.is_open())
{
file.open(s);
}
for (map<String,int>::iterator it=trainingData.idMap.begin(); it!=trainingData.idMap.end(); ++it)
{
cout << it->second << " " << it->first << endl;
file << it->first << endl << it->second << endl;
}
file.close();
}
catch(cv::Exception & e){
if(file.is_open())
file.close();
int code = e.code;
string message = e.err;
cerr << "cv::Exeption code: " << code << " " << message << endl;
return false;
}
std::streampos fileLength = iosFileSize(s);
cout << "Saved map to: " << filename << " length: " << fileLength << endl;
return true;
}
return false;
}
My contains one entry and console output indicates that two lines: string, string representing number have been written to my file.
Subsequent opening file for reading and reading using getline or using stream operator indicates that file is empty:
bool Recognizer::loadMap(const char * s)
{
std::streampos fileLenght = iosFileSize(s);
std::ifstream file(s, ios::in);
try{
if(file.is_open())
{
string name;
string lineName;
string lineTag;
int tag;
int count = 0;
while(getline(file,name))
{
if(getline(file,lineTag))
{
tag = stoi(lineTag,0,10);
count++;
cout << tag << " " << name << endl;
trainingData.idMap[name]=tag;
trainingData.namesMap[tag]=name;
}
}trainingData.personsCount=count;
file.close();
}
}
catch(cv::Exception & e){
if(file.is_open())
file.close();
int code = e.code;
string message = e.err;
cerr << "cv::Exeption code: " << code << " " << message << endl;
return false;
}
cout << "Loaded map from: " << s << " lenght: "<< fileLenght << endl;
return true;
}
I also copied from one of stackoverflow answers method returning file lenght and using it to verify lenghth of the file after write operation:
std::streampos iosFileSize( const char* filePath ){
std::streampos fsize = 0;
std::ifstream file( filePath, std::ios::binary );
fsize = file.tellg();
file.seekg( 0, std::ios::end );
fsize = file.tellg() - fsize;
file.close();
return fsize;
}
The file path passed to saveMap and loadMap seems to be legit. With path that the app could not write to, attempt to write caused exception.
There are no errors returned by write operation but both, attempts to read and iosFileSize() indicate that file is empty.
I am not sure if i need call file.open() and file.close() or file is open and closed automatically when output stream is created and later goes out of scope.
I experimented with those with the same result ( call to file.is_open returns true so the block calling file.open() is skipped.
What am I doing wrong?
I appreciate all responses.
It does not seem like you call file.flush(); anywhere in Recognizer::saveMap() after writing to the file stream. std::ofstream::flush() saves changes you've made to the file. Add file.flush(); between when you make changes to the code and when you close the file. See if that remedies your issue.
I also had the same issue. Using file.flush() everytime after you insert to a file can save your file.
However if you insert something like this, say,
file << "Insert This"; You will need to add file.flush().
But some people have issues, like if you just insert file << "Insert This" << endl; , this works fine. The key point here is that, std::endl calls flush() everytime it is used internally. you can say it is a shortend form of "\n" + flush().
I believe from looking at your code that you are overwriting your data when you open the file in the second program you should be using something like this.
std::fstream fs;
fs.open ("test.txt", ios::app)
instead of doing the ios::in
I'm working on what will be a pretty large and complicated system and trying to make sure it's as watertight as possible right from the start. Whilst running some memory checks, I noticed something odd when using stringstreams: they don't always seem to release all the memory when they get deleted/go out of scope.
I've tried searching the internet for answers, but most are old (so possibly out of date) and/or are more concerned with refreshing the contents than releasing the memory, so I've not really been able to tell if it's a known issue or a common mistake I'm making.
I've written a simple test to show what's going on:
#include <iostream>
#include <string>
#include <sstream>
#include <fstream>
using namespace std;
float getMemUsage(int& pid)
{
if (pid < 0)
pid = getpid();
char buf[30];
snprintf(buf, 30, "/proc/%u/statm", (unsigned)pid);
FILE* pf = fopen(buf, "r");
if (pf)
{
unsigned size; // total program size
//unsigned resident;// resident set size
//unsigned share;// shared pages
//unsigned text;// text (code)
//unsigned lib;// library
//unsigned data;// data/stack
//unsigned dt;// dirty pages (unused in Linux 2.6)
fscanf(pf, "%u" /* %u %u %u %u %u"*/, &size/*, &resident, &share, &text, &lib, &data*/);
fclose(pf);
return size/1024.0;
}
else
return -1.0;
}
int main(int argc, char* argv[])
{
if (argc < 2)
cerr << "no file specified\n";
ifstream file;
file.open(argv[1]);
int pid = -1;
const float memUseAtStart = getMemUsage(pid);
{
float memUseBefore = getMemUsage(pid);
stringstream sstream;
float memUseAfter = getMemUsage(pid);
cerr << "\tMemory use change after stringstream declaration: " << memUseAfter - memUseBefore << endl;
memUseBefore = getMemUsage(pid);
filebuf* pbuf = file.rdbuf();
memUseAfter = getMemUsage(pid);
cerr << "\tMemory use change after getting file buffer: " << memUseAfter - memUseBefore << endl;
memUseBefore = getMemUsage(pid);
sstream << pbuf;
memUseAfter = getMemUsage(pid);
cerr << "\tMemory use change after copying file contents: " << memUseAfter - memUseBefore << endl;
memUseBefore = getMemUsage(pid);
sstream.clear();
sstream.str( string() );
memUseAfter = getMemUsage(pid);
cerr << "\tMemory use change after 'clearing': " << memUseAfter - memUseBefore << endl;
}
cerr << "Overall memory use change: " << getMemUsage(pid) - memUseAtStart << endl;
file.close();
return 0;
}
Which gives me the following output when called with a file larger than around 32K:
Memory use change after stringstream declaration: 0
Memory use change after getting file buffer: 0
Memory use change after copying file contents: 0.0322266
Memory use change after 'clearing': 0
Overall memory use change: 0.00195312
I'm running on Linux (SL6.6) and compiling with gcc 4.1.2 (though I've also tried clang and ICC with similar results).
Obviously, it's not a huge leak; it's just a little annoying that I can't make it completely tidy... Is there something I can/should do to release the memory manually? Or is it just something weird (with my setup and/or stringstream itself) I'll have to live with?
NB The intended use for the stringstream is to read in some file contents above and then parse them line by line; I would try using istringstream but I couldn't figure out how to set its value from the ifstream...
Thanks!
I'm trying to make some experiments on disk I/O using cache and not using it. In order to perform a read directly from the disk, I open the file with the O_DIRECT flag (defining the variable DISK_DIRECT).
Now the two branches of the if beneath, should perform the same operation, with the difference that one is helped by the cache and the other not.
The files to which I try to access are stored on disk and they do not change over time.
Also the two branches access to the same files.
At some point here, when I use fread I get ferror() to be true. While when I use read everything goes fine.
I'm sure they access the same files.
Do you have any idea why this could happen?
EDIT
Ok, i'm posting here an minimal example. the code i use is:
#include <iostream>
#include <fcntl.h>
#include <sys/types.h>
#include <unistd.h>
#include <fstream>
#include <sstream>
using namespace std;
typedef float fftwf_complex [2] ;
void fetch_level(unsigned long long tid, unsigned short level, fftwf_complex* P_read, fftwf_complex* P_fread, int n_coeff_per_level, FILE** files_o_direct, fstream* & files) {
int b_read;
fseek(files_o_direct[level],(long int) (tid * sizeof(fftwf_complex)*n_coeff_per_level), SEEK_SET);
b_read = fread(reinterpret_cast<char*>(P_fread),sizeof(fftwf_complex), n_coeff_per_level,files_o_direct[level]);
if(b_read == 0){
cerr << "nothing read\n";
}
files[level].seekg((streamoff) (tid * sizeof(fftwf_complex)*n_coeff_per_level), files[level].beg);
files[level].read(reinterpret_cast<char*>(P_read),
sizeof(fftwf_complex) * n_coeff_per_level);
}
void open_files (fstream* & files){
for(int i=0; i<1;i++) {
std::ostringstream oss;
oss << "./Test_fread_read/1.txt.bin";
files[i].open(oss.str().c_str(),
std::ios::in | std::ios::out |
std::ios::binary | std::ios::ate);
if (!files[i])
{
cerr << "fstream could not open " << oss.str() << endl;
}
}
}
void open_files_o_direct (FILE** files_o_direct, int* fd){
for(unsigned int i=0;i<1; i++){
std::ostringstream oss;
oss << "./Test_fread_read/1.txt.bin";
fd[i]=open(oss.str().c_str(), O_RDONLY | O_DIRECT);
files_o_direct[i] = fdopen(fd[i], "rb");
if(!files_o_direct[i])
cerr << "Could not open " << oss.str() << endl;
}
}
int close_files(FILE** files_o_direct, int* fd, fstream* & files) {
for(unsigned int i=0; i<1; i++){
//#if defined (DISK_DIRECT)
if(files_o_direct[i])
close(fd[i]);
//#else
if(files[i].is_open())
files[i].close();
//#endif
}
return 0;
}
int main(){
FILE**files_o_direct = new FILE* [256];
fstream* files = new fstream [256];
int * fd = new int [256];
fftwf_complex * P_read = new fftwf_complex [1];
fftwf_complex * P_fread = new fftwf_complex [1];
open_files_o_direct(files_o_direct, fd);
open_files(files);
fetch_level(2, 0, P_read, P_fread, 1, files_o_direct, files);
cout << "P_read: " << P_read[0][0] << " P_fread: " << P_fread[0][0] << endl;
cout << "P_read: " << P_read[0][1] << " P_fread: " << P_fread[0][1] << endl;
fetch_level(7, 0, P_read, P_fread, 1, files_o_direct, files);
cout << "P_read: " << P_read[0][0] << " P_fread: " << P_fread[0][0] << endl;
cout << "P_read: " << P_read[0][1] << " P_fread: " << P_fread[0][1] << endl;
fetch_level(8, 0, P_read, P_fread, 1, files_o_direct, files);
cout << "P_read: " << P_read[0][0] << " P_fread: " << P_fread[0][0] << endl;
cout << "P_read: " << P_read[0][1] << " P_fread: " << P_fread[0][1] << endl;
close_files(files_o_direct, fd, files);
delete [] P_read;
delete [] P_fread;
delete [] files;
delete [] files_o_direct;
return 0;
}
and the file which is accessed is:
0.133919 0.0458176
1.67441 2.40805
0.997525 -0.279977
-2.39672 -3.076
-0.0390913 0.854464
-0.0176478 -1.3142
-0.667981 -0.486272
0.831051 0.282802
-0.638032 -0.630943
-0.669854 -1.49762
which is stored in a binary format and that can be download from here: 1.txt.bin.
The output i get is:
nothing read
P_read: 0.997525 P_fread: 0
P_read: -0.279977 P_fread: 0
nothing read
P_read: 0.831051 P_fread: 0
P_read: 0.282802 P_fread: 0
nothing read
P_read: -0.638032 P_fread: 0
P_read: -0.630943 P_fread: 0
The problem persists even if i change the type of fftwf_complex from float[2] to simple float.
If i remove the fseek line everything works correctly.
This if (b_read == 0), will be true at the end of the file, and you will enter this branch
if(ferror(this->files_o_direct[level]))
fseek(this->files_o_direct[level], 0, SEEK_END); //ftell here returns 4800000
cerr << "nothing read\n";
even if ferror returns 0, the end of the file was reached anyway
fseek(this->files_o_direct[level], 0, SEEK_END);
makes no sense, and "nothing read\n" will be output either or not ferror returns nonzero.
From the manual page
fread() does not distinguish between end-of-file and error, and callers must use feof(3) and ferror(3) to determine which occurred.
so you have to check feof and if it is false you use ferror.
For who ever may have the same problem here there is the answer:
The O_DIRECT flag may impose alignment restrictions on the length and
address of user-space buffers and the file offset of I/Os. In Linux
alignment restrictions vary by filesystem and kernel version and
might be absent entirely. However there is currently no
filesystem-independent interface for an application to discover these
restrictions for a given file or filesystem. Some filesystems
provide their own interfaces for doing so, for example the
XFS_IOC_DIOINFO operation in xfsctl(3).
Under Linux 2.4, transfer sizes, and the alignment of the user buffer
and the file offset must all be multiples of the logical block size
of the filesystem. Since Linux 2.6.0, alignment to the logical block
size of the underlying storage (typically 512 bytes) suffices. The
logical block size can be determined using the ioctl(2) BLKSSZGET
operation or from the shell using the command:
blockdev --getss
linux reference page
So my main function looks like this:
void main {
uint32 errmsg;
uint32 mydata;
if (LOG) {
std::ofstream file;
file.open(fileName,ios_base::app);
file << "!!!mydata: " << mydata << ",&mydata: " << &mydata << endl;
file.close();
}
errmsg = ReadReg32(0, 0, &mydata);
if (LOG) {
std::ofstream file;
file.open(fileName,ios_base::app);
file << "!!! returned mydata: " << mydata << ",&mydata: " << &mydata << endl;
file.close();
}
}
and calls ReadReg32 below:
static uint32 ReadReg32(uint32 brdNum, uint32 address, uint32 *data)
{
std::ofstream file;
if (LOG) {
file.open(fileName,ios_base::app);
file << " ReadReg32: brdNum =" << brdNum << ", address =" << address << endl;
file << " ReadReg32 ... Data WAS: " << *data << endl;
file.close();
}
/* AD:TW Check if we need to do init */
bool initstatus = checkDeferredInit();
ADMXRC3_HANDLE phCard;
ADMXRC3_STATUS status;
//open card
if((status = ADMXRC3_Open(brdNum, &phCard)) != ADMXRC3_SUCCESS)
return status;
//read data
if((status = ADMXRC3_Read(phCard, NON_PREFETCHABLE_SPACE, 0, address, 4, &data)) != ADMXRC3_SUCCESS){
ADMXRC3_Close(phCard);
return status;
}
if (LOG) {
file.open(fileName,ios_base::app);
file << " ReadReg32 ... Data IS: " << data << ", &data: " << &data << ", Read status =" << status << endl;
file.close();
}
//close card
status = ADMXRC3_Close(phCard);
if (LOG) {
file.open(fileName,ios_base::app);
file << " ReadReg32 ... Close card status =" << status << ", Returning ..." << endl;
file.close();
}
return status;
}
The relevant output is this:
!!!mydata: 348,&mydata: 000000000012F050
ReadReg32: brdNum =0, address =0
ReadReg32 ... Data WAS: 348
checkDeferredInit
ReadReg32 ... Data IS: 0000000081040102, &data: 000000000012EF30, Read status =0
ReadReg32 ... Close card status =0, Returning ...
!!! returned mydata: 348,&mydata: 000000000012F050
where 0000000081040102 is the correct number i expect to see, but it never returns that to the calling function, i.e. mydata is never updated.
Interesting notes:
in ADMXRC3_Read, if I don't use &data, i get "incorrect" data values
in ADMXRC3_Read, the type for where i'm sticking in &data is
supposed to be void *pbuffer?
Any thoughts are worth like a case of beer, or a moderately fine boxed wine. ;)
I do appreciate any thoughts you have, this has me stumped...
Well at first glance it seems you are using & too much.
static uint32 ReadReg32(uint32 brdNum, uint32 address, uint32 *data)
{
....
//read data
if((status = ADMXRC3_Read(phCard, NON_PREFETCHABLE_SPACE, 0, address, 4, &data)) != ADMXRC3_SUCCESS){
ADMXRC3_Close(phCard);
return status;
}
Here in read data part, you are calling ADMXRC3_Read with &data param. But data is already pointer! So you actually pass adress of variable that is local to your function.
To fix this, call the ADMXRC3_Read with data only.
IMPORTANT: In C and C++ everything is passed by value! So the data is just an variable holding 32-bit number (if you have 32bit addresses), nothing more.
EDIT: In my note I was referring to this FAQ: http://c-faq.com/ptrs/passbyref.html and I wrote it wrong. See the FAQ for more info.
I have a problem where I try to compress a file's data. Everything works up to the compression call, but it isn't the compression call itself, as the segfault is thrown before it. Showing my code will make it much clearer:
std::cout << "FILENAME: ";
std::cin >> filename;
if(!fileExists(filename))
{
std::cout << "ERR: FILE NOT FOUND." << std::endl;
continue;
}
std::cout << "Compressing file data...";
writeFile(filename, zlib_compress(readFile(filename)));
std::cout << " Done." << std::endl;
At the function zlib_compress...
std::string zlib_compress(const std::string& str)
{
std::cout << "DEBUG" << std::endl;
z_stream zs; // z_stream is zlib's control structure
memset(&zs, 0, sizeof(zs));
if (deflateInit(&zs, 9) != Z_OK)
std::cout << "deflateInit failed while compressing." << std::endl;
zs.next_in = (Bytef*)str.data();
zs.avail_in = str.size(); // set the z_stream's input
int ret;
char outbuffer[1073741824];
std::string outstring;
// retrieve the compressed bytes blockwise
do
{
zs.next_out = reinterpret_cast<Bytef*>(outbuffer);
zs.avail_out = sizeof(outbuffer);
ret = deflate(&zs, Z_FINISH);
if (outstring.size() < zs.total_out)
{
// append the block to the output string
outstring.append(outbuffer, zs.total_out - outstring.size());
}
} while(ret == Z_OK);
deflateEnd(&zs);
if(ret != Z_STREAM_END) // an error occurred that was not EOF
{
std::ostringstream oss;
oss << "Exception during zlib compression: (" << ret << ") " << zs.msg;
std::cout << oss.str();
}
return outstring;
}
I know, I know, that function needs work, I just C&P'd from somewhere to try it out.
But the thing is this:
std::cout << "DEBUG" << std::endl; is never called. The compiler says that the seg fault is coming from here:
std::string zlib_compress(const std::string& str)
> {
But why...? It was working earlier. I just don't know what went wrong!
Edit: Debugger output.
#0 00000000 0x00402cbb in __chkstk_ms() (??:??)
#1 004013BE zlib_compress(str=...) (C:\Users\***\Documents\Work\Programming\Compressor\z.cpp:5)
#2 00401DDA _fu15___ZSt4cout() (C:\Users\***\Documents\Work\Programming\Compressor\main.cpp:80)
char outbuffer[1073741824];
That's too large to put on the stack
You are taking a constant reference to a string as a parameter in your zlib_compress - you need to make sure that memory is available (whatever is returned from your readfile) in your zlib_compress. It would be good if you can share the prototype of your readFile function too.