C++/C Virtual/Embeddable File System [Cross Compatible (Library)]? - c++
I want to experiment with some Virtual File Systems, like having 1 file in which all data is stored (1 file = virtual disk). For example I can open the virtual disk and put files into it, write data to the files, read data from file, open files and use fseek etc..
Is there any library I can use for that? License stuff etc is not important. I just want to test it on my machine because I'm borred, So I want to try this in C++/C.
Thanks in advance! :)
If the library is windows only then it's also okay, maybe I will find a linux library so I can make 2 projects?
Edit:
Thanks to BRPocock I know my question is a bit unclear. What I really want is a library which has the ability to store files, read files, and perform file operations on a VFS which the library already provides. And ofcourse mounting. So, What I would preffer is if there is a library which gives me this functions in C++:
OpenVirtualDrive(const name[]);//
CloseVirtualDrive(handle);//
fopen(const name[], mode);//open file, return handle
fclose(handle);//close handle
ftemp(); //open file as temporary file
fremove(const name[]);//delete file
fwrite(handle, array[]);//write array to file
fread(handle, array[], size = sizeof array, pack = false);//read from file
fputchar(handle, value, utf8 = true);//put char into file
fgetchar(handle, value, utf8 = true);//read char from file, move pointer
fblockwrite(handle, const buffer[], size = sizeof buffer);//write block
fblockread(handle, buffer[], size = sizeof buffer);//read block, move pointer
fseek(handle, position = 0, seek_whence: whence = seek_start);//move pointer
flength(handle);//size of file
fexist(const pattern[]);//check if file exists
fmatch(name[], const pattern[], index = 0, size = sizeof name);//search for file
This is just pseudo code :P
Linux (and many BSDs, including, I believe, MacOSX) uses the FUSE system (http://fuse.sourceforge.net/) to provide those kinds of services. You should be able to find many examples on the 'Net.
I believe the “accepted” way to do the same thing on Windows is to write it as a device-driver loadable module (.dll) … a quick Googling points at http://msdn.microsoft.com/en-us/windows/hardware/gg463062 as the starting-point, perhaps.
Our SolFS will do the job. Evaluation license will be ok for you and we offer free licenses for non-commercial public projects as well. If you want a filesystem visible from other applications, you need OS edition of SolFS, and if you are going to call its functions only from your application, then SolFS Application Edition is enough.
Related
Correct way to extract array data from binary?
There is a classic way to embed resource files as a C language array into a binary file, so that we can store some external resource files such as .jpeg or .txt files into a binary.
For example, in the header file we can define an array:
const unsigned char xd_data[] = {
77,90,144,0,3,0,0,0,4,0,0,0,255,255,0,0,184,0,0,0,0,0,0,0,64,0,0,0,0,
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,240,0,0,
0,14,31,186,14,0,180,9,205,33,184,1,76,205,33,84,104,105,115,32,112,114,
111,103,114,97,109,32,99,97,110,110,111,116,32,98,101,32,114,117,110,
32,105,110,32,68,79,83,32,109,111,100,101,46,13,13,10,36,0,0,0,0,0,0,
0,66,163,223,218,6,194,177,137,6,194,177,137,6,194,177,137,105,221,187,
137,13,194,177,137,133,222,191,137,3,194,177,137,105,221,181,137,4,194,
177,137,136,202,238,137,4,194,177,137,6,194,176,137,73,194,177,137,133,
202,236,137,13,194,177,137,48,228,187,137,11,194,177,137,193,196,183,
137,7,194,177,137,82,105,99,104,6,194,177,137,0,0,0,0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,0,0,0,80,69,0,0,76,1,4,0,65,162,32,86,0,0,0,0,0,0,0,
0,224,0,47,1,11,1,6,0,0,100,0,0,0,74,0,0,0,0,0,0,228,113,0,0,0,16,0,0,
0,128,0,0,0,0,64,0,0,16,0,0,0,2,0,0,4,0,0,0,0,0,0,0,4,0,0,0,0,0,0,0,0,
224,0,0,0,4,0,0,0,0,0,0,2,0,0,0,0,0,16,0,0,16,0,0,0,0,16,0,0,16,0,0,0,
0,0,0,16,0,0,0,0,0,0,0,0,0,0,0,124,140,0,0,140,0,0,0,0,208,0,0,0,16,0
};
which contains the contents of the resource file, and it will be compile into the final binary.
There are lots of tools and tutorials on the web about this old trick, such as: http://www.rowleydownload.co.uk/arm/documentation/index.htm?http://www.rowleydownload.co.uk/arm/documentation/embed.htm, https://www.fourmilab.ch/xd/ and http://gareus.org/wiki/embedding_resources_in_executables#c_include_method.
However, looks like most of these pages are talking about how to embed the data into binary file using C style array.
My question is, what is the correct way to find the start address of the resource files in the compiled binary in order to extract them? I.e., how can I find the start address of xd_data in the compiled binary?
If you mean finding the byte address in the file where a data block starts just like objdump does but programmatically, then you can use the Binary File Descriptor library (BFD), see here and here.
if you stored data for example an image and you want to load it (for printing or what ever you want) then if you have a function (library) that load it from memory, as example void loadResImage(void * mem); just do loadResImage(xd_data), if not but you have a function that load it from the file, in that case save it to a temp file eg:
int fd=open("tmpfile");
int ret=write(fd,xd_data, sizeof(xd_data));
close(fd);
loadImageFile("tmpfile");
but if you want to access the data outside the program itself (hex editor for example, or an other program), in that case you have to add a starting mark and optionally an ending mark or sizeof data. eg:
const unsigned char xd_data[]={
...
'M','A','G','I','C'};
in example above the end of the data is known, you just do a search to find it. same way, play around and find a suitable way to store the size of the data. but beware of the compiler optimizations.
Recovering Files on Windows and C
Well this time I'm trying to write a program in C which recover deleted files from a disk, it could be an external disk, I have an idea than i had used before on linux, it is to open the disk as a kind of file and scaning the Headers and file footers of everything within the disk, the point is I'm not sure if there's allow on windows to open a disk as an File, basiclly I have the logic how to develope this program, but I'm not sure how to implement it on windows, anybody can give me a hand with this?. The code I used on linux to open a disk as a file was: Edit: That was a sample of what I was using guys, it's just to give you an idea of what I was doing, the correct syntax I used was the next: direccion = ui->linea->text().toLatin1().constData(); f = fopen(direccion,"rb"); I used QT creator on linux, and direccion variable was a TextField value which contained the file path of the disk through a button function that open a QFileDialog... could I use it in windows as well? Thank you before hand..
"The code I used on linux to open a disk as a file was:"
File *f = fopen("E:\", "rb");
I seriously doubt you ever got this code working on any linux system (or windows either).
You'll need to escape the backslash path delimiter, if it's presented in any string literal:
FILE* f = fopen("E:\\", "rb");
// ^
Also all that filesystem path style you are presenting to access a particular disk, is about accessing a windows file path/disk.
No linux file system has notion about drive characters, and the file path delimiter value used is '/', not '\\'.
To recover deleted files, you can't use fopen or fstream::open because the file was deleted. Check the return value from the function or test the stream state. The way to recover deleted files is: Get the Master File Table as raw data. Search for the record containing a string similar to the deleted filename. Change the entry in the Master File Table to "undeleted". Write the Master File Table back to the drive. The above usually requires platform specific API, which is different on Linux and Windows platforms.
iOS file size during write using only C/C++ APIs
Purpose: I am monitoring file writes in a particular directory on iOS using BSD kernel queues, and poll for file sizes to determine write ends (when the size stops changing). The basic idea is to refresh a folder only after any number of file copies coming from iTunes sync. I have a completely working Objective-C implementation for this but I have my reasons for needing to implement the same thing in C++ only. Problem: The one thing stopping me is that I can't find a C or C++ API that will get the correct file size during a write. Presumably, one must exist because Objective-C's [NSFileManager attributesOfItemAtPath:] seems to work and we all know it is just calling a C API underneath. Failed Solutions: I have tried using stat() and lstat() to get st_size and even st_blocks for allocated block count, and they return correct sizes for most files in a directory, but when there is a file write happening that file's size never changes between poll intervals, and every subsequent file iterated in that directory have a bad size. I have tried using fseek and ftell but they are also resulting in a very similar issue. I have also tried modified date instead of size using stat() and st_mtimespec, and the date doesn't appear to change during a write - not that I expected it to. Going back to NSFileManager's ability to give me the right values, does anyone have an idea what C API call that [NSFileManager attributesOfItemAtPath:] is actually using underneath? Thanks in advance. Update: It appears that this has less to do with in-progress write operations and more with specific files. After closer inspection there are some files which always return a size, and other files that never return a size when using the C API (but will work fine with the Objective-C API). Even creating a copy of the "good" files the C API does not want to give a size for the copy but works fine with the original "good" file. I have both failures and successes with text (xml) files and binary (zip) files. I am using iTunes to add these files to the iPad's app's Documents directory. It is an iPad Mini Retina. Update 2 - Answer: Probably any of the above file size methods will work, if your path isn't invisibly trashed, like mine was. See accepted answer on why the path was trashed.
Well this weird behavior turned out to be a problem with the paths, which result in strings that will print normally, but are likely trashed in memory enough that file descriptors sometimes didn't like it (thus only occurring in certain file paths). I was using the dirent API to iterate over the files in a directory and concatenating the dir path and file name erroneously. Bad Path Concatenation: Obviously (or apparently not-so-obvious at runtime) str-copying over three times is not going to end well. char* fullPath = (char*)malloc(strlen(dir) + strlen(file) + 2); strcpy(fullPath, dir); strcpy(fullPath, "/"); strcpy(fullPath, file); long sizeBytes = getSize(fullPath); free(fullPath); Correct Path Concatenation: Use proper str-concatenation. char* fullPath = (char*)malloc(strlen(dir) + strlen(file) + 2); strcpy(fullPath, dir); strcat(fullPath, "/"); strcat(fullPath, file); long sizeBytes = getSize(fullPath); free(fullPath); Long story short, it was sloppy work on my part, via two typos.
Is it possible to store binary files inside an exe
Is it possible to do this: (for educational purpose). suppose I have a image file "image.jpg" I want to create a program when it executes it should create this image. That means the data of the image is stored in the exe. Is this possible to do? Something like this: link the image file from resource.rc then tell the compiler to get the data and store it (something like this unsigned char data_buffer[]="binary data of the image" then when the program is executed I can write this data to a file) (I'm using C++ with mingw compiler) Any help is highly appreciated.
There are several options:
1) Add it as a byte array in a source file. It is trivial to write an auxiliary program that reads the bytes from the files and writes the C source. E.g.:
data_jpg.c:
unsigned char data_jpg[] = {1,2,3... };
data_jpg.h:
extern char data_jpg[];
const size_t data_jpg_size = 1000;
2) Add it as a binary resource to the executable. You said "exe", did you? So you are likely on Windows. Window EXE files can have binary resources, that can be located using the resource API. See the FindResource, LoadResource and GlobalLock, functions.
resource.rc
ID_DATA_JPG FILE "data.jpg"
3) Convert the binary file directly into a OBJ file and link it into the executable. In the old good days of turbo-c used to be a BINOBJ tool for that. And GNU tools can do it, AFAIk, but with MS tools, I really cannot tell.
With a PE file, you can add data(include bin data) to the PE file's tail as your resource. You just remember the PE file's size. But I'm not sure of that whether you need change the PE's checksum. And use VC++ Compiler to embed resources would be pretty much easy.
Embedding a file into a program
I want to embed a file in a program. It will be used for default configuration files if none are provided. I have realized I could just use default values, but I want to extract the file and place it on the disk so it can be modified.
By embedding do you mean distributing your program without the file? Then you can convert it to configuration initialization code in your build toolchain. Add a makefile step (or whatever tool you're using) - a script that converts this .cfg file into some C++ code file that initializes a configuration data structure. That way you can just modify the .cfg file, rebuild the project, and have the new values reflected inside. By the way, on Windows, you may have luck embedding your data in a resource file.
One common thing you can do is to represent the file data as an array of static bytes:
// In a header file:
extern const char file_data[];
extern const size_t file_data_size;
// In a source file:
const char file_data[] = {0x41, 0x42, ... }; // etc.
const size_t file_data_size = sizeof(file_data);
Then the file data will just be a global array of bytes compiled into your executable that you can reference anywhere. You'll have to either rewrite your file processing code to be able to handle a raw byte array, or use something like fmemopen(3) to open a pseudo-file handle from the data and pass that on to your file handling code.
Of course, to get the data into this form, you'll need to use some sort of preprocessing step to convert the file into a byte array that the compiler can accept. A makefile would be good for this.
Embedded data are often called "resources". C++ provides no native support, but it can be managed in almost all executable file formats. Try searching for resource managers for c++.
If it's any Unix look into mapping the file into process memory with mmap(2). Windows has something similar but I never played with it.