Embedding a file into a program - c++
I want to embed a file in a program. It will be used for default configuration files if none are provided. I have realized I could just use default values, but I want to extract the file and place it on the disk so it can be modified.
By embedding do you mean distributing your program without the file?
Then you can convert it to configuration initialization code in your build toolchain. Add a makefile step (or whatever tool you're using) - a script that converts this .cfg file into some C++ code file that initializes a configuration data structure. That way you can just modify the .cfg file, rebuild the project, and have the new values reflected inside.
By the way, on Windows, you may have luck embedding your data in a resource file.
One common thing you can do is to represent the file data as an array of static bytes:
// In a header file:
extern const char file_data[];
extern const size_t file_data_size;
// In a source file:
const char file_data[] = {0x41, 0x42, ... }; // etc.
const size_t file_data_size = sizeof(file_data);
Then the file data will just be a global array of bytes compiled into your executable that you can reference anywhere. You'll have to either rewrite your file processing code to be able to handle a raw byte array, or use something like fmemopen(3) to open a pseudo-file handle from the data and pass that on to your file handling code.
Of course, to get the data into this form, you'll need to use some sort of preprocessing step to convert the file into a byte array that the compiler can accept. A makefile would be good for this.
Embedded data are often called "resources". C++ provides no native support, but it can be managed in almost all executable file formats. Try searching for resource managers for c++.
If it's any Unix look into mapping the file into process memory with mmap(2). Windows has something similar but I never played with it.
Related
Correct way to extract array data from binary?
There is a classic way to embed resource files as a C language array into a binary file, so that we can store some external resource files such as .jpeg or .txt files into a binary.
For example, in the header file we can define an array:
const unsigned char xd_data[] = {
77,90,144,0,3,0,0,0,4,0,0,0,255,255,0,0,184,0,0,0,0,0,0,0,64,0,0,0,0,
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,240,0,0,
0,14,31,186,14,0,180,9,205,33,184,1,76,205,33,84,104,105,115,32,112,114,
111,103,114,97,109,32,99,97,110,110,111,116,32,98,101,32,114,117,110,
32,105,110,32,68,79,83,32,109,111,100,101,46,13,13,10,36,0,0,0,0,0,0,
0,66,163,223,218,6,194,177,137,6,194,177,137,6,194,177,137,105,221,187,
137,13,194,177,137,133,222,191,137,3,194,177,137,105,221,181,137,4,194,
177,137,136,202,238,137,4,194,177,137,6,194,176,137,73,194,177,137,133,
202,236,137,13,194,177,137,48,228,187,137,11,194,177,137,193,196,183,
137,7,194,177,137,82,105,99,104,6,194,177,137,0,0,0,0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,0,0,0,80,69,0,0,76,1,4,0,65,162,32,86,0,0,0,0,0,0,0,
0,224,0,47,1,11,1,6,0,0,100,0,0,0,74,0,0,0,0,0,0,228,113,0,0,0,16,0,0,
0,128,0,0,0,0,64,0,0,16,0,0,0,2,0,0,4,0,0,0,0,0,0,0,4,0,0,0,0,0,0,0,0,
224,0,0,0,4,0,0,0,0,0,0,2,0,0,0,0,0,16,0,0,16,0,0,0,0,16,0,0,16,0,0,0,
0,0,0,16,0,0,0,0,0,0,0,0,0,0,0,124,140,0,0,140,0,0,0,0,208,0,0,0,16,0
};
which contains the contents of the resource file, and it will be compile into the final binary.
There are lots of tools and tutorials on the web about this old trick, such as: http://www.rowleydownload.co.uk/arm/documentation/index.htm?http://www.rowleydownload.co.uk/arm/documentation/embed.htm, https://www.fourmilab.ch/xd/ and http://gareus.org/wiki/embedding_resources_in_executables#c_include_method.
However, looks like most of these pages are talking about how to embed the data into binary file using C style array.
My question is, what is the correct way to find the start address of the resource files in the compiled binary in order to extract them? I.e., how can I find the start address of xd_data in the compiled binary?
If you mean finding the byte address in the file where a data block starts just like objdump does but programmatically, then you can use the Binary File Descriptor library (BFD), see here and here.
if you stored data for example an image and you want to load it (for printing or what ever you want) then if you have a function (library) that load it from memory, as example void loadResImage(void * mem); just do loadResImage(xd_data), if not but you have a function that load it from the file, in that case save it to a temp file eg:
int fd=open("tmpfile");
int ret=write(fd,xd_data, sizeof(xd_data));
close(fd);
loadImageFile("tmpfile");
but if you want to access the data outside the program itself (hex editor for example, or an other program), in that case you have to add a starting mark and optionally an ending mark or sizeof data. eg:
const unsigned char xd_data[]={
...
'M','A','G','I','C'};
in example above the end of the data is known, you just do a search to find it. same way, play around and find a suitable way to store the size of the data. but beware of the compiler optimizations.
Is it possible to store binary files inside an exe
Is it possible to do this: (for educational purpose). suppose I have a image file "image.jpg" I want to create a program when it executes it should create this image. That means the data of the image is stored in the exe. Is this possible to do? Something like this: link the image file from resource.rc then tell the compiler to get the data and store it (something like this unsigned char data_buffer[]="binary data of the image" then when the program is executed I can write this data to a file) (I'm using C++ with mingw compiler) Any help is highly appreciated.
There are several options:
1) Add it as a byte array in a source file. It is trivial to write an auxiliary program that reads the bytes from the files and writes the C source. E.g.:
data_jpg.c:
unsigned char data_jpg[] = {1,2,3... };
data_jpg.h:
extern char data_jpg[];
const size_t data_jpg_size = 1000;
2) Add it as a binary resource to the executable. You said "exe", did you? So you are likely on Windows. Window EXE files can have binary resources, that can be located using the resource API. See the FindResource, LoadResource and GlobalLock, functions.
resource.rc
ID_DATA_JPG FILE "data.jpg"
3) Convert the binary file directly into a OBJ file and link it into the executable. In the old good days of turbo-c used to be a BINOBJ tool for that. And GNU tools can do it, AFAIk, but with MS tools, I really cannot tell.
With a PE file, you can add data(include bin data) to the PE file's tail as your resource. You just remember the PE file's size. But I'm not sure of that whether you need change the PE's checksum. And use VC++ Compiler to embed resources would be pretty much easy.
Accessing text data files from a static library function
How to I enable a static library to pull in data available in ascii data files? I am trying to add a model to a simulation as a library which contains functions that read data from data files. I am able to compile and run the functions from a main program outside the actual full simulation, but once I put the functions as a library on the host for the simulation the data no longer gets read. As the path to the data is changing depending on the user, I cannot provide an absolute data path to the ascii data files. Is there a way to use objcopy to make the data files into object code in the library or how can I best access the data from my static library?
There are several solutions to open a file that has an unknown location at compile time. Prompt the user for the name of the file, including directory. Use an environment variable to designate the directory containing the file ... Fortran 2003 has an intrinsic to obtain the value of an environment variable. Obtain the information from a command line argument ... again Fortran 2003 has an intrinsic for this purpose. With all of these, construct the filename as a string variable and provide that variable to the FILE keyword of the OPEN statement.
I don't know why you inclouded the Fortran tag, but in Fortran you: tell the code to open a file you want using a character string to read from it and to close it There is no difference between a main program or a library.
If you have a function like, say:
void read_data_from_files() { ... }
You'll need to change it in the DLL to be more like:
DataObject read_data_from_file(const char* file_path) { ... }
And then call it appropriately.
You'll need to design DataObject.
C++/C Virtual/Embeddable File System [Cross Compatible (Library)]?
I want to experiment with some Virtual File Systems, like having 1 file in which all data is stored (1 file = virtual disk). For example I can open the virtual disk and put files into it, write data to the files, read data from file, open files and use fseek etc.. Is there any library I can use for that? License stuff etc is not important. I just want to test it on my machine because I'm borred, So I want to try this in C++/C. Thanks in advance! :) If the library is windows only then it's also okay, maybe I will find a linux library so I can make 2 projects? Edit: Thanks to BRPocock I know my question is a bit unclear. What I really want is a library which has the ability to store files, read files, and perform file operations on a VFS which the library already provides. And ofcourse mounting. So, What I would preffer is if there is a library which gives me this functions in C++: OpenVirtualDrive(const name[]);// CloseVirtualDrive(handle);// fopen(const name[], mode);//open file, return handle fclose(handle);//close handle ftemp(); //open file as temporary file fremove(const name[]);//delete file fwrite(handle, array[]);//write array to file fread(handle, array[], size = sizeof array, pack = false);//read from file fputchar(handle, value, utf8 = true);//put char into file fgetchar(handle, value, utf8 = true);//read char from file, move pointer fblockwrite(handle, const buffer[], size = sizeof buffer);//write block fblockread(handle, buffer[], size = sizeof buffer);//read block, move pointer fseek(handle, position = 0, seek_whence: whence = seek_start);//move pointer flength(handle);//size of file fexist(const pattern[]);//check if file exists fmatch(name[], const pattern[], index = 0, size = sizeof name);//search for file This is just pseudo code :P
Linux (and many BSDs, including, I believe, MacOSX) uses the FUSE system (http://fuse.sourceforge.net/) to provide those kinds of services. You should be able to find many examples on the 'Net. I believe the “accepted” way to do the same thing on Windows is to write it as a device-driver loadable module (.dll) … a quick Googling points at http://msdn.microsoft.com/en-us/windows/hardware/gg463062 as the starting-point, perhaps.
Our SolFS will do the job. Evaluation license will be ok for you and we offer free licenses for non-commercial public projects as well. If you want a filesystem visible from other applications, you need OS edition of SolFS, and if you are going to call its functions only from your application, then SolFS Application Edition is enough.
How do you include images as resources in a C++ executable?
Is it possible include images (jpegs) as resources in a win32 c++ executable? If so how?
If it's Windows only then use a custom resource. If you want something cross-platform then do what I did for a recent project - create an app that will encode the JPEG as a char* buffer in a header file and then include these headers in your main project. You will also need to store the size of the buffer as it will be sure to contain NULs.
For example, I have an app that you can pass a load of files to be encoded and for each file you get a header file that looks something like this:
#ifndef RESOURCE_SOMEFILE_JPG_HPP
#define RESOURCE_SOMEFILE_JPG_HPP
namespace resource {
const char* SOMEFILE_JPG[] =
{
...raw jpeg data...
};
const int SOMEFILE_JPG_LEN = 1234;
} // resource
#endif // RESOURCE_SOMEFILE_JPG_HPP
The app has to escape special non-printable chars in \x format, but it's pretty simple. The app uses the boost::program_options library so a list of files to encode can be stored in a config file. Each file gets its own header like similar to the above.
However, be warned - this only works for small files as some compilers have a limit on the maximum size a static char buffer can be. I'm sure there are other ways to do this but this scheme works for me (a C++ web app that stores the HTML, CSS, JavaScript and image files in this way).
Here's the MSDN documentation about resource files. http://msdn.microsoft.com/en-us/library/aa380599(VS.85).aspx
Maybe this is what you're looking for?