Related
I have a small demo executable wrote in C++ that depends only on one 5kb PNG image being loaded before it can run, which is used for a pixel text I made. Because of this one file, I would need to give out a ZIP archive instead of just one executable file, which creates enough friction between download and 'play' that I believe would dissuade some from trying it out.
My question is, is there anyway to embed the PNG file (and any other file really) into the Executable or source code so that it is a single file, and the executable can use it?
I have the ability to parse the PNG as a byte stream, so it does not need converted to pixel data.
Thanks in advance! (Other questions with a similar title to this exist, but they and their answers seem to get into more specific issues and weren't very helpful)
edit:The compiler is Visual C++ 2010 and this is on Windows (though I would want to avoid windows specific utilities for this)
edit2: Alf's answer seemed like the most portable method, so I quickly wrote a function to parse the PNG file into a TXT or header file that could be read as a unsigned char array. It appears to be identical in this form to the PNG file itself, but my png loader won't accept the array. When loading it from memory, the PNG parser takes a (void * buffer, size_t length) if it matters.
The code if you wanted to see, but I'll still accept other answers if you think they're better than this method:
void compileImagePNGtoBinary(char * filename, char * output){
FILE * file = fopen(filename, "rb");
FILE * out = fopen(output, "w");
unsigned char buffer[32];
size_t count;
fprintf(out, "#pragma once \n\n static unsigned char TEXT_PNG_BYTES[] = { ");
while(!feof(file)){
count = fread(buffer, 1, 32, file);
for(int n = 0; n < count; ++n){
fprintf(out, "0x%02X, ", buffer[n]);
};
};
fprintf(out, "};");
fclose(file);
fclose(out);
};
Final Edit: ImageMagick which Alf also mentioned did exactly what I needed of it, thanks!
A portable way is to define a function like
typedef unsigned char Byte;
Byte const* pngFileData()
{
static Byte const data =
{
// Byte data generated by a helper program.
};
return data;
}
Then all you have to do is to write a little helper program that reads the PNG file as binary and generates the C++ curly braces initializer text. Edit: #awoodland has pointed out in comment to the question, that ImageMagick has such a little helper program…
Of course, for a Windows-specific program, instead use the ordinary Windows resource scheme.
Cheers & hth.,
Look at XD:
http://www.fourmilab.ch/xd/
Finally, xd can read a binary file and emit a C language data
declaration which contains the data from the file. This is handy when
you wish to embed binary data within C programs.
Personally, I'd use resources for windows, but if you require a truly portable way that doesn't involve knowledge of the executable format, this is the way to go. PNG, JPG, whatever...
Base64 encode the file and put it in a string somewhere in your code ;)
You can embed any arbitrary file into your program resources: (MSDN) User-Defined Resource.
A user-defined resource-definition statement defines a resource that contains application-specific data. The data can have any format and can be defined either as the content of a given file (if the filename parameter is given) or as a series of numbers and strings (if the raw-data block is specified).
nameID typeID filename
The filename specifies the name of a file containing the binary data of the resource. The contents of the file are included as the resource. RC does not interpret the binary data in any way. It is the programmer's responsibility to ensure that the data is properly aligned for the target computer architecture.
Once you've done that you can use the LoadResource function to access the bytes contained in the file.
This is executable-format dependent, which means inherently operating system/compiler dependent. Windows offers the Resources system for this as mentioned in this question.
On linux I use this. It's based off a few examples I found when trying to do some 4k demos, albeit modified a bit. I believe it can work on windows too, but not with the default VS inline assembly. My workaround is #defining a macro to either use this code or the windows resource system that #MarkRansom suggests (quite painful to get working, but does work eventually).
//USAGE: call BINDATA(name, file.txt) and access the char array &name.
#ifndef EMBED_DATA_H
#define EMBED_DATA_H
#ifdef _WIN32
//#error The VS ASM compiler won't work with this, but you can get external ones to do the trick
#define BINDATA #error BINDATA requires nasm
#else
__asm__(
".altmacro\n" \
".macro binfile p q\n" \
" .global \\p\n" \
"\\p:\n" \
" .incbin \\q\n" \
"\\p&_end:\n" \
" .byte 0\n" \
" .global \\p&_len\n" \
"\\p&_len:\n" \
" .int(\\p&_end - \\p)\n" \
".endm\n\t"
);
#ifdef __cplusplus
extern "C" {
#endif
#define BINDATA(n, s) \
__asm__("\n\n.data\n\tbinfile " #n " \"" #s "\"\n"); \
extern char n; \
extern int n##_len;
#ifdef __cplusplus
}
#endif
#endif
#endif
If I want to embed static data into an executable, I would package it into a .lib/.a file or a header file as an array of unsigned chars. That's if you are looking for a portable approach.
I have created a command line tool that does both actually here. All you have to do is list files, and pick option -l64 to output a 64bit library file along with a header that includes all pointers to each data.
You can explore more options as well.for example, this option:
>BinPack image.png -j -hx
will output the data of image.png into a header file, as hexadecimal and lines will be justified per -j option.
const unsigned char BP_icon[] = {
0x89,0x50,0x4e,0x47,0x0d,0x0a,0x1a,0x0a,0x00,0x00,0x00,0x0d,0x49,0x48,0x44,0x52,
0x00,0x00,0x01,0xed,0x00,0x00,0x01,0xed,0x08,0x06,0x00,0x00,0x00,0x34,0xb4,0x26,
0xfb,0x00,0x00,0x02,0xf1,0x7a,0x54,0x58,0x74,0x52,0x61,0x77,0x20,0x70,0x72,0x6f,
0x66,0x69,0x6c,0x65,0x20,0x74,0x79,0x70,0x65,0x20,0x65,0x78,0x69,0x66,0x00,0x00,
0x78,0xda,0xed,0x96,0x5d,0x92,0xe3,0x2a,0x0c,0x85,0xdf,0x59,0xc5,0x2c,0x01,0x49,
0x08,0x89,0xe5,0x60,0x7e,0xaa,0xee,0x0e,0xee,0xf2,0xef,0x01,0x3b,0x9e,0x4e,0xba,
0xbb,0x6a,0xa6,0x66,0x5e,0x6e,0x55,0x4c,0x8c,0x88,0x0c,0x07,0xd0,0x27,0x93,0x84,
0xf1,0xef,0x3f,0x33,0xfc,0xc0,0x45,0xc5,0x52,0x48,0x6a,0x9e,0x4b,0xce,0x11,0x57,
0x2a,0xa9,0x70,0x45,0xc3,0xe3,0x79,0xd5,0x5d,0x53,0x4c,0xbb,0xde,0xd7,0xe8,0x57,
0x8b,0x9e,0xfd,0xe1,0x7e,0xc0,0xb0,0x02,0x2b,0xe7,0x03,0xcf,0xa7,0xa5,0x87,0xff,
0x1a,0xf0,0xb0,0x54,0xd1,0xd2,0x0f,0x42,0xde,0xae,0x07,0xc7,0xf3,0x83,0x92,0x4e,
0xcb,0xfe,0x22,0xc4,0xa7,0x91,0xb5,0xa2,0xd5,0xee,0x97,0x50,0xb9,0x84,0x84,0xcf,
0x07,0x74,0x09,0xd4,0x73,0x5b,0x31,0x17,0xb7,0x8f,0x5b,0x38,0xc6,0x69,0xaf}
I came here looking for a bash script, so that I can generate the C array of bytes in a mostly-cross-platform compatible way (I depend on mingw bash for my windows builds anyway) without having to compile a helper tool or depend on any tools that don't come standard with a normal bash shell. Here's my take:
#!/bin/sh
set -e
SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"
OUT_FILE="$SCRIPT_DIR/src/alloverse_binary_schema.h"
BINARY_FILE="$SCRIPT_DIR/include/allonet/schema/alloverse.bfbs"
VAR_NAME="alloverse_schema"
echo "static const unsigned char ${VAR_NAME}_bytes[] = {" > "$OUT_FILE"
hexdump -ve '1/1 "0x%02x, "' "$BINARY_FILE" >> "$OUT_FILE"
echo "0x00}; static const int ${VAR_NAME}_size = sizeof(${VAR_NAME}_bytes); " >> "$OUT_FILE"
I can then just #include this from the C file where I use it, and use foo_bytes and foo_size as needed:
#include "alloverse_binary_schema.h"
bool allo_initialize(void)
{
g_alloschema = reflection_Schema_as_root(alloverse_schema_bytes);
}
This script should be adaptable to your needs by adjusting OUT_FILE, BINARY_FILE and VAR_NAME (perhaps taking them as arguments to the script).
On my home Linux laptop, I like to write wrapper programs and GUI helpers for things I use frequently. However, I don't like Bash scripting very much, so I do a lot of stuff in C++. However, a lot of times, this requires me to use the system() function from the cstdlib.
This system() command is awesome, but I wanted a way to call system() and receive the stdout/stderror. The system() command only returns the return code from the command. So, in a Bash script, one can do:
myVar=$(ls -a | grep 'search string')
echo $myVar
and myVar will output whatever the stdout was for the command. So I began writing a wrapper class that will add a pipe-to-file to the end of the command, open the file, read all of the piped stdout, and return it as either one long string or as a vector of strings. The intricacies of the class are not really relevant here (I don't think anyway), but the above example would be done like this:
SystemCommand systemCommand;
systemCommand.setCommand("ls -a | grep \'search string\' ");
systemCommand.execute();
std::cout << systemCommand.outputAsString() << std::endl;
Behind the scenes, when systemCommand.execute() is called, the class ensures that the command will properly pipe all stdout/stderr to a randomly generated filename, in the current working directory. So for example, the above command would end up being
"ls -a | grep 'search string' >> 1452-24566.txt 2>&1".
The class then goes attempts to open and read from that file, using ifstream:
std::ifstream readFromFile;
readFromFile.open(_outputFilename);
if (readFromFile.is_open()) {
//Read all contents of file into class member vector
...
readFromFile.close();
//Remove temporary file
...
} else {
//Handle read failure
}
So here is my main question will std::ifstream ever fail to open a recently created file in the current working directory? If so, what would be a way to make it more robust (specifically on Linux)?
A side/secondary question: Is there a very simplified way to achieve what I'm trying to achieve without using file pipes? Perhaps some stuff available in unistd.h? Thanks for your time.
So here is my main question will std::ifstream ever fail to open a recently created file in the current working directory?
Yes.
Mount a USB thumb drive (or some other removable media)
cd to the mount
Execute your program. While it's executing, remove the drive.
Watch the IO error happen.
There's a ton of other reasons too. Filesystem corruption, hitting the file descriptor limit, etc.
If so, what would be a way to make it more robust (specifically on Linux)?
Make temporary files in /tmp, whose entire purpose is for temporary files. Or don't create a file at all, and use pipes for communication instead (Like what popen does, like harmic suggested). Even so, there are no guarantees; try to gracefully handle errors.
I am trying to get my C++ program to open up an existing Excel spreadsheet (along with a bunch of applications), however it keeps returning an error that a file does not exist. I am using the following code:
int main(){
system("open ~/path/file");
//--open applications using same command--//
}
The file is definitely there and this command works to open all the applications, so I'm not sure what I am doing wrong.
Thanks in advance!!
Very probably, the system /bin/sh -which by definition is used by system(3)- does not expand ~.
You might try something like
char cmd[256];
snprintf(cmd, sizeof(cmd), "open %s/path/file", getenv("HOME"));
if (0 != system(cmd))
{ fprintf(stderr, "%s failed\n", cmd); exit(EXIT_FAILURE); };
since interactive shells usually expand ~ as $HOME and HOME is generally an environment variable.
(With C++, you could use std::string operations instead of snprintf)
My snprintf + system trick is not at all failproof. If $HOME contains spaces or bizarre characters like ; or ', it wont work. And snprintf itself might fail (e.g. because $HOME is huge).
Of course, you'll better test before that getenv("HOME") is not NULL. You might use getpwuid(3) with getuid(2) if getenv("HOME") fails by returning NULL.
On Linux you probably want xdg-open instead of open.
I have a small demo executable wrote in C++ that depends only on one 5kb PNG image being loaded before it can run, which is used for a pixel text I made. Because of this one file, I would need to give out a ZIP archive instead of just one executable file, which creates enough friction between download and 'play' that I believe would dissuade some from trying it out.
My question is, is there anyway to embed the PNG file (and any other file really) into the Executable or source code so that it is a single file, and the executable can use it?
I have the ability to parse the PNG as a byte stream, so it does not need converted to pixel data.
Thanks in advance! (Other questions with a similar title to this exist, but they and their answers seem to get into more specific issues and weren't very helpful)
edit:The compiler is Visual C++ 2010 and this is on Windows (though I would want to avoid windows specific utilities for this)
edit2: Alf's answer seemed like the most portable method, so I quickly wrote a function to parse the PNG file into a TXT or header file that could be read as a unsigned char array. It appears to be identical in this form to the PNG file itself, but my png loader won't accept the array. When loading it from memory, the PNG parser takes a (void * buffer, size_t length) if it matters.
The code if you wanted to see, but I'll still accept other answers if you think they're better than this method:
void compileImagePNGtoBinary(char * filename, char * output){
FILE * file = fopen(filename, "rb");
FILE * out = fopen(output, "w");
unsigned char buffer[32];
size_t count;
fprintf(out, "#pragma once \n\n static unsigned char TEXT_PNG_BYTES[] = { ");
while(!feof(file)){
count = fread(buffer, 1, 32, file);
for(int n = 0; n < count; ++n){
fprintf(out, "0x%02X, ", buffer[n]);
};
};
fprintf(out, "};");
fclose(file);
fclose(out);
};
Final Edit: ImageMagick which Alf also mentioned did exactly what I needed of it, thanks!
A portable way is to define a function like
typedef unsigned char Byte;
Byte const* pngFileData()
{
static Byte const data =
{
// Byte data generated by a helper program.
};
return data;
}
Then all you have to do is to write a little helper program that reads the PNG file as binary and generates the C++ curly braces initializer text. Edit: #awoodland has pointed out in comment to the question, that ImageMagick has such a little helper program…
Of course, for a Windows-specific program, instead use the ordinary Windows resource scheme.
Cheers & hth.,
Look at XD:
http://www.fourmilab.ch/xd/
Finally, xd can read a binary file and emit a C language data
declaration which contains the data from the file. This is handy when
you wish to embed binary data within C programs.
Personally, I'd use resources for windows, but if you require a truly portable way that doesn't involve knowledge of the executable format, this is the way to go. PNG, JPG, whatever...
Base64 encode the file and put it in a string somewhere in your code ;)
You can embed any arbitrary file into your program resources: (MSDN) User-Defined Resource.
A user-defined resource-definition statement defines a resource that contains application-specific data. The data can have any format and can be defined either as the content of a given file (if the filename parameter is given) or as a series of numbers and strings (if the raw-data block is specified).
nameID typeID filename
The filename specifies the name of a file containing the binary data of the resource. The contents of the file are included as the resource. RC does not interpret the binary data in any way. It is the programmer's responsibility to ensure that the data is properly aligned for the target computer architecture.
Once you've done that you can use the LoadResource function to access the bytes contained in the file.
This is executable-format dependent, which means inherently operating system/compiler dependent. Windows offers the Resources system for this as mentioned in this question.
On linux I use this. It's based off a few examples I found when trying to do some 4k demos, albeit modified a bit. I believe it can work on windows too, but not with the default VS inline assembly. My workaround is #defining a macro to either use this code or the windows resource system that #MarkRansom suggests (quite painful to get working, but does work eventually).
//USAGE: call BINDATA(name, file.txt) and access the char array &name.
#ifndef EMBED_DATA_H
#define EMBED_DATA_H
#ifdef _WIN32
//#error The VS ASM compiler won't work with this, but you can get external ones to do the trick
#define BINDATA #error BINDATA requires nasm
#else
__asm__(
".altmacro\n" \
".macro binfile p q\n" \
" .global \\p\n" \
"\\p:\n" \
" .incbin \\q\n" \
"\\p&_end:\n" \
" .byte 0\n" \
" .global \\p&_len\n" \
"\\p&_len:\n" \
" .int(\\p&_end - \\p)\n" \
".endm\n\t"
);
#ifdef __cplusplus
extern "C" {
#endif
#define BINDATA(n, s) \
__asm__("\n\n.data\n\tbinfile " #n " \"" #s "\"\n"); \
extern char n; \
extern int n##_len;
#ifdef __cplusplus
}
#endif
#endif
#endif
If I want to embed static data into an executable, I would package it into a .lib/.a file or a header file as an array of unsigned chars. That's if you are looking for a portable approach.
I have created a command line tool that does both actually here. All you have to do is list files, and pick option -l64 to output a 64bit library file along with a header that includes all pointers to each data.
You can explore more options as well.for example, this option:
>BinPack image.png -j -hx
will output the data of image.png into a header file, as hexadecimal and lines will be justified per -j option.
const unsigned char BP_icon[] = {
0x89,0x50,0x4e,0x47,0x0d,0x0a,0x1a,0x0a,0x00,0x00,0x00,0x0d,0x49,0x48,0x44,0x52,
0x00,0x00,0x01,0xed,0x00,0x00,0x01,0xed,0x08,0x06,0x00,0x00,0x00,0x34,0xb4,0x26,
0xfb,0x00,0x00,0x02,0xf1,0x7a,0x54,0x58,0x74,0x52,0x61,0x77,0x20,0x70,0x72,0x6f,
0x66,0x69,0x6c,0x65,0x20,0x74,0x79,0x70,0x65,0x20,0x65,0x78,0x69,0x66,0x00,0x00,
0x78,0xda,0xed,0x96,0x5d,0x92,0xe3,0x2a,0x0c,0x85,0xdf,0x59,0xc5,0x2c,0x01,0x49,
0x08,0x89,0xe5,0x60,0x7e,0xaa,0xee,0x0e,0xee,0xf2,0xef,0x01,0x3b,0x9e,0x4e,0xba,
0xbb,0x6a,0xa6,0x66,0x5e,0x6e,0x55,0x4c,0x8c,0x88,0x0c,0x07,0xd0,0x27,0x93,0x84,
0xf1,0xef,0x3f,0x33,0xfc,0xc0,0x45,0xc5,0x52,0x48,0x6a,0x9e,0x4b,0xce,0x11,0x57,
0x2a,0xa9,0x70,0x45,0xc3,0xe3,0x79,0xd5,0x5d,0x53,0x4c,0xbb,0xde,0xd7,0xe8,0x57,
0x8b,0x9e,0xfd,0xe1,0x7e,0xc0,0xb0,0x02,0x2b,0xe7,0x03,0xcf,0xa7,0xa5,0x87,0xff,
0x1a,0xf0,0xb0,0x54,0xd1,0xd2,0x0f,0x42,0xde,0xae,0x07,0xc7,0xf3,0x83,0x92,0x4e,
0xcb,0xfe,0x22,0xc4,0xa7,0x91,0xb5,0xa2,0xd5,0xee,0x97,0x50,0xb9,0x84,0x84,0xcf,
0x07,0x74,0x09,0xd4,0x73,0x5b,0x31,0x17,0xb7,0x8f,0x5b,0x38,0xc6,0x69,0xaf}
I came here looking for a bash script, so that I can generate the C array of bytes in a mostly-cross-platform compatible way (I depend on mingw bash for my windows builds anyway) without having to compile a helper tool or depend on any tools that don't come standard with a normal bash shell. Here's my take:
#!/bin/sh
set -e
SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"
OUT_FILE="$SCRIPT_DIR/src/alloverse_binary_schema.h"
BINARY_FILE="$SCRIPT_DIR/include/allonet/schema/alloverse.bfbs"
VAR_NAME="alloverse_schema"
echo "static const unsigned char ${VAR_NAME}_bytes[] = {" > "$OUT_FILE"
hexdump -ve '1/1 "0x%02x, "' "$BINARY_FILE" >> "$OUT_FILE"
echo "0x00}; static const int ${VAR_NAME}_size = sizeof(${VAR_NAME}_bytes); " >> "$OUT_FILE"
I can then just #include this from the C file where I use it, and use foo_bytes and foo_size as needed:
#include "alloverse_binary_schema.h"
bool allo_initialize(void)
{
g_alloschema = reflection_Schema_as_root(alloverse_schema_bytes);
}
This script should be adaptable to your needs by adjusting OUT_FILE, BINARY_FILE and VAR_NAME (perhaps taking them as arguments to the script).
I have this windows console app which takes a file, do some calculations, and then writes the output to a specified file. The input is specified in "app.exe -input fullfilename" format. I need to call this application from my C++ program, but I have a problem with spaces in paths to files. When I call the app directly from cmd.exe by typing (without specifying output file for clarity)
"c:\first path\app.exe" -input "c:\second path\input.file"
everything works as expected. But, when I try using cstdlib std::system() function, i.e.
std::system(" \"c:\\first path\\app.exe\" -input \"c:\\second path\\input.file\" ");
the console prints out that c:\first is not any valid command. It's probably common mistake and has simple solution, but I have been unable to find any. Thx for any help.
Instead of std::system(), you should use the _wspawnv function from the Windows API. Use _wspawnvp if you want to search for the program in PATH, rather than specifying a full path to it.
#include <stdio.h>
#include <wchar.h>
...
const WCHAR *app = L"C:\\path to\\first app.exe";
const WCHAR *argv[] = {app, L"-input", L"c:\\second path\\input file.txt"};
_wpspawnv(_P_WAIT, app, argv);
You could also use _spawnv / _spawnvp if you are 100% sure that your input filename will never, ever contain anything else than ASCII.
Don't try to put the quotes in the std::system() call. Try the following:
std::system("c:\\first\\ path\\app.exe -input c:\\second\\ path\\input.file");