I have a binary file which i want to embed directly into my source code, so it will be compiled into the .exe file directly, instead of reading it from a file, so the data would already be in the memory when i launch the program.
How do i do this?
Only idea i got was to encode my binary data into base64, put it in a string variable and then decode it back to raw binary data, but this is tricky method which will cause pointless memory allocating. Also, i would like to store the data in the .exe as compact as the original data was.
Edit: The reason i thought of using base64 was because i wanted to make the source code files as small as possible too.
The easiest and most portable way would be to write a small
program which converts the data to a C++ source, then compile
that and link it into your program. This generated file might
look something like:
unsigned char rawData[] =
{
0x12, 0x34, // ...
};
There are tools for this, a typical name is "bin2c". The first search result is this page.
You need to make a char array, and preferably also make it static const.
In C:
Some care might be needed since you can't have a char-typed literal, and also because generally the signedness of C's char datatype is up to the implementation.
You might want to use a format such as
static const unsigned char my_data[] = { (unsigned char) 0xfeu, (unsigned char) 0xabu, /* ... */ };
Note that each unsigned int literal is cast to unsigned char, and also the 'u' suffix that makes them unsigned.
Since this question was for C++, where you can have a char-typed literal, you might consider using a format such as this, instead:
static const char my_data[] = { '\xfe', '\xab', /* ... */ };
since this is just an array of char, you could just as well use ordinary string literal syntax. Embedding zero-bytes should be fine, as long as you don't try to treat it as a string:
static const char my_data[] = "\xfe\xdab ...";
This is the most compact solution. In fact, you could probably use that for C, too.
You can use resource files (.rc). Sometimes they are bad, but for Windows based application that's the usual way.
Why base64? Just store the file as it is in one char*.
Related
I'm currently processing a binary file using C++...
At some point I read a byte in and the char * read is "\x3" which seems to be a control character.
But when i got to write it back out using:
const char *control = "\x3";
fout.write(control, sizeof(control));
And then i read the binary file back in the read value is "\x11C".
How does one write the control character array back out to file the correct way?
Your code is writing 4-8 characters to the binary file instead of the 1 you seem to be expecting. control is treated as a normal pointer, and sizeof(control) is interpreting said pointer without considering the data it points to, and is returning a value of 4-8.
The best way to fix this is to declare control as a single character, which is what you seem to intend:
char control = '\x3';
fout.write(&control, sizeof(control));
The other way, if you actually need to write multiple characters, is like this:
const std::string control = "\x3";
fout.write(control.data(), control.size());
Either method will correctly output the number of characters you expect.
Another method to write string literals, is by declaring them as an array:
static char const data[] = "Hello World!\n";
fout.write(data, sizeof(data) - 1U);
The - 1U is so that the terminating NUL is not written. Remove as you wish.
Since the data array is declared with no capacity, so the compiler determines the length based on the content.
The sizeof can be used since the size of a character is 1 (by definition).
A nice advantage of this method is that the size is known at compile time. No searching for the length is required.
I want to test serialized data conversion in my application, currently the object is stored in file and read the binary file and reloading the object.
In my unit test case I want to test this operation. As the file operations are costly I want to hard code the binary file content in the code itself.
How can I do this?
Currently I am trying like this,
std::string FileContent = "\00\00\00\00\00.........";
and it is not working.
You're right that a string can contain '\0', but here you're still initializing it from const char*, which, by definition, stops at the first '\0'. I'd recommend you to use uint8_t[] or even uint32_t[] (that is, without passing to std::string), even if the second might have up to 3 bytes of overhead (but it's more compact when in source). That's e.g. how X bitmaps are usually stored.
Another possibility is base64 encoding, which is printable but needs (a relatively quick) decoding.
If you really want to put the const char[] to a std::string, first convert the pointer to const char*, then use the two-iterator constructor of std::string. While it's true that std::string can hold '\0', it's somewhat an antipattern to store binary in a string, thus I'm not giving the exact code, just the hint.
The following should do what you need, however probably not recommended as most people wouldn't expect an std::string to contain null bytes.
std::string FileContent { "\x00\x00\x00\x00\x00", 5 };
So I've looked around for how to convert a string to a short and found a lot on how to convert a string to an integer. I would leave a question as a comment on those threads, but I don't have enough reputation. So, what I want to do is convert a string to a short, because the number should never go above three or below zero and shorts save memory (as far as I'm aware).
To be clear, I'm not referring to ASCII codes.
Another thing I want to be able to do is to check if the conversion of the string to the short fails, because I'll be using a string which consists of a users input.
I know I can do this with a while loop, but if there's a built in function to do this in C++ that would be just as, or more, efficient than a while loop, I would love to hear about it.
Basically, an std::stos function is missing for unknown reasons, but you can easily roll your own. Use std::stoi to convert to int, check value against short boundaries given by e.g. std::numeric_limits<short>, throw std::range_error if it's not in range, otherwise return that value. There.
If you already have the Boost library installed you might use boost::lexical_cast for convenience, but otherwise I would avoid it (mainly for the verbosity and library dependency, and it's also a little inefficient).
Earlier boost::lexical_cast was known for not being very efficient, I believe because it was based internally on stringstreams, but as reported in comments here the modern version is faster than conversion via stringstream, and for that matter than via scanf.
An efficient way is to use boost::lexical_cast:
short myShort = boost::lexical_cast<short>(myString);
You will need to install boost library and the following include: #include <boost/lexical_cast.hpp>
You should catch bad_lexical_cast in case the cast fails:
try
{
short myShort = boost::lexical_cast<short>(myString);
}
catch(bad_lexical_cast &)
{
// Do something
}
You can also use ssprintf with the %hi format specifier.
Example:
short port;
char szPort[] = "80";
sscanf(szPort, "%hi", &port);
the number should never go above three or below zero
If you really really need to save memory, then this will also fit in a char (regardless whether char is signed or unsigned).
Another 'extreme' trick: if you can trust there are no weird things like "002" then what you have is a single character string. If that is the case, and you really really need performance, try:
char result = (char)( *ptr_c_string - '0' );
I am compressing string. And the compressed string sometimes having NULL character inside before the end NULL. I want to return the string till the end null.But the compressor function is returning the sting till the occurring of the first NULL. I made a question for c before about it. But consecutively I need also the solution in C++ now, and in next C#. Please help me.Thanks.
char* compressor(char* str)
{
char *compressed_string;
//After some calculation
compressed_string="bk`NULL`dk";// at the last here is automatic an NULL we all know
return compressed_string;
}
void main()
{
char* str;
str=compressor("Muhammad Ashikuzzaman");
printf("Compressed Value = %s",str);
}
The output is : Compressed Value = bk;
And all other characters from compressor function is not here. Is there any way to show all the string.
The fundamental problem that you have is that compression algorithms operate on binary data rather than text. If you compress something, then expect some of the compressed bytes to be zero. Thus the compressed data cannot be stored in a null-terminated string.
You need to change your mindset to work with binary data.
To compress do the following:
Convert from text to binary using some well-defined encoding. For instance, UTF-8. This will yield an array of unsigned char.
Compress the unsigned char, which will again yield an array of unsigned char, but now compressed.
To decompress you just reverse these steps.
Since you are writing C++ code you would be well advised to use standard containers. Such as std::string or std::wstring and std::vector<T>.
The exact same principles apply in all languages. When you come to code this in C#, you need to convert from text to binary. Use Encoding.GetBytes() to do that. That yields a byte array, byte[]. Compress that to another byte array. And so on.
But you really must first overcome this desire to attempt to store binary data in text data types.
I have an external_jpeg_func() that takes jpeg data in a char array to do stuff with it. I am unable to modify this function. In order to provide it the char array, I do something like the following:
//what the funcs take as inputs
std::string my_get_jpeg();
void external_jpeg_func(const char* buf, unsigned int size);
int main ()
{
std::string myString = my_get_jpeg();
external_jpeg_func(myString.data(), myString.length() );
}
My question is: Is it safe to use a string to transport the char array around? Does jpeg (or perhaps any binary file format) be at risk of running into characters like '\0' and cause data loss?
My recommendation would be to use std::vector<char>, instead of std::string, in this case; the danger with std::string is that it provides a c_str() function and most developers assume that the contents of a std::string are NUL-terminated, even though std::string provides a size() function that can return a different value than what you would get by stopping at NUL. That said, as long as you are careful to always use the constructor that takes a size parameter, and you are careful not to pass the .c_str() to anything, then there is no problem with using a string here.
While there is no technical advantage to using a std::vector<char> over a std::string, I feel that it does a better job of communicating to other developers that the content is to be interpreted as an arbitrary byte sequence rather than NUL-terminated textual content. Therefore, I would choose the former for this added readability. That said, I have worked with plenty of code that uses std::string for storing arbitrary bytes. In fact, the C++ proto compiler generates such code (though, I should add, that I don't think this was a good choice for the readability reasons that I mentioned).
std::string does not treat null characters specially, unless you don't give it an explicit string length. So your code will work fine.
Although, in C++03, strings are technically not required to be stored in contiguous memory. Just about every std::string implementation you will find will in fact store them that way, but it is not technically required. C++11 rectifies this.
So, I would suggest you use a std::vector<char> in this case. std::string doesn't buy you anything over a std::vector<char>, and it's more explicit that this is an array of characters and not a possibly printable string.
I think it is better to use char array char[] or std::vector<char>. This is standard way to keep images. Of course, binary file may contain 0 characters.