Extract hexdump or RAW data of a file to text - c++

I was wondering if there is a way to output the hexdump or raw data of a file to txt file.
for example
I have a file let's say "data.jpg" (the file type is irrelevant) how can I export the HEXdump (14ed 5602 etc) to a file "output.txt"?
also how I can I specify the format of the output for example, Unicode or UTF?
in C++

You can use a loop, fread and fprintf: With read you get the byte-value of the bytes, then with fprintf you can use the %x to print hexadecimal to a file.
http://www.cplusplus.com/reference/clibrary/cstdio/fread/
http://www.cplusplus.com/reference/clibrary/cstdio/fprintf/
If you want this to be fast you load whole machine-words (int or long long) instead of single bytes, if you want this to be even faster you fread a whole array, then sprintf a whole array, then fprintf that array to the file.

Maybe something like this?
#include <sstream>
#include <iostream>
#include <iomanip>
#include <iterator>
#include <algorithm>
int main()
{
std::stringstream buffer( "testxzy" );
std::istreambuf_iterator<char> it( buffer.rdbuf( ) );
std::istreambuf_iterator<char> end; // eof
std::cout << std::hex << std::showbase;
std::copy(it, end, std::ostream_iterator<int>(std::cout));
std::cout << std::endl;
return 0;
}
You just have to replace buffer with an ifstream that reads the binary file, and write the output to a textfile using an ofstream instead of cout.

This is pretty old -- if you want Unicode, you'll have to add that yourself.
#include <stdio.h>
#include <stdlib.h>
int main(int argc, char **argv) {
unsigned long offset = 0;
FILE *input;
int bytes, i, j;
unsigned char buffer[16];
char outbuffer[60];
if ( argc < 2 ) {
fprintf(stderr, "\nUsage: dump filename [filename...]");
return EXIT_FAILURE;
}
for (j=1;j<argc; ++j) {
if ( NULL ==(input=fopen(argv[j], "rb")))
continue;
printf("\n%s:\n", argv[j]);
while (0 < (bytes=fread(buffer, 1, 16, input))) {
sprintf(outbuffer, "%8.8lx: ", offset+=16);
for (i=0;i<bytes;i++) {
sprintf(outbuffer+10+3*i, "%2.2X ",buffer[i]);
if (!isprint(buffer[i]))
buffer[i] = '.';
}
printf("%-60s %*.*s\n", outbuffer, bytes, bytes, buffer);
}
fclose(input);
}
return 0;
}

Related

Hex value comparison and save to file C++

I am reading H.264 bitstream as Hex file in c++. I want to insert some string whenever some certain condition met.Like in the attached image if hex value of 00 00 00 01 occurs anywhere in the file i want to add some string like ABC before 00 00 00 01 in the file and save this as a new file. Write now my approach is to read h.264 file as hex. convert it into string and make a string comparison. if there is a way i can do a straight hex comparison? Here is my current code
#include "stdafx.h"
#include <iostream>
#include <fstream>
#include <sstream>
#include <string>
#include <iomanip>
using namespace std;
int _tmain(int argc, _TCHAR* argv[])
{
unsigned char x;
string s1,s2,s3;
s2="Mushahid Hussain";
s3="0000000141";
std::ifstream input("d:\\Jm\\videos\\trying2.264", std::ios::binary);
input >> std::noskipws;
while (input >> x) {
long constant = 0x0000000168;
std::ostringstream buffer;
buffer << std::hex << std::setw(2) << std::setfill('0')
<< (int)x;
s1=buffer.str();
if (s1.find(s1) != std::string::npos) {
cout<<"hello";
s1+=s2;
}
std::ofstream outfile;
outfile.open("d:\\Jm\\bin\\trying5.264", std::ios_base::app);
outfile << s1;
}
return 0;
}
Edit 1
As answered by Tommylee2k i am able to append string . But problem is that at the end of file hex CD value is appending like shown in the attached image.
perhaps a better approach is to read the file binary into a memory buffer, and find the memcmp() instead.
When you found your pattern, you write the block ahead of the match, then your "ABC"-string, and continue searching the rest of the file
#include <stdio.h>
#include <string.h>
#include <memory.h>
#include <malloc.h>
char pattern[4]= { 0x00,0x00,0x01,0x67 };
char *memfind(char *s, int len, char *p, int plen) {
int n=0;
char *pos = s;
while ((pos-s)<(len-plen)) {
while ( *(pos+n) == *(p+n) && n<=plen) n++;
if (n==plen)
return pos;
pos++;n=0;
}
return NULL;
}
int main() {
FILE *in = fopen("in.vid", "r+");
FILE *out = fopen("out.vid", "wb");
// get Filesize
size_t size = 0;
fseek(in, 0L, SEEK_END);
size = ftell(in);
// read whole file in
char *buffer = malloc(size);
fseek (in, 0L, SEEK_SET);
fread (buffer, size, 1, in);
char *currentPos = buffer;
char *found;
if (buffer) {
while (1) {
found = memfind(currentPos, size-(currentPos-buffer), pattern, sizeof(pattern));
if (found==NULL) break;
fwrite(currentPos, 1, (size_t) (found-currentPos), out);
fwrite("ABC", sizeof("ABC"), 1, out);
fwrite(pattern, sizeof(pattern),1,out);
currentPos=found+4;
}
fwrite (currentPos, 1, (size_t) size - (currentPos-buffer), out);
free(buffer);
}
fclose (in);
fclose (out);
return 0;
}

Outputting raw bytes from a file, first byte is corrupted

So I have written a small program that reads the contents of a file into a char array(because fstream seems to only support char pointers). What I want to do is send the raw bytes to the console. AFAIK char is an 8 bit data type so it should not be too hard.
However if I just print members of the array, I get the characters corresponding to the ASCII values, so I am using a static cast. This works fine, except the first byte does not seem to get cast properly.
I am using a PNG file as the test.bin file. PNG files always begin with the byte sequence of 137,80,78,71,13,10,26,10. However the fist byte is printed incorrectly. I have a feeling it has to do something with the value being over 127. However, I cannot change the read buffer data type to anything else (like unsigned char, or unsigned short int), because foo.read() from fstream only supports char destination buffers.
How do I get fstream to read the raw bytes into a usable unsigned type?
My code:
#include <iostream>
#include <fstream>
#include <sys/stat.h>
#define filename "test.bin"
void pause(){
std::string dummy;
std::cout << "Press enter to continue...";
std::getline(std::cin, dummy);
}
int main(int argc, char** argv) {
using std::cout;
using std::endl;
using std::cin;
// opening file
std::ifstream fin(filename, std::ios::in | std::ios::binary);
if (!fin.is_open()) {
cout << "error: open file for input failed!" << endl;
pause();
abort();
}
//getting the size of the file
struct stat statresults;
if (stat(filename, &statresults) == 0){
cout<<"File size:"<<statresults.st_size<<endl;
}
else{
cout<<"Error determining file size."<<endl;
pause();
abort();
}
//setting up read buffer and reading the entire file into the buffer
char* rBuffer = new char[statresults.st_size];
fin.read(rBuffer, statresults.st_size);
//print the first 8 bytes
int i=0;
for(i;i<8;i++) {
cout<<static_cast<unsigned short>(rBuffer[i])<<";";
}
pause();
fin.clear();
fin.close();
delete [] rBuffer;
pause();
return 0;
}
-119 signed is 137 unsigned (both are 1000 1001 in binary).
This gets sign-extended into the short 1111 1111 1000 1001, which is 65,417 unsigned.
I assume this is the value you're seeing.
To read into an unsigned buffer:
unsigned char* rBuffer = new unsigned char[statresults.st_size];
fin.read(reinterpret_cast<char*>(rBuffer), statresults.st_size);
How about trying something other than fin.read()?
Instead of:
char* rBuffer = new char[statresults.st_size];
fin.read(rBuffer, statresults.st_size);
You could use:
unsigned char* rBuffer = new unsigned char[statresults.st_size];
for(int i = 0; i < statresults.st_size; i++)
{
fin.get(rBuffer[i]);
}
You likely want to be using unsigned char as your "byte". You could try something like this:
using byte = unsigned char;
...
byte* buffer = new byte[statresults.st_size];
fin.read( reinterpret_cast<char*>( buffer ), statresults.st_size );
...
delete[] buffer;

Retrieving Char digits from a text file and appending them into a String?

I am trying to get some chars read from a file and into a string. For example #2%4$$3. My program will take out 243 from the text file, now I need it to be in a string so I can convert it to two hundred and 43 and given to an INT var.
Any help would be Superbly appreciated! :)
Thanks,
Brian
Try this program
#include <stdio.h>
#include <string.h>
int main(int argc, char **argv)
{
char file_buff[1024]="!#25#3$#4";
char num_buff[1024];
int len,i,k;
// open file and read the entire contents specied as per the size
FILE *f = fopen("file.txt", "r");
fgets(file_buff, 1024, f);
printf("String read: %s\n", file_buff);
fclose(f);
len=strlen(file_buff);
// parse the buffer for digits and store it in a buffer
for(i=0,k=0;i<len;i++)
{
if((file_buff[i] >= 0x30) && (file_buff[i] <= 0x39))
{
num_buff[k]=file_buff[i];
k++;
}
}
num_buff[k]='\0';
printf("%s",num_buff);
return 0;
}

How to copy the output of linux command to a C++ variable

I'm calling a LINUX command from within a C++ programme which creates the following output. I need to copy the first column of the output to a C++ variable (say a long int). How can I do it?? If that is not possible how can I copy this result into a .txt file with which I can work with?
Edit
0 +0
2361294848 +2361294848
2411626496 +50331648
2545844224 +134217728
2713616384 +167772160
I have this stored as a file, file.txt and I'm using the following code to
extract the left column with out the 0 to store it at integers
string stringy="";
int can_can=0;
for(i=begin;i<length;i++)
{
if (buffer[i]==' ' && can_can ==1) //**buffer** is the whole text file read in char*
{
num=atoi(stringy.c_str());
array[univ]=num; // This where I store the values.
univ+=1;
can_can=1;
}
else if (buffer[i]==' ' && can_can ==0)
{
stringy="";
}
else if (buffer[i]=='+')
{can_can=0;}
else{stringy.append(buffer[i]);}
}
I'm getting a segmentation error for this. What can be done ?
Thanks in advance.
Just create a simple streambuf wrapper around popen()
#include <iostream>
#include <stdio.h>
struct SimpleBuffer: public std::streambuf
{
typedef std::streambuf::traits_type traits;
typedef traits::int_type int_type;
SimpleBuffer(std::string const& command)
: stream(popen(command.c_str(), "r"))
{
this->setg(&c[0], &c[0], &c[0]);
this->setp(0, 0);
}
~SimpleBuffer()
{
if (stream != NULL)
{
fclose(stream);
}
}
virtual int_type underflow()
{
std::size_t size = fread(c, 1, 100, stream);
this->setg(&c[0], &c[0], &c[size]);
return size == 0 ? EOF : *c;
}
private:
FILE* stream;
char c[100];
};
Usage:
int main()
{
SimpleBuffer buffer("echo 55 hi there Loki");
std::istream command(&buffer);
int value;
command >> value;
std::string line;
std::getline(command, line);
std::cout << "Got int(" << value << ") String (" << line << ")\n";
}
Result:
> ./a.out
Got int(55) String ( hi there Loki)
It is popen you're probably looking for. Try
man popen
.
Or see this little example:
#include <iostream>
#include <stdio.h>
using namespace std;
int main()
{
FILE *in;
char buff[512];
if(!(in = popen("my_script_from_command_line", "r"))){
return 1;
}
while(fgets(buff, sizeof(buff), in)!=NULL){
cout << buff; // here you have each line
// of the output of your script in buff
}
pclose(in);
return 0;
}
Unfortunately, it’s not easy since the platform API is written for C. The following is a simple working example:
#include <cstdio>
#include <iostream>
int main() {
char const* command = "ls -l";
FILE* fpipe = popen(command, "r");
if (not fpipe) {
std::cerr << "Unable to execute commmand\n";
return EXIT_FAILURE;
}
char buffer[256];
while (std::fgets(buffer, sizeof buffer, fpipe)) {
std::cout << buffer;
}
pclose(fpipe);
}
However, I’d suggest wrapping the FILE* handle in a RAII class to take care of resource management.
You probably want to use popen to execute the command. This will give you a FILE * that you can read its output from. From there, you can parse out the first number with (for example) something like:
fscanf(inpipe, "%d %*d", &first_num);
which, just like when reading from a file, you'll normally repeat until you receive an end of file indication, such as:
long total = 0;
while (1 == fscanf(inpipe, "%l %*d", &first_num))
total = first_num;
printf("%l\n", total);

How to write to a memory buffer with a FILE*?

Is there any way to create a memory buffer as a FILE*. In TiXml it can print the xml to a FILE* but i cant seem to make it print to a memory buffer.
There is a POSIX way to use memory as a FILE descriptor: fmemopen or open_memstream, depending on the semantics you want: Difference between fmemopen and open_memstream
I guess the proper answer is that by Kevin. But here is a hack to do it with FILE *. Note that if the buffer size (here 100000) is too small then you lose data, as it is written out when the buffer is flushed. Also, if the program calls fflush() you lose the data.
#include <stdio.h>
#include <stdlib.h>
int main(int argc, char **argv)
{
FILE *f = fopen("/dev/null", "w");
int i;
int written = 0;
char *buf = malloc(100000);
setbuffer(f, buf, 100000);
for (i = 0; i < 1000; i++)
{
written += fprintf(f, "Number %d\n", i);
}
for (i = 0; i < written; i++) {
printf("%c", buf[i]);
}
}
fmemopen can create FILE from buffer, does it make any sense to you?
I wrote a simple example how i would create an in-memory FILE:
#include <unistd.h>
#include <stdio.h>
int main(){
int p[2]; pipe(p); FILE *f = fdopen( p[1], "w" );
if( !fork() ){
fprintf( f, "working" );
return 0;
}
fclose(f); close(p[1]);
char buff[100]; int len;
while( (len=read(p[0], buff, 100))>0 )
printf(" from child: '%*s'", len, buff );
puts("");
}
C++ basic_streambuf inheritance
In C++, you should avoid FILE* if you can.
Using only the C++ stdlib, it is possible to make a single interface that transparently uses file or memory IO.
This uses techniques mentioned at: Setting the internal buffer used by a standard stream (pubsetbuf)
#include <cassert>
#include <cstring>
#include <fstream>
#include <iostream>
#include <ostream>
#include <sstream>
/* This can write either to files or memory. */
void write(std::ostream& os) {
os << "abc";
}
template <typename char_type>
struct ostreambuf : public std::basic_streambuf<char_type, std::char_traits<char_type> > {
ostreambuf(char_type* buffer, std::streamsize bufferLength) {
this->setp(buffer, buffer + bufferLength);
}
};
int main() {
/* To memory, in our own externally supplied buffer. */
{
char c[3];
ostreambuf<char> buf(c, sizeof(c));
std::ostream s(&buf);
write(s);
assert(memcmp(c, "abc", sizeof(c)) == 0);
}
/* To memory, but in a hidden buffer. */
{
std::stringstream s;
write(s);
assert(s.str() == "abc");
}
/* To file. */
{
std::ofstream s("a.tmp");
write(s);
s.close();
}
/* I think this is implementation defined.
* pusetbuf calls basic_filebuf::setbuf(). */
{
char c[3];
std::ofstream s;
s.rdbuf()->pubsetbuf(c, sizeof c);
write(s);
s.close();
//assert(memcmp(c, "abc", sizeof(c)) == 0);
}
}
Unfortunately, it does not seem possible to interchange FILE* and fstream: Getting a FILE* from a std::fstream
You could use the CStr method of TiXMLPrinter which the documentation states:
The TiXmlPrinter is useful when you
need to:
Print to memory (especially in non-STL mode)
Control formatting (line endings, etc.)
https://github.com/Snaipe/fmem is a wrapper for different platform/version specific implementations of memory streams
It tries in sequence the following implementations:
open_memstream.
fopencookie, with growing dynamic buffer.
funopen, with growing dynamic buffer.
WinAPI temporary memory-backed file.
When no other mean is available, fmem falls back to tmpfile()