Converting csv read hex string values to uint8_t in c++ - c++

So I am currently able to read some hex values from a csv file using ifstream. I am able to read the values and assign them to a string variable. But I want to convert them to uint8_t values. So essentially I want something like this uint8_t A = 0x41;
I have tried atoi , stoi , and strtol but i don't seem to get any valid output.(I might be using them wrong)
int read_csv(){
using namespace std;
ifstream fin;
fin.open("simpletrajhex.txt");
while (fin.good ()){
char[] line;
getline (fin, line, ',');
uint8_t hex1 = (uint8_t)stoi(line,NULL,16);
cout << line << " " << hex1 << endl;
}
return 1;
}
When I read my csv file i get something similar
0xFF
0x5F
0x02
0x00
0xFF
But these are strings and I need each value to be converted to uint8_t
and when I try convert nothing shows up.

You have a number of problems with your code, but two primary problems related to your conversion to uint8_t. A general problem with attempting to read a .csv file with getline (fin, line, ','); is you have no way of tracking the number of values present in each line. Reading with the ',' delimiter will cause getline to skip the line-ending and simple read the next value until EOF is encountered. Better to read an entire line into line with getline (fin, line); and then create a stringstream from line allowing you to read the value in the line until the end of the stringstream is reached (limiting the value conversions to those within each line).
The primary obstacle to storing the values as uint8_t is your failure the validate the the result of the stoi conversion is within the range of uint8_t before making the assignment. Further, since uint8_t is an unsigned value, use of stoul is more appropriate. While the C-style cast to (uint8_t) is valid, it is better to use a static_cast<uint8_t>(...) (though both provide the same result). Finally, your attempt to output << hex1 ... will always fail as the << operator expects an int (or unsigned) value.
Putting all of those pieces together, you can rework your read_csv() to take the name of the file to open as a parameter rather than hardcoding the filename in the function (don't do that) and do something like:
int read_csv (const string& name)
{
string line;
ifstream fin (name);
while (getline (fin, line)) { /* read entire line into line */
stringstream ss (line); /* create stringstream from line */
string field; /* string to hold each field */
cout << "line: " << line << '\n';
while (getline (ss, field, ',')) { /* read each hex value from line */
uint64_t tmp = stoul (field, 0, 0); /* convert to uint64_t */
if (tmp <= UINT8_MAX) { /* validate in range of uint8_t */
uint8_t hex1 = static_cast<uint8_t>(tmp); /* store uint8_t */
/* output with cast to unsigned for << */
cout << " " << field << " -> " <<
static_cast<uint32_t>(hex1) << '\n';
}
}
}
return 1;
}
note: rather than casing within cout you can also precede hex1 with a numeric + to force the promotion, e.g.
cout << " " << field << " -> " << +hex1 << '\n';
also note: that using 0 for the base in stoul, the numeric base is auto-detected: if the prefix is 0, the base is octal, if the prefix is 0x or 0X, the base is hexadecimal, otherwise the base is decimal.
A short example making use of the function could be:
#include <iostream>
#include <fstream>
#include <sstream>
#include <string>
#include <cstdint>
using namespace std;
int read_csv (const string& name)
{
string line;
ifstream fin (name);
while (getline (fin, line)) { /* read entire line into line */
stringstream ss (line); /* create stringstream from line */
string field; /* string to hold each field */
cout << "line: " << line << '\n';
while (getline (ss, field, ',')) { /* read each hex value from line */
uint64_t tmp = stoul (field, 0, 0); /* convert to uint64_t */
if (tmp <= UINT8_MAX) { /* validate in range of uint8_t */
uint8_t hex1 = static_cast<uint8_t>(tmp); /* store uint8_t */
/* output with cast to unsigned for << */
cout << " " << field << " -> " <<
static_cast<uint32_t>(hex1) << '\n';
}
}
}
return 1;
}
int main (int argc, char **argv) {
if (argc < 2) {
cerr << "error: insufficient input\n" <<
"usage: " << argv[0] << " <file>\n";
return 1;
}
read_csv (argv[1]);
}
(note: using namespace std; is generally encouraged, so while it can ease typing for short example programs, you will generally want to avoid including the entire std namespace in each source file (and especially avoid including it in header files))
Example Input File
$ cat dat/hexcsv.csv
0xff,0x5f
0x02,0x00
0xff,0xaa
Example Use/Output
Each of the numeric values are store in the uint8_t variable hex1 before being output in decimal.
$ ./bin/read_csv_uint8 dat/hexcsv.csv
line: 0xff,0x5f
0xff -> 255
0x5f -> 95
line: 0x02,0x00
0x02 -> 2
0x00 -> 0
line: 0xff,0xaa
0xff -> 255
0xaa -> 170

You cannot store hex value as it is in an uint8_t variable, so uint8_t A = 0x41 is not possible. Instead, you will have to get a decimal equivalent of the hex number into uint8_t.
Try using std::stoul
Get the decimal equivalent of hex and get the hex back from the decimal when you need it.

Related

How do I read in a specified number of characters from a file while still iterating through it?

I have a file of data like this:
Judy Henn 2 Oaklyn Road Saturday 2001
Norman Malnark 15 Manor Drive Saturday 2500
Rita Fish 210 Sunbury Road Friday 750
I need to assign the first 20 characters as the name, next 20 as address, next 10 as day, and the number as yardSize, using the istream::get() method. My professor is requiring the use of .get() to accomplish this.
I am having a really hard time figuring out how to assign the data from the file to the right variables while still looping.
struct Customer{
char name[21];
char address[21];
char day[11];
int yardSize;
};
int main(){
const int arrSize = 50;
Customer custArr[arrSize];
int i = 0;
//set up file
ifstream dataFile;
dataFile.open("Data.txt");
//try to open file
if(!dataFile){
cout << "couldn't open file";
}
//while dataFile hasn't ended
while(!dataFile.eof()){
dataFile.get(custArr[i].name, 21);
cout << custArr[i].name;
i++;
}
}; //end
I would have thought that the while loop would assign the first 21 characters into custArr[i].name, then loop over and over until the end of file. However, when I print out custArr[i].name, I get this and ONLY this:
Judy Henn 2 Oaklyn Road Saturday 2001
I'm not sure how to go about assigning a specified number of characters to a variable, while still iterating through the entire file.
First off, the character counts you mentioned don't match the data file you have shown. There are only 19 characters available for the name, not 20. And only 9 characters available for the day, not 10.
After fixing that, your code is still broken, as it is reading only into the Customer::name field. So it will try to read Judy Henn into custArr[0].name, then 2 Oaklyn Road into custArr[1].name, then Saturday into custArr[2].name, and so on.
I would suggest something more like this instead:
#include <iostream>
#include <fstream>
#include <sstream>
#include <string>
using namespace std;
struct Customer
{
char name[21];
char address[21];
char day[11];
int yardSize;
};
int main()
{
const int arrSize = 50;
Customer custArr[arrSize];
string line;
int i = 0;
//set up file
ifstream dataFile("Data.txt");
if (!dataFile)
{
cout << "couldn't open file";
return 0;
}
//while dataFile hasn't ended
while ((i < arrSize) && getline(dataFile, line))
{
istringstream iss(line);
if (iss.get(custArr[i].name, 21) &&
iss.get(custArr[i].address, 21) &&
iss.get(custArr[i].day, 11) &&
iss >> custArr[i].yardSize)
{
cout << custArr[i].name;
++i;
}
}
return 0;
}
Reading fixed-width (mainframe type) records isn't something C++ was written to do specifically. While C++ provides a wealth of string manipulation functions, reading fixed-width records is still something you have to put together yourself using basic I/O functions.
In addition to using to the great answer by #RemyLebeau, a similar approach using std::vector<Customer> instead of an array of customers eliminates bounds concerns. By using a std::vector instead of an array, you can adapt the code to read as many records as needed (up to the limits of your physical memory) without the fear of adding information past an array bound.
Additionally, as currently written, you leave the leading and trailing whitespace in each array. For example, your name array would hold " Judy Henn " instead of just "Judy Henn". Generally you will always want to trim leading and trailing whitespace from what you store as a variable. Otherwise, when you use the stored characters you will have to have someway to deal with the whitespace each time the contents are used. While std::string provides a number of methods you can use to trim leading and trailing whitespace, your use of plain old char[] will require a manual removal.
Adding code to trim the excess leading and trailing whitespace from the character arrays in the collection of Customer could be written as follows.
#include <iostream>
#include <fstream>
#include <sstream>
#include <string>
#include <vector>
#include <cstring>
#define NAMLEN 20 /* if you need a constant, #define one (or more) */
#define ADDRLEN 21 /* (these marking the fixed-widths of the fields) */
#define DAYLEN 10
struct Customer {
char name[21];
char address[21];
char day[11];
int yardSize;
};
int main (int argc, char **argv) {
if (argc < 2) { /* validate at least one argument given for filename */
std::cerr << "error: insufficient no. of arguments\n"
"usage: " << argv[0] << " <filename>\n";
return 1;
}
std::string line {}; /* string to hold each line read from file */
std::vector<Customer> customers {}; /* vector of Customer struct */
std::ifstream f (argv[1]); /* file stream (filename in 1st arg) */
if (!f.is_open()) { /* validate file open for reading */
std::cerr << "error: file open failed '" << argv[1] << "'.\n"
<< "usage: " << argv[0] << " <filename>\n";
return 1;
}
while (getline (f, line)) { /* read each line into line */
std::stringstream ss (line); /* create stringstream from line */
Customer tmp {}; /* declare temporary instance */
char *p; /* pointer to trim leading ws from name */
size_t wslen; /* whitespace len to use in trim */
ss.get (tmp.name, NAMLEN); /* read up to NAMLEN chars from ss */
if (ss.gcount() != NAMLEN - 1) { /* validate gcount()-1 chars read */
std::cerr << "error: invalid format for name.\n";
continue;
}
for (int i = NAMLEN - 2; tmp.name[i] == ' '; i--) /* loop from end of name */
tmp.name[i] = 0; /* overwrite spaces with nul-char */
for (p = tmp.name; *p == ' '; p++) {} /* count leading spaces */
wslen = strlen (p); /* get remaining length */
memmove (tmp.name, p, wslen + 1); /* move name to front of array */
ss.get (tmp.address, ADDRLEN); /* read up to ADDRLEN chars from ss */
if (ss.gcount() != ADDRLEN - 1) { /* validate gcount()-1 chars read */
std::cerr << "error: invalid format for address.\n";
continue;
}
for (int i = ADDRLEN - 2; tmp.address[i] == ' '; i--)/* loop from end of name */
tmp.address[i] = 0; /* overwrite spaces with nul-char */
ss.get (tmp.day, DAYLEN); /* read up to DAYLEN chars from ss */
if (ss.gcount() != DAYLEN - 1) { /* validate gcount()-1 chars read */
std::cerr << "error: invalid format for day.\n";
continue;
}
for (int i = DAYLEN - 2; tmp.day[i] == ' '; i--) /* loop from end of name */
tmp.day[i] = 0; /* overwrite spaces with nul-char */
if (!(ss >> tmp.yardSize)) { /* extract final int value from ss */
std::cerr << "error: invalid format for yardSize.\n";
continue;
}
customers.push_back(tmp); /* add temp to vector */
}
for (Customer c : customers) /* output information */
std::cout << "\n'" << c.name << "'\n'" << c.address << "'\n'" <<
c.day << "'\n'" << c.yardSize << "'\n";
}
(note: the program expects the filename to read to be provided on the command line as the first argument. You can change how you provide the filename to suite your needs, but you should not hardcode filenames or use MagicNumbers in your code. You shouldn't have to re-compile your program just to read from another filename)
Also note that in the for() loop trimming whitespace, you are dealing with 0-based indexes instead of a 1-based count of characters which is why you are using gcount() - 1 or the total number of chars minus two, e.g. NAMLEN - 2 to loop from the last character in the array back towards the beginning.
The removal of trailing whitespace simply loops from the last character in each string from the end of each array back toward the beginning overwriting each space with a nul-terminating character. To trim leading whitespace from name, the number of whitespace characters are counted and then C memmove() is used to move the name back to the beginning of the array.
Example Use/Output
$ ./bin/read_customer_day_get dat/customer_day_get.txt
'Judy Henn'
'2 Oaklyn Road'
'Saturday'
'2001'
'Norman Malnark'
'15 Manor Drive'
'Saturday'
'2500'
'Rita Fish'
'210 Sunbury Road'
'Friday'
'750'
The output of each value has been wrapped in single-quotes to provide visual confirmation that the name field has had both leading and trailing whitespace removed, while address and day have both had trailing whitespace removed.

Need to write string to a file in hex format in c++

I am having a string which contains the hex value:
string str = "e101";
I need to write this in a file as 2 bytes. While I am trying to write a file, it will write like the following 4 bytes value:
65 31 30 31
I am using the following operation for file write:
myfile.open ("file.cf3",std::ios::binary);
myfile << str << "\n";
myfile.close();
but I want to write it as a 2 bytes value.
For example, if i g How to write it as 2 bytes to a file?
std::string wut="b6306edf953a6ac8d17d70bda3e93f2a3816eac333d1ac78";
i want the output like
.0n..:j..}p...?*8...3..x
I think your question is ambiguous ...
Keep in mind that, from your string, every two char you have 1 byte (not two).
So you want to write two numbers (meaning in ascii) representing the hex value of the string...
If this is the right interpretation, you need to split the string in pairs of chars and then convert each one to the equivalent integer.
Here is my code ...
It writes out to stdout, but you can modify it easily in order to write to file instead to the screen.
#include <iostream>
#include <stdio.h>
#include <string>
#include <cstring>
#include <stdlib.h>
using namespace std;
int main () {
string str = "e101";
string two_char;
unsigned char byte;
for (int i=0 ; i<str.size(); i+=2) {
two_char = str.substr(i,2);
byte = strtol(two_char.c_str(),0,16);
cout << two_char << " " << (int)byte <<"\n";
}
}
Here is an example for a solution.
#include <fstream>
#include <iostream>
#include <string>
int main() {
std::ofstream file("file.txt", std::ios::binary);
if(!file.is_open()) {
return -1;
}
std::string str("e101");
for (std::size_t i = 0; i < str.length() - 1; ++++i) {
file << static_cast<char>(str[i] * 16 + str[i + 1]);
}
file.close();
}
You can simply iterate over your string and take two characters as one byte. You multiply the first character with 16 and add the second character.
In answer to your original question about writing 2-bytes out in binary to a file in C++, you have a basic 2-step process. (1) convert your string representation of the number to a numeric value using stoi with base 16. This provides a numeric values you can store in an unsigned short. (2) write that value out to your file with f.write, not frwite where f is your open stream reference.
If you want to format the output in hex for cout, then you must set the flags for cout to output numeric values in hex-format (though not directly part of your question, it ties in the stream I/O formatting if desired.)
So essentially you have your string and convert it to a number, e.g.
std::string str = "e101";
unsigned short u = stoi(str, 0, 16);
Now u holds a numeric value converted from the text in str using base-16 that you can simply write to your file as a 2-byte value, e.g.
std::string filename = "out.bin"; /* output filename */
...
std::ofstream f (filename, f.trunc | f.binary); /* open out in binary */
if (!f.write(reinterpret_cast<char*>(&u), sizeof u)) { /* write 2 bytes */
std::cerr << "error: write of short to file failed.\n";
return 1;
}
Putting it altogether, you could do something short that outputs the hex value being written with cout as well as writing it to the file "out.bin", e.g.
#include <fstream>
#include <iostream>
#include <iomanip>
#include <string>
int main (void) {
std::string filename = "out.bin"; /* output filename */
std::string str = "e101";
unsigned short u = stoi(str, 0, 16);
/* output converted value to terminal in hex */
std::cout.setf(std::ios::hex, std::ios::basefield); /* set hex output */
std::cout << "writing value to file: " << u << '\n'; /* for cout */
/* output converted value to file */
std::ofstream f (filename, f.trunc | f.binary); /* open out in binary */
if (!f.write(reinterpret_cast<char*>(&u), sizeof u)) { /* write 2 bytes */
std::cerr << "error: write of short to file failed.\n";
return 1;
}
}
Example Use/Output
$ ./bin/stoi_short
writing value to file: e101
Resulting Output File
Confirm by dumping the contents of the file with a hexdump program, e.g.
$ hexdump out.bin
0000000 e101
0000002
Look things over and let me know if you have further questions.

C++ XOR encryption - decryption issue

I followed a tutorial on stephan-brumme website for XOR encryption (unfortunately I cannot include URL because I do not have enough reputation). What I want to do is following: read the content of example.txt file and decrypt the text that it includes. For example, this is the content of example.txt:
\xe7\xfb\xe0\xe0\xe7
This, when decrypted using password "password" should return "hello". This is the code I got:
#include <string>
#include <iostream>
#include <fstream>
using namespace std;
std::string decode(const std::string& input)
{
const size_t passwordLength = 9;
static const char password[passwordLength] = "password";
std::string result = input;
for (size_t i = 0; i < input.length(); i++)
result[i] ^= ~password[i % passwordLength];
return result;
}
int main(int argc, char* argv[])
{
string line;
ifstream myfile ("example.txt");
if (myfile.is_open())
{
while ( getline (myfile,line) )
{
cout << decode(line);
}
myfile.close();
}
return 0;
}
And this is the result of running the application:
click for image
As you can see, the decryption was not successful. Now, if I make it so it doesn't read the .txt, but directly decrypts the text, like this:
cout << decode("\xe7\xfb\xe0\xe0\xe7");
It works perfectly:
click for image
What am I doing wrong here?
Many thanks in advance! :)
Character XOR by same character is zero, so the result may include zero. std::string doesn't like that because zero terminates the string.
You also can use std::vector<char> instead of std::string for the actual encoding/decoding. You would have to change the decode function to handle vector<char>
And read/write the file in binary.
Edit: Using std::string only, and std::string decode(const std::string& input)
int main()
{
std::string line = "hello";
{
line = decode(line);
std::ofstream myfile("example.txt", std::ios::binary);
myfile.write(line.data(), line.size());
//Edit 2 *************
//std::cout << std::hex;
//for (char c : line)
// std::cout << "\\x" << (0xff & c);
//*************
//This will make sure width is always 2
//For example, it will print "\x01\x02" instead of "\x1\x2"
std::cout << std::hex << std::setfill('0');
for (char c : line)
std::cout << "\\x" << std::setw(2) << (0xff & c);
std::cout << std::endl;
}
{
std::ifstream myfile("example.txt", std::ios::binary | std::ios::ate);
int filesize = (int)myfile.tellg();
line.resize(filesize);
myfile.seekg(0);
myfile.read(&line[0], filesize);
line = decode(line);
std::cout << line << std::endl;
}
return 0;
}
I bet example.txt contains the characters '\', 'x', 'e', '7' etc. You have to read those, process all the backslash escapes, and then feed it to decode.
\xe7 is a common way of representing a single character with hex value E7. (Which is quite likely to be the single character 'รง' depending on your character set). If you want to store (encrypted) readable text, I suggest dropping the \x, and having the file contain lines like "e7fbe0e0e7". Then
- read each line into a string.
- Convert each pair of characters from a hex number into an integer, and store the result in a char.
- Store that char in the string.
- Then xor decrypt the string.
Alternatively, ensure the file contains the actual binary characters you need it to.
Also beware that you are XOR-ing with the terminating nul byte of the password. Did you mean to do that?

Writing hex to a file

I have a program that reads the hex of a file, modifies it, and stores the modified hex in a std::string.
For example, how would I write this to a file
std::string wut="b6306edf953a6ac8d17d70bda3e93f2a3816eac333d1ac78";
and get its value
.0n..:j..}p...?*8...3..x
in the outputted file?
I'd prefer not to use sprintf, but I guess if it's necessary, I'll do what I must.
If I understand your question correctly you want the text converted to it's numeric equivalent and then written to file. Given the hint you provided in your question it looks like this should be done byte by byte. Below is one way to achieve this. Note the need to convert each byte from a string to an integer value.
#include <string>
#include <sstream>
#include <iostream>
#include <fstream>
#include <ios>
std::string wut = "b6306edf953a6ac8d17d70bda3e93f2a3816eac333d1ac78";
int main()
{
std::ofstream datafile("c:\\temp\\temp1.dat", std::ios_base::binary | std::ios_base::out);
char buf[3];
buf[2] = 0;
std::stringstream input(wut);
input.flags(std::ios_base::hex);
while (input)
{
input >> buf[0] >> buf[1];
long val = strtol(buf, nullptr, 16);
datafile << static_cast<unsigned char>(val & 0xff);
}
}
The answer of #Peter R will lead to an output which is not 100% equal, due to the stringstream interpreting more than one '0's in a row in an unintended way.
Example: If we want to write the hex value "00000000", the stringstream would output " 000000".
The solution below works in every case, no matter how many zeros are contained in the hex string:
// Input
std::string hex = "180f00005e2c3415"; // (or longer)
std::basic_string<uint8_t> bytes;
// Iterate over every pair of hex values in the input string (e.g. "18", "0f", ...)
for (size_t i = 0; i < hex.length(); i += 2)
{
uint16_t byte;
// Get current pair and store in nextbyte
std::string nextbyte = hex.substr(i, 2);
// Put the pair into an istringstream and stream it through std::hex for
// conversion into an integer value.
// This will calculate the byte value of your string-represented hex value.
std::istringstream(nextbyte) >> std::hex >> byte;
// As the stream above does not work with uint8 directly,
// we have to cast it now.
// As every pair can have a maximum value of "ff",
// which is "11111111" (8 bits), we will not lose any information during this cast.
// This line adds the current byte value to our final byte "array".
bytes.push_back(static_cast<uint8_t>(byte));
}
// we are now generating a string obj from our bytes-"array"
// this string object contains the non-human-readable binary byte values
// therefore, simply reading it would yield a String like ".0n..:j..}p...?*8...3..x"
// however, this is very useful to output it directly into a binary file like shown below
std::string result(begin(bytes), end(bytes));
Then you can simply write this string to a file like this:
std::ofstream output_file("filename", std::ios::binary | std::ios::out);
if (output_file.is_open())
{
output_file << result;
output_file.close();
}
else
{
std::cout << "Error could not create file." << std::endl;
}

Reading hexadecimals from a txt file in C++

I have a txt file which contains integers in hexadecimal form in each line like:
232B2344
A2BC34893
DEF9433
.....
I would like an elegant way of reading this .txt file in C++ and store them in arrays so that I can make computations on them. Also given the hex 232B2344 say I would like to be in position to say the value of each byte entry
First include a header
#include "sstream"
Then get the hexadecimal value in a simple char[] array string using file handling.
char str[100];
Declare an Unsigned int variable...
unsigned int value;
Then declare a "std::stringstream" variable. e.g.
std::stringstream ss;
ss<<std::hex<<str;
ss>>value;
Now the "value" contains the hexadecimal value in file in form of integer.
As far as you describe, each number in the file is a 4 bytes unsigned integer (that fit in 32 bits). Here you have a c++ version for parsing the file and getting the numbers array:
std::vector<std::string> fromFileToArray( const std::string & fileName )
{
std::string line;
std::vector<uint32_t> numbers;
std::stringstream ss;
uint32_t tmp;
std::ifstream numbersFile( fileName.c_str() );
while( numbersFile.good() )
{
getline( numbersFile, line );
ss << std::hex << line;
ss >> tmp;
numbers.push_back( tmp );
}
}
How about this (though it is written with C function, one could help me port this to use C++ functions):
const char hexstring[] = "deadbeef10203040b00b1e50", *pos = hexstring;
unsigned char val[12];
size_t count = 0;
/* WARNING: no sanitization or error-checking whatsoever - MAGIC. DO NOT TOUCH. */
for(count = 0; count < sizeof(val)/sizeof(val[0]); count++) {
sscanf(pos, "%2hhx", &val[count]);
pos += 2 * sizeof(char);
}
/* Does it work? */
printf("0x");
for(count = 0; count < sizeof(val)/sizeof(val[0]); count++)
printf("%02x", val[count]);
printf("\n");
In case of an odd number of hex digits in the string, you have to make sure you prefix it with a starting 0. For example, the string "f00f5" will be evaluated as {0xf0, 0x0f, 0x05} erroneously by the above example, instead of the proper {0x0f, 0x00, 0xf5}.
To read value in hex format - use std::hex from <iomanip>. Like in this very short example:
#include <iostream>
#include <iomanip>
int main() {
int value;
std::cin >> std::hex >> value;
std::cout << value;
}
Just read as many times as your array has elements.