I want to perform XOR operation in AES before and after encryption in AES (like DESX) with new keys. But XOR operation takes too much time.
How can I reduce the XOR operation time?
Here is my code:
string XOR(string value, string key)
{
string retval(value);
short unsigned int klen=key.length();
short unsigned int vlen=value.length();
short unsigned int k=0;
short unsigned int v=0;
for(v;v<vlen;v++)
{
retval[v]=value[v]^key[k];
k=(++k<klen?k:0);
}
return retval;
}
int main(int argc, char* argv[])
{
AutoSeededRandomPool prng;
byte key1[AES::DEFAULT_KEYLENGTH];
prng.GenerateBlock(key1, sizeof(key1));
byte key[AES::DEFAULT_KEYLENGTH];
prng.GenerateBlock(key, sizeof(key));
byte key2[AES::DEFAULT_KEYLENGTH];
prng.GenerateBlock(key2, sizeof(key2));
byte iv[AES::BLOCKSIZE];
prng.GenerateBlock(iv, sizeof(iv));
string plain = "AESX CBC Mode Test";
string cipher,encoded, encodediv, encodedkey1, encodedkey,
encodedkey2, recovered, prerecovered, postrecovered,
prewhiten, postwhiten;
// Pretty print key1
StringSource(key1, sizeof(key1), true,
new HexEncoder(
new StringSink(encodedkey1)
) // HexEncoder
); // StringSource
cout << "key1: " << encodedkey1 << endl;
// Pretty print iv
StringSource(iv, sizeof(iv), true,
new HexEncoder(
new StringSink(encodediv)
) // HexEncoder
); // StringSource
cout << "iv: " << encodediv << endl;
// Pretty print key
StringSource(key, sizeof(key), true,
new HexEncoder(
new StringSink(encodedkey)
) // HexEncoder
); // StringSource
cout << "key: " << encodedkey << endl;
// Pretty print key2
StringSource(key2, sizeof(key2), true,
new HexEncoder(
new StringSink(encodedkey2)
) // HexEncoder
); // StringSource
cout << "key2: " << encodedkey2 << endl;
cout << "plain text: " << plain << endl;
prewhiten = XOR(plain, encodedkey1);
try
{
cout << "pre whiten text: " << prewhiten << endl;
CBC_Mode< AES >::Encryption e;
e.SetKeyWithIV(key, sizeof(key), iv);
// The StreamTransformationFilter removes
// padding as required.
StringSource s(prewhiten, true,
new StreamTransformationFilter(e,
new StringSink(cipher)
) // StreamTransformationFilter
); // StringSource
}
catch(const CryptoPP::Exception& e)
{
cerr << e.what() << endl;
exit(1);
}
/*********************************\
\*********************************/
// Pretty print
encoded.clear();
StringSource(cipher, true,
new HexEncoder(
new StringSink(encoded)
) // HexEncoder
); // StringSource
cout << "cipher text: " << encoded << endl;
postwhiten = XOR(encoded, encodedkey2);
cout << "post whiten text: " << postwhiten << endl;
//decryption
prerecovered = XOR(postwhiten, encodedkey2);
encoded.clear();
StringSource(prerecovered, true,
new HexEncoder(
new StringSink(encoded)
) // HexEncoder
); // StringSource
cout << "pre recovered text: " << encoded << endl;
try
{
CBC_Mode< AES >::Decryption d;
d.SetKeyWithIV(key, sizeof(key), iv);
// The StreamTransformationFilter removes
// padding as required.
StringSource s(prerecovered, true,
new HexDecoder(
new StreamTransformationFilter(d,
new StringSink(recovered)
) // StreamTransformationFilter
)//HexDecoder
); // StringSource
cout << "recovered text: " << recovered << endl;
}
catch(const CryptoPP::Exception& e)
{
cerr << e.what() << endl;
exit(1);
}
postrecovered = XOR(recovered, encodedkey1);
cout << "post recovered text: " << postrecovered << endl;
return 0;
}
Any help would be appreciated.
How can I do XOR operation in Crypto++?
There are two ways to use the library to perform an XOR. First, there are two functions xorbuf in misc.h. The first implementation is shown below, and it uses a single in/out buffer with a mask:
void xorbuf(byte *buf, const byte *mask, size_t count)
{
size_t i=0;
if (IsAligned<word32>(buf) && IsAligned<word32>(mask))
{
if (!CRYPTOPP_BOOL_SLOW_WORD64 && IsAligned<word64>(buf) && IsAligned<word64>(mask))
{
for (i=0; i<count/8; i++)
((word64*)(void*)buf)[i] ^= ((word64*)(void*)mask)[i];
count -= 8*i;
if (!count)
return;
buf += 8*i;
mask += 8*i;
}
for (i=0; i<count/4; i++)
((word32*)(void*)buf)[i] ^= ((word32*)(void*)mask)[i];
count -= 4*i;
if (!count)
return;
buf += 4*i;
mask += 4*i;
}
for (i=0; i<count; i++)
buf[i] ^= mask[i];
}
There's a second xorbuf(byte *output, const byte *input, const byte *mask, size_t count) that uses separate in and out buffers with a mask.
The second way to XOR is use an ArrayXorSink from filters.h. Internally, ArrayXorSink calls xorbuf for you. You would use this is you prefer pipelines.
size_t ArrayXorSink::Put2(const byte *begin, size_t length, int messageEnd, bool blocking)
{
// Avoid passing NULL pointer to xorbuf
size_t copied = 0;
if (m_buf && begin)
{
copied = STDMIN(length, SaturatingSubtract(m_size, m_total));
xorbuf(m_buf+m_total, begin, copied);
}
m_total += copied;
return length - copied;
}
string XOR(string value, string key)
{
string retval(value);
short unsigned int klen=key.length();
short unsigned int vlen=value.length();
short unsigned int k=0;
short unsigned int v=0;
for(v;v<vlen;v++)
{
retval[v]=value[v]^key[k];
k=(++k<klen?k:0);
}
return retval;
}
For this, you could do something like the following. It asks the compiler to inline the function, and it passes the value and key by constant reference to avoid the copies.
inline string XOR(const string& value, const string& key)
{
ASSERT(key.length() == value.length());
string retval(value);
xorbuf(&retval[0], &key[0], retval.length());
return retval;
}
The trick is, you have to take the address of element 0 to get the non-const pointer and avoid potential undefined behavior. You may need to cast to a byte*.
How can I reduce the XOR operation time?
You probably want to use an operand size larger that 1-byte when its feasible. The library's xorbuf uses word32 and word64 when available.
In addition, if you have AVX, then you can operate on buffers up to 512-bit. If you keep your buffers aligned, then GCC will try use the larger buffers at -O3 and above. -O3 is significant because that's when GCC starts aggressive vectorization and using features provided by AVX and AVX2.
The Crypto++ library endured a non-trivial amount of pain a couple of years ago because its buffers were not aligned as GCC expected, and it was causing SEGFAULT's -O3 and above. See, for example, Crash on Cygwin i386 with -O3.
The alignment problem was not limited to Cygwin; Cygwin happened to demonstrate it. The problem surfaced on occasion under other platforms and CPUs, like ARM when NEON was enabled. We believe all the issues have been cleared.
Related
I am new to C++ and I am trying to make a program that reads keystrokes. This is a function I made that looks for a certain key.
void printKey(short vk)
{
if ((GetAsyncKeyState(vk) >> 15) & 1)
{
LPWSTR key;
GetKeyNameTextW(MapVirtualKeyW(vk, MAPVK_VK_TO_CHAR) << 16, key, sizeof(key));
wcout << key;
}
}
I know that the key detection works as I have put code that just prints true or false inside of the if statement so I know that that part is working. For example when I type "s" it prints true if I pass in the virtual key code 0x53 (virtual key code for s). Once I knew that part worked I tried to use the MapVirtualKeyW and GeyKeyNameTextW functions to get the name of the key so I wouldn't have to hard code all of them in. The code compiles but stops immediately after printing Running....
Here is the whole code
#include <iostream>
#include <Windows.h>
#pragma comment(lib, "User32.lib")
using std::cout;
using std::endl;
using std::wcout;
void printKey(short vk)
{
if ((GetAsyncKeyState(vk) >> 15) & 1)
{
LPWSTR key;
GetKeyNameTextW(MapVirtualKeyW(vk, MAPVK_VK_TO_CHAR) << 16, key, sizeof(key));
wcout << key;
}
}
int main()
{
cout << "Running...." << endl;
while (true)
{
for (int i = 48; i <= 90; i++)
{
printKey(i);
}
}
return 0;
}
the range 48-90 is for all the letter and number keys. The idea is that whenever I press a key it should print to the console.
I am fairly confident the issue is in this block of code
LPWSTR key;
GetKeyNameTextW(MapVirtualKeyW(vk, MAPVK_VK_TO_CHAR) << 16, key, sizeof(key));
wcout << key;
Any help is appreciated!
In addition to the comments above, GetKeynameText needs a buffer for the key name, so instead of:
LPWSTR key;
GetKeyNameTextW(MapVirtualKeyW(vk, MAPVK_VK_TO_CHAR) << 16, key, sizeof(key));
you want something like:
WCHAR key [128];
GetKeyNameTextW(MapVirtualKeyW(vk, MAPVK_VK_TO_CHAR) << 16, key, sizeof(key) / sizeof (WCHAR));
You might also flush wcout after writing to it.
i try to make code which is multithreaded and at certain step i have to retrieve array made on heap.
this is the code:
the following function will call another function called read_bi5_to_bin and it will pass to it unsigned char* initialized as nullptr data_bin_buffer
int HTTPRequest::read_bi5_main(boost::filesystem::path p, ptime epoch)
{
boost::unique_lock<boost::mutex> read_bi5_to_bin_lock(m_read_bi5_to_binMutex,boost::defer_lock);
unsigned char *buffer;
size_t buffer_size;
int counter;
size_t raw_size = 0;
std::string filename_string = p.generic_string();
path p2 = p;
p2.replace_extension(".bin");
std::string filename_string_2_bin =p2.generic_string() ;
path p3 = p;
p3.replace_extension(".csv");
std::string filename_string_2_csv = p3.generic_string();
const char *filename = filename_string.c_str();
const char *filename_2_bin = filename_string_2_bin.c_str();
const char *filename_2_csv = filename_string_2_csv.c_str();
if (fs::exists(p) && fs::is_regular(p))
{
buffer_size = fs::file_size(p);
buffer = new unsigned char[buffer_size];
}
else {
//??5-17-2020 isolate multithreaded error
read_bi5_to_bin_lock.lock();
BOOST_LOG((*mHTTPRequest_LoggingInstance_shared_pointer).mloggerCoutLog) << "Error: couldn't access the data file. |"
<< filename << "|" << std::endl;
read_bi5_to_bin_lock.unlock();
return 2;
}
std::ifstream fin(filename, std::ifstream::binary);
//fin.open(filename, std::ifstream::binary);
fin.read(reinterpret_cast<char*>(buffer), buffer_size);
fin.close();
//5-11-2020 the next line will be commented and put in HTTPCLIent constructor
//mHTTPRequest_Symbol_str= mHTTPRequest_HTTPClient_shared_pointer->Get_mHttpClient_HttpSymbolPrepareGet_shared_pointer()->mSymbol_strGet() ;
std::size_t pos = mHTTPRequest_Symbol_str.find("JPY");// position of "h_ticks.bi5" in str
double PV;
if (pos != std::string::npos)
{
PV = PV_YEN_PAIR;
}
else
{
PV = PV_DOLLAR_PAIR;
}
//??5-17-2020 isolate multithreaded error
read_bi5_to_bin_lock.lock();
//5-20-2020
//boost::shared_ptr<unsigned char> data_bin_buffer = boost::make_shared<unsigned char>() ;
//n47::tick_data *data = n47::read_bi5_to_bin(
// buffer, buffer_size, epoch, PV, &raw_size, data_bin_buffer.get());
unsigned char* data_bin_buffer = nullptr;
n47::tick_data *data = n47::read_bi5_to_bin(
buffer, buffer_size, epoch, PV, &raw_size, data_bin_buffer);
//5-11-2020 here i will save binary file
//boost::unique_lock<boost::mutex> read_bi5_to_bin_lock(m_read_bi5_to_binMutex);
std::string file_name_path_string=output_compressed_file_2(data_bin_buffer, raw_size, filename_2_bin);
read_bi5_to_bin_lock.unlock();
path file_name_path_2{ file_name_path_string };
buffer_size = 0;
if (fs::exists(file_name_path_2) && fs::is_regular(file_name_path_2))
{
//??5-17-2020 isolate multithreaded error
read_bi5_to_bin_lock.lock();
BOOST_LOG((*mHTTPRequest_LoggingInstance_shared_pointer).mloggerCoutLog) << boost::this_thread::get_id() <<"\t we can access the data .bin file. |"
<< filename_2_bin << "| with size ="<< fs::file_size(file_name_path_2) << std::endl;
read_bi5_to_bin_lock.unlock();
}
else {
//??5-17-2020 isolate multithreaded error
read_bi5_to_bin_lock.lock();
BOOST_LOG((*mHTTPRequest_LoggingInstance_shared_pointer).mloggerCoutLog) << "Error: couldn't access the data .bin file. |"
<< filename_2_bin << "|" << std::endl;
read_bi5_to_bin_lock.unlock();
return 2;
}
n47::tick_data_iterator iter;
//5-11-2020 here i will save file.csv from data which is pointer to vector to pointers to ticks
if (data == 0) {
//??5-17-2020 isolate multithreaded error
read_bi5_to_bin_lock.lock();
BOOST_LOG((*mHTTPRequest_LoggingInstance_shared_pointer).mloggerCoutLog) << "Failure: Failed to load the data!" << std::endl;
read_bi5_to_bin_lock.unlock();
//5-14-2020 file is empty
//return 0;
}
//5-15-2020 take care that without else ,error happens with empty files because data is pointer to vector of pointers to ticks .so when data is made inside read_bi5 ,it is made as null pointer and later it is assigned to vector if file has ticks.if file does not have ticks ,then it is just returned as null pointer .so when dereferencing null pointer we got error
else if (data->size() != (raw_size / n47::ROW_SIZE)) {
//??5-17-2020 isolate multithreaded error
read_bi5_to_bin_lock.lock();
BOOST_LOG((*mHTTPRequest_LoggingInstance_shared_pointer).mloggerCoutLog) << "Failure: Loaded " << data->size()
<< " ticks but file size indicates we should have loaded "
<< (raw_size / n47::ROW_SIZE) << std::endl;
read_bi5_to_bin_lock.unlock();
//5-14-2020 file is empty
//return 0;
}
//??5-17-2020 isolate multithreaded error
read_bi5_to_bin_lock.lock();
BOOST_LOG((*mHTTPRequest_LoggingInstance_shared_pointer).mloggerCoutLog) << "time, bid, bid_vol, ask, ask_vol" << std::endl;
read_bi5_to_bin_lock.unlock();
counter = 0;
std::ofstream out_csv(filename_string_2_csv);
if (data == 0)
{
}
else if (data != 0)
{
////read_bi5_to_bin_lock.lock();
for (iter = data->begin(); iter != data->end(); iter++) {
//5-11-2020 here i will save file.csv from data which is pointer to vector to pointers to ticks>>>>>>>here i should open file stream for output and save data to it
out_csv << ((*iter)->epoch + (*iter)->td) << ", "
<< (*iter)->bid << ", " << (*iter)->bidv << ", "
<< (*iter)->ask << ", " << (*iter)->askv << std::endl;
//??5-17-2020 isolate multithreaded error
read_bi5_to_bin_lock.lock();
BOOST_LOG((*mHTTPRequest_LoggingInstance_shared_pointer).mloggerCoutLog) <<
boost::this_thread::get_id() << "\t"<<((*iter)->epoch + (*iter)->td) << ", "
<< (*iter)->bid << ", " << (*iter)->bidv << ", "
<< (*iter)->ask << ", " << (*iter)->askv << std::endl;
read_bi5_to_bin_lock.unlock();
counter++;
}
////read_bi5_to_bin_lock.unlock();
}
out_csv.close();
//5-13-2020
//??5-17-2020 isolate multithreaded error
read_bi5_to_bin_lock.lock();
BOOST_LOG((*mHTTPRequest_LoggingInstance_shared_pointer).mloggerCoutLog) << ".end." << std::endl << std::endl
<< "From " << raw_size << " bytes we read " << counter
<< " records." << std::endl
<< raw_size << " / " << n47::ROW_SIZE << " = "
<< (raw_size / n47::ROW_SIZE) << std::endl;
read_bi5_to_bin_lock.unlock();
delete data;
delete[] buffer;
delete [] data_bin_buffer;
return 0;
}
then inside read_bi5_to_bin function ,there will be a call to another function n47::lzma::decompress . this is the code of read_bi5_to_bin:
tick_data* read_bi5_to_bin(
unsigned char *lzma_buffer, size_t lzma_buffer_size, pt::ptime epoch,
float point_value, size_t *bytes_read, unsigned char* buffer_decompressed) {
tick_data *result = 0;
// decompress
int status;
buffer_decompressed = n47::lzma::decompress(lzma_buffer,
lzma_buffer_size, &status, bytes_read);
if (status != N47_E_OK)
{
bytes_read = 0;
}
else {
// convert to tick data (with read_bin).
result = read_bin(buffer_decompressed, *bytes_read, epoch, point_value);
//delete[] buffer;
}
return result;
}
then inside n47::lzma::decompress there will be making an array on heap called outBuffer .this buffer i need to retrieve at read_bi5_main in data_bin_buffer
this is code for n47::lzma::decompress
unsigned char *decompress(
unsigned char *inBuffer, size_t inSize, int *status, size_t *outSize) {
unsigned char *outBuffer = 0;
elzma_file_format format = ELZMA_lzma;
elzma_decompress_handle handle;
handle = elzma_decompress_alloc();
if (handle == 0) {
*status = -1;
} else {
// decompression...
datastream ds(inBuffer, inSize);
*status = elzma_decompress_run(
handle,
inputCallback, static_cast<void*>(&ds),
outputCallback, static_cast <void*>(&ds),
format);
if (*status == ELZMA_E_OK) {
*outSize = ds.outData.size();
outBuffer = new unsigned char[ ds.outData.size() ];
std::copy(ds.outData.begin(), ds.outData.end(), outBuffer);
}
elzma_decompress_free(&handle);
}
return outBuffer;
}
when i run this it gives error.a very helpful one here told me that it is the data_bin_buffer which is the problem because it is just one byte.and advised me to avoid shared pointer.when i convert it to normal pointer it gives another error if initialzed it to nullptr.should i not initialize the pointer???
Your read_bi5_to_bin code is broken.
tick_data* read_bi5_to_bin(
unsigned char *lzma_buffer, size_t lzma_buffer_size, pt::ptime epoch,
float point_value, size_t *bytes_read, unsigned char* buffer_decompressed) {
tick_data *result = 0;
The code receives a pointer called buffer_decompressed.
// decompress
int status;
buffer_decompressed = n47::lzma::decompress(lzma_buffer,
lzma_buffer_size, &status, bytes_read);
Then it throws away the value it received and gets some other value.
if (status != N47_E_OK)
{
bytes_read = 0;
}
else {
// convert to tick data (with read_bin).
result = read_bin(buffer_decompressed, *bytes_read, epoch, point_value);
//delete[] buffer;
}
return result;
}
And it never returns or frees the value it stored in buffer_decompressed.
Ooops.
I found the problem in my code.
it is the way i pass pointer to unsigned char to read_bi5_to_bin .
at first I thought passing the pointer variable is correct"as if i am passing lvalue reference" but i was so much wrong.reference is different than pointer.
pointer variable is lvalue variable of type pointer.so it is copied into function making new lvalue variable in function .the new variable has same value of pointer passed.this value is the address to first unsigned char in array.
but what i really need is to pass pointer to pointer to unsigned char.then pointer to pointer will be copied into function.the value of copied variable will be address of pointer to unsigned char.this is what i need retrieve from function.
I used & operator to get pointer to pointer .then I read very useful comment suggesting using ** so i changed the code to use **.voila,it works
this is the modified code:
read_bi5_to_bin_lock2.lock();
//5-20-2020
//boost::shared_ptr<unsigned char> data_bin_buffer = boost::make_shared<unsigned char>() ;
//n47::tick_data *data = n47::read_bi5_to_bin(
// buffer, buffer_size, epoch, PV, &raw_size, data_bin_buffer.get());
unsigned char *data_bin_buffer = 0 ;
n47::tick_data *data = n47::read_bi5_to_bin(
buffer, buffer_size, epoch, PV, &raw_size, &data_bin_buffer);
//5-11-2020 here i will save binary file
//boost::unique_lock<boost::mutex> read_bi5_to_bin_lock(m_read_bi5_to_binMutex);
std::string file_name_path_string=output_compressed_file_2(&data_bin_buffer, raw_size, filename_2_bin);
read_bi5_to_bin_lock2.unlock();
and this is the modified code in read_bi5_to_bin:
tick_data* read_bi5_to_bin(
unsigned char *lzma_buffer, size_t lzma_buffer_size, pt::ptime epoch,
float point_value, size_t *bytes_read, unsigned char** buffer_decompressed) {
tick_data *result = 0;
// decompress
int status;
*buffer_decompressed = n47::lzma::decompress(lzma_buffer,
lzma_buffer_size, &status, bytes_read);
thanks alot to DavidSchwartz for his very helpful comment.
i have a problem with one assignment that i have. I have to read a .ts file, read the packets that are inside and extract header information from each packet.
I have created a struct Packet that will hold all the info of the header, and i also have a vector in which i will push_back each Packet.
The problem is that the for loop stops for some reason on the 163rd loop. If i loop until lets say i=160, then the code escapes ends the loop, but when i print the vector.size() i get a really huge number which doesn't make sense. i guess it should be an integer value as high as the pushed back number of Packets.Here is the code that i have so far:
int main() {
FILE *ts_file = NULL;
ts_file = fopen64("/home/ddd/Desktop/Assignment/Streams/ddd.ts", "rb");
if (ts_file == NULL){
cout << "No file detected on this path, try again" << endl; // prints !!!Hello World!!!
}
TS_Analyzer *ts_analyzer;
ts_analyzer->parse_file(ts_file);
cout << "Finished main" << endl;
return 0;
}
void TS_Analyzer::parse_file(FILE *ts_file){
cout << "Inside parser" << endl;
fseek(ts_file,0,SEEK_END);
long file_size = ftell(ts_file);
rewind (ts_file);
number_of_packets = file_size/PACKET_SIZE;
unsigned int current_header_add = 0;
unsigned int i=0;
for (unsigned int j=1; i<number_of_packets; j++)
{
i++;
unsigned char TS_raw_header[4];
cout << "current position " << int(current_header_add) << endl;
current_header_add = ftell(ts_file);
fread(&TS_raw_header, sizeof(TS_raw_header), 1, ts_file);
Packet current_packet;
current_packet.sync_byte = TS_raw_header[0];
current_packet.transport_error_indicator = (TS_raw_header[1] & 0x80) >> 7;
current_packet.payload_start_indicator = (TS_raw_header[1] & 0x40) >> 6;
current_packet.transport_priority = (TS_raw_header[1] & 0x20) >> 5;
current_packet.PID = ((TS_raw_header[1] & 31) << 8) | TS_raw_header[2];
current_packet.transport_scrambling_control = (TS_raw_header[3] & 0xC0);
current_packet.adaption_field_control = (TS_raw_header[3] & 0x30) >> 4;
current_packet.continuity_counter = (TS_raw_header[3] & 0xF);
stream_packets.push_back(current_packet);
//cout << hex << int(current_packet.PID) << endl;
//cout << dec << "continuity counter " << int(current_packet.continuity_counter) << endl;
cout << " i " << int(i) << endl;
fseek(ts_file, 184, SEEK_CUR);
}
cout << "##" << endl;
cout << stream_packets.size() << endl;
}
class TS_Analyzer: public Analyzer {
public:
TS_Analyzer();
~TS_Analyzer();
struct Packet {
unsigned char sync_byte;
unsigned char transport_error_indicator;
unsigned char payload_start_indicator;
unsigned char transport_priority;
unsigned int PID;
unsigned char transport_scrambling_control;
unsigned char adaption_field_control;
unsigned char continuity_counter;
};
std::vector<Packet>stream_packets;
int number_of_packets = 0;
void parse_file(FILE *);
};
Any ideas of why the vector push_back breaks the for loop and why i cannot get a correct vector size?
If I put this code through the clang compiler, I get an error on following code:
TS_Analyzer *ts_analyzer;
ts_analyzer->parse_file(ts_file);
>> variable 'ts_analyzer' is uninitialized when used here
I guess you are encountering undefined behavior: As ts_analyzer as ptr is any random value, the data in its members is also very random.
I'm actually surprised that this code runs at all without crashing, though you can always be lucky.
If you like to fix this, try avoiding pointers by creating the object at the stack:
TS_Analyzer ts_analyzer;
ts_analyzer.parse_file(ts_file);
or if you really need allocated memory, at least fill in the pointer:
auto ts_analyzer = std::make_unique<TS_Analyzer>();
ts_analyzer->parse_file(ts_file);
I have a problem I neither can solve on my own nor find answer anywhere. I have a file contains such a string:
01000000d08c9ddf0115d1118c7a00c04
I would like to read the file in the way, that I would do manually like that:
char fromFile[] =
"\x01\x00\x00\x00\xd0\x8c\x9d\xdf\x011\x5d\x11\x18\xc7\xa0\x0c\x04";
I would really appreciate any help.
I want to do it in C++ (the best would be vc++).
Thank You!
int t194(void)
{
// imagine you have n pair of char, for simplicity,
// here n is 3 (you should recognize them)
char pair1[] = "01"; // note:
char pair2[] = "8c"; // initialize with 3 char c-style strings
char pair3[] = "c7"; //
{
// let us put these into a ram based stream, with spaces
std::stringstream ss;
ss << pair1 << " " << pair2 << " " << pair3;
// each pair can now be extracted into
// pre-declared int vars
int i1 = 0;
int i2 = 0;
int i3 = 0;
// use formatted extractor to convert
ss >> i1 >> i2 >> i3;
// show what happened (for debug only)
std::cout << "Confirm1:" << std::endl;
std::cout << "i1: " << i1 << std::endl;
std::cout << "i2: " << i2 << std::endl;
std::cout << "i3: " << i3 << std::endl << std::endl;
// output is:
// Confirm1:
// i1: 1
// i2: 8
// i3: 0
// Shucks, not correct.
// We know the default radix is base 10
// I hope you can see that the input radix is wrong,
// because c is not a decimal digit,
// the i2 and i3 conversions stops before the 'c'
}
// pre-delcare
int i1 = 0;
int i2 = 0;
int i3 = 0;
{
// so we try again, with radix info added
std::stringstream ss;
ss << pair1 << " " << pair2 << " " << pair3;
// strings are already in hex, so we use them as is
ss >> std::hex // change radix to 16
>> i1 >> i2 >> i3;
// now show what happened
std::cout << "Confirm2:" << std::endl;
std::cout << "i1: " << i1 << std::endl;
std::cout << "i2: " << i2 << std::endl;
std::cout << "i3: " << i3 << std::endl << std::endl;
// output now:
// i1: 1
// i2: 140
// i3: 199
// not what you expected? Though correct,
// now we can see we have the wrong radix for output
// add output radix to cout stream
std::cout << std::hex // add radix info here!
<< "i1: " << i1 << std::endl
// Note: only need to do once for std::cout
<< "i2: " << i2 << std::endl
<< "i3: " << i3 << std::endl << std::endl
<< std::dec;
// output now looks correct, and easily comparable to input
// i1: 1
// i2: 8c
// i3: c7
// So: What next?
// read the entire string of hex input into a single string
// separate this into pairs of chars (perhaps using
// string::substr())
// put space separated pairs into stringstream ss
// extract hex values until ss.eof()
// probably should add error checks
// and, of course, figure out how to use a loop for these steps
//
// alternative to consider:
// read 1 char at a time, build a pairing, convert, repeat
}
//
// Eventually, you should get far enough to discover that the
// extracts I have done are integers, but you want to pack them
// into an array of binary bytes.
//
// You can go back, and recode to extract bytes (either
// unsigned char or uint8_t), which you might find interesting.
//
// Or ... because your input is hex, and the largest 2 char
// value will be 0xff, and this fits into a single byte, you
// can simply static_cast them (I use unsigned char)
unsigned char bin[] = {static_cast<unsigned char>(i1),
static_cast<unsigned char>(i2),
static_cast<unsigned char>(i3) };
// Now confirm by casting these back to ints to cout
std::cout << "Confirm4: "
<< std::hex << std::setw(2) << std::setfill('0')
<< static_cast<int>(bin[0]) << " "
<< static_cast<int>(bin[1]) << " "
<< static_cast<int>(bin[2]) << std::endl;
// you also might consider a vector (and i prefer uint8_t)
// because push_back operations does a lot of hidden work for you
std::vector<uint8_t> bytes;
bytes.push_back(static_cast<uint8_t>(i1));
bytes.push_back(static_cast<uint8_t>(i2));
bytes.push_back(static_cast<uint8_t>(i3));
// confirm
std::cout << "\nConfirm5: ";
for (size_t i=0; i<bytes.size(); ++i)
std::cout << std::hex << std::setw(2) << std::setfill(' ')
<< static_cast<int>(bytes[i]) << " ";
std::cout << std::endl;
Note: The cout (or ss) of bytes or char can be confusing, not always giving the result you might expect. My background is embedded software, and I have surprisingly small experience making stream i/o of bytes work. Just saying this tends to bias my work when dealing with stream i/o.
// other considerations:
//
// you might read 1 char at a time. this can simplify
// your loop, possibly easier to debug
// ... would you have to detect and remove eoln? i.e. '\n'
// ... how would you handle a bad input
// such as not hex char, odd char count in a line
//
// I would probably prefer to use getline(),
// it will read until eoln(), and discard the '\n'
// then in each string, loop char by char, creating char pairs, etc.
//
// Converting a vector<uint8_t> to char bytes[] can be an easier
// effort in some ways. A vector<> guarantees that all the values
// contained are 'packed' back-to-back, and contiguous in
// memory, just right for binary stream output
//
// vector.size() tells how many chars have been pushed
//
// NOTE: the formatted 'insert' operator ('<<') can not
// transfer binary data to a stream. You must use
// stream::write() for binary output.
//
std::stringstream ssOut;
// possible approach:
// 1 step reinterpret_cast
// - a binary block output requires "const char*"
const char* myBuff = reinterpret_cast<const char*>(&myBytes.front());
ssOut.write(myBuff, myBytes.size());
// block write puts binary info into stream
// confirm
std::cout << "\nConfirm6: ";
std::string s = ssOut.str(); // string with binary data
for (size_t i=0; i<s.size(); ++i)
{
// because binary data is _not_ signed data,
// we need to 'cancel' the sign bit
unsigned char ukar = static_cast<unsigned char>(s[i]);
// because formatted output would interpret some chars
// (like null, or \n), we cast to int
int intVal = static_cast<int>(ukar);
// cast does not generate code
// now the formatted 'insert' operator
// converts and displays what we want
std::cout << std::hex << std::setw(2) << std::setfill('0')
<< intVal << " ";
}
std::cout << std::endl;
//
//
return (0);
} // int t194(void)
The below snippet should be helpful!
std::ifstream input( "filePath", std::ios::binary );
std::vector<char> hex((
std::istreambuf_iterator<char>(input)),
(std::istreambuf_iterator<char>()));
std::vector<char> bytes;
for (unsigned int i = 0; i < hex.size(); i += 2) {
std::string byteString = hex.substr(i, 2);
char byte = (char) strtol(byteString.c_str(), NULL, 16);
bytes.push_back(byte);
}
char* byteArr = bytes.data()
The way I understand your question is that you want just the binary representation of the numbers, i.e. remove the ascii (or ebcdic) part. Your output array will be half the length of the input array.
Here is some crude pseudo code.
For each input char c:
if (isdigit(c)) c -= '0';
else if (isxdigit(c) c -= 'a' + 0xa; //Need to check for isupper or islower)
Then, depending on the index of c in your input array:
if (! index % 2) output[outputindex] = (c << 4) & 0xf0;
else output[outputindex++] = c & 0x0f;
Here is a function that takes a string as in your description, and outputs a string that has \x in front of each digit.
#include <iostream>
#include <algorithm>
#include <string>
std::string convertHex(const std::string& str)
{
std::string retVal;
std::string hexPrefix = "\\x";
if (!str.empty())
{
std::string::const_iterator it = str.begin();
do
{
if (std::distance(it, str.end()) == 1)
{
retVal += hexPrefix + "0";
retVal += *(it);
++it;
}
else
{
retVal += hexPrefix + std::string(it, it+2);
it += 2;
}
} while (it != str.end());
}
return retVal;
}
using namespace std;
int main()
{
cout << convertHex("01000000d08c9ddf0115d1118c7a00c04") << endl;
cout << convertHex("015d");
}
Output:
\x01\x00\x00\x00\xd0\x8c\x9d\xdf\x01\x15\xd1\x11\x8c\x7a\x00\xc0\x04
\x01\x5d
Basically it is nothing more than a do-while loop. A string is built from each pair of characters encountered. If the number of characters left is 1 (meaning that there is only one digit), a "0" is added to the front of the digit.
I think I'd use a proxy class for reading and writing the data. Unfortunately, the code for the manipulators involved is just a little on the verbose side (to put it mildly).
#include <vector>
#include <algorithm>
#include <iterator>
#include <iostream>
#include <iomanip>
#include <string>
#include <sstream>
struct byte {
unsigned char ch;
friend std::istream &operator>>(std::istream &is, byte &b) {
std::string temp;
if (is >> std::setw(2) >> std::setprecision(2) >> temp)
b.ch = std::stoi(temp, 0, 16);
return is;
}
friend std::ostream &operator<<(std::ostream &os, byte const &b) {
return os << "\\x" << std::setw(2) << std::setfill('0') << std::setprecision(2) << std::hex << (int)b.ch;
}
};
int main() {
std::istringstream input("01000000d08c9ddf115d1118c7a00c04");
std::ostringstream result;
std::istream_iterator<byte> in(input), end;
std::ostream_iterator<byte> out(result);
std::copy(in, end, out);
std::cout << result.str();
}
I do really dislike how verbose iomanipulators are, but other than that it seems pretty clean.
You can try a loop with fscanf
unsigned char b;
fscanf(pFile, "%2x", &b);
Edit:
#define MAX_LINE_SIZE 128
FILE* pFile = fopen(...);
char fromFile[MAX_LINE_SIZE] = {0};
char b = 0;
int currentIndex = 0;
while (fscanf (pFile, "%2x", &b) > 0 && i < MAX_LINE_SIZE)
fromFile[currentIndex++] = b;
I'm using Cryptopp to generate a random string.
This is the code:
const unsigned int BLOCKSIZE = 16 * 8;
byte pcbScratch[ BLOCKSIZE ];
// Construction
// Using a ANSI approved Cipher
CryptoPP::AutoSeededX917RNG<CryptoPP::DES_EDE3> rng;
rng.GenerateBlock( pcbScratch, BLOCKSIZE );
// Output
std::cout << "The generated random block is:" << std::endl;
string str = "";
for( unsigned int i = 0; i < BLOCKSIZE; i++ )
{
std::cout << "0x" << std::setbase(16) << std::setw(2) << std::setfill('0');
std::cout << static_cast<unsigned int>( pcbScratch[ i ] ) << " ";
str += pcbScratch[i];
}
std::cout << std::endl;
std::cout << str <<std::endl;
I've put int the code a new var: string str = "".
Then in the for append for each result, the part of the string.
But my output is dirty! I see only strange ASCII char.
How can I set well the string?
Thank you.
You will want to some output encoding, e.g.
base64
hex
because what you are seeing is the raw binary data, interpreted as if it were text. Random characters are the consequence
AFAICT (google) you should be able to use something like this
#include <base64.h>
string base64encoded;
StringSource(str, true, new Base64Encoder(new StringSink(base64encoded)));
Appending arbitrary bytes (chars) to the end of a string is going to result in that containing some non-printable characters:
http://en.wikipedia.org/wiki/Control_character
You don't mention what you wanted or expected. Did you want the string to be the same as what got sent to std::cout? If so, you can use a stringstream via #include <sstream>:
std::stringstream ss;
for( unsigned int i = 0; i < BLOCKSIZE; i++ )
{
ss << "0x" << std::setbase(16) << std::setw(2) << std::setfill('0');
ss << static_cast<unsigned int>(pcbScratch[i]);
}
str = ss.str();
You can also use Crypto++'s built in HexEncoder:
std::cout << "The generated random block is:" << std::endl;
string str = "0x";
StringSource ss(pcbScratch, BLOCKSIZE, true,
new HexEncoder(
new StringSink(str),
true, // uppercase
2, // grouping
" 0x" // separator
) // HexDecoder
); // StringSource
The StringSource 'owns' the HexEncoder, so there's no need to call delete.