Serialize encrypted password to XML - c++

I have an application that contains userbase which is stored in xml file and loaded at the start of the program. I use QXmlStreamWriter/Reader for this purpose. The problem occurs when I try to serialize encrypted (hashed?) form of password (using QCryptographicHash and Sha256 for this).
QCryptographicHash return QByteArray, which can be converted to QString (necessary for use of QXmlStreamWriter/Reader). Relevant code below. Everything works fine before the serialization (I can log in), but when I read data from xml, after finding hashed password the function behaves like it found the EOF, and only about 2 chars are loaded to the QString by QXmlStreamReader.
In the code please ignore reservations etc (it's a cinema panel), relevant fragments are passwords, I supply full function just in case.
I hope I explained what the issue is, here are fragments of my code (note: before adding the hashing everything worked fine)
Register function (the hashing, pass is a QString):
QString hash = QCryptographicHash::hash(pass.toUtf8(), QCryptographicHash::Sha256);
User* user_pointer;
user_pointer = new User(name, hash, admin);
Writing function:
QFile file("users/users.xml");
if(!file.open(QIODevice::WriteOnly))
throw "Error podczas otwierania bazy użytkowników!";
QXmlStreamWriter writer (&file);
writer.setAutoFormatting(true);
writer.writeStartDocument();
writer.writeStartElement("USERS");
int list_size = userList.size();
for(int i = 0; i < list_size; i++)
{
writer.writeStartElement("USER");
writer.writeTextElement("name", userList.at(i)->name);
writer.writeTextElement("pass", userList.at(i)->password);
writer.writeTextElement("admin", QString::number(userList.at(i)->is_admin));
writer.writeStartElement("RESERVATIONS");
for(int m = 0; m < userList.at(i)->reservList.size(); m++)
{
writer.writeStartElement("reservation");
writer.writeTextElement("moviename", userList.at(i)->reservList.at(m)->movie_name);
writer.writeTextElement("date", userList.at(i)->reservList.at(m)->date.toString("dd.MM.yyyy"));
writer.writeTextElement("hour", (userList.at(i)->reservList.at(m)->hour).toString("hhmm"));
writer.writeTextElement("paid", QString::number(userList.at(i)->reservList.at(m)->paid));
for(int n = 0; n < userList.at(i)->reservList.at(m)->placeList.size(); n++)
writer.writeTextElement("place", QString::number(userList.at(i)->reservList.at(m)->placeList.at(n)));
writer.writeEndElement();
}
writer.writeEndElement();
writer.writeEndElement();
}
writer.writeEndDocument();
file.close();
}
Reading function:
QFile file("users/users.xml");
if(!file.open(QIODevice::ReadOnly))
throw "Brak bazy danych użytkowników lub błąd jej otworzenia!";
QXmlStreamReader reader;
reader.setDevice(&file);
reader.readNext();
QString user_name;
QString user_pass;
bool admin;
QString movie_name;
QTime hour;
QDate date;
bool paid;
User* user_pointer = NULL;
int user_counter = -1;
Reservation* reserv_pointer = NULL;
int reserv_counter = -1;
while(!reader.atEnd())
{
if(reader.isStartElement())
{
if(reader.name() == "USER")
{
reserv_counter = -1;
}
if(reader.name() == "name")
user_name = reader.readElementText();
if(reader.name() == "pass")
user_pass = reader.readElementText();
if(reader.name() == "admin")
{
admin = reader.readElementText().toInt();
user_pointer = new User(user_name, user_pass, admin);
userList.append(user_pointer);
user_counter++;
}
if(reader.name() == "reservation")
{
reserv_counter++;
}
if(reader.name() == "moviename")
movie_name = reader.readElementText();
if(reader.name() == "hour")
hour = QTime::fromString(reader.readElementText(), "hhmm");
if(reader.name() == "date")
date = QDate::fromString(reader.readElementText(), "dd.MM.yyyy");
if(reader.name() == "paid")
{
paid = reader.readElementText().toInt();
reserv_pointer = new Reservation(movie_name, date, hour, paid);
userList.at(user_counter)->reservList.append(reserv_pointer);
}
if(reader.name() == "place")
{
userList.at(user_counter)->reservList.at(reserv_counter)->placeList.append(reader.readElementText().toInt());
}
reader.readNextStartElement();
}
else
reader.readNext();
}
file.close();
}

The hash value is not a string, it is a sequence of arbitrary byte values, some of those might happen to be problematic when interpreting it as a string.
You have an implicit conversion from QByteArray to QString, for which the documentation says that:
The byte array is converted to Unicode using the fromUtf8() function.
This function stops conversion at the first NUL character found, or
the end of the ba byte array.
You could for example use explicit conversion which specifies the length:
QString::fromUtf8(byteArray.data(), length);
As Frank Osterfeld noted in the comments, using UTF8 is not a good idea, I have tested from and to Latin1 extensively for a project I was working on, and the binary data is identical, however in looks "funky" in textual form, which may not play well with the XML reading and writing, toHex() will fix that by limiting the character set to 0-F:
QByteArray b; // hash
QString ss = QString::fromLatin1(b.toHex()); // to QString
b = QByteArray::fromHex(ss.toLatin1()); // back to QByteArray

Your problem is that XML is a text format but encrypted passwords are a binary format. The two are not compatible. You need to have some way to encode the binary data in a text-friendly format.
As #ddriver has mentioned, one way to do this is to use QByteArray::toHex() since it will convert all bytes to human readable text characters. The cost, however, is a 100% increase in size (2 characters returned for each byte of the password hash)
Another, more ubiquitous, and efficient, method for transmitting binary data in textual form is to use the QByteArray::toBase64(). While it also returns binary data in a textual form, there's only a 33 1/3% increase in size (4 bytes returned for every 3 bytes of password hash).
(You might recognise this encoding since its the nonsensical text that usually ends with one or two = characters, and is the usual encoding used to transmit binary data in emails)

Related

Losing a value from pointer vector after loop that reads the file (using dirent)

I'm writing a program in C++ that behaves like a Twitter Service. The program is supposed to store data in files as follows:
f_username (stores the users that username follows), so it creates a file for each user that follows someone. I have the function load_users() that reads each file to populate all clients (users) information in a vector called client_db (global variable) of type struct (defined below). So, when a f_username file is read, the vectors client_followers and client_following are populated for each user. The function works perfectly, except for the first file it reads. It saves the information in client_followers and client_following in the while loop that reads the file, but the entries for both vectors are deleted somehow after this loop. I added the comments in the code where I'm facing this issue only for the first f_username file it reads.
I believe it has something to do with the fact I'm using pointers, but I have no clue how to fix this issue. This is the first time ever I have this weird logic error. How is possible that it only "deletes" the data from the vectors that is read from first file, but not from the other files (I have 3 files)?
Any help would be really appreciated, I tried my best to explain this problem.
**UPDATE: Problem fixed by replacing the types of vectors client_followers and client_following with a string type without declaring them as pointers, and making sure the actual entries in client_db are updated correctly. **
//Client struct that holds a user's username, followers, and users they follow
struct Client {
std::string username;
bool connected = false;
int following_file_size = 0;
std::vector<Client*> client_followers;
std::vector<Client*> client_following;
ServerReaderWriter<Message, Message>* stream = 0;
bool operator==(const Client& c1) const{
return (username == c1.username);
}
};
void load_users()
{
//load users (followers and following)
DIR *dir;
struct dirent *ent;
if ((dir = opendir ("./")) != NULL) {
while ((ent = readdir (dir)) != NULL) {
std::string filename = ent->d_name;
int index = filename.find("f_");
if(index != std::string::npos)
{
std::ifstream in; //read following file of username
//get username
std::string user = filename.substr(index+2);
int txt_index = user.find(".txt");
user.erase(txt_index,4);
int user_index = find_user(user); //check if user is in client's db already
//client_db is a global variable
if(user_index < 0){ //if not, add client to client_db
Client c;
c.username = user;
client_db.push_back(c);
}
Client* user1 = &client_db[find_user(user)];
std::string username2;
in.open("./" + filename);
while(!in.eof())
{
getline(in,username2);
int following_index = find_user(username2);
if(following_index < 0) //create entry for the user
{
Client c2;
c2.username = username2;
client_db.push_back(c2);
}
Client* user2 = &client_db[find_user(username2)];
//it adds the information for the first file, but entries in client_followers
//and client_following is deleted after this loop ends, how is this possible?
user2->client_followers.push_back(user1);
user1->client_following.push_back(user2);
}
//When I print information of client_following and client_followers, the information obtained from the first file is not printed (e.g. size of client_following is 0).
in.close();
//break;
}
}
closedir (dir);
} else {
/* could not open directory */
//perror ("");
//return EXIT_FAILURE;
return;
}
}

How to retrieve integer value from JSON data in Qt

How can I retrieve the integer value from the QJsonValue object? Suppose that I have the following JSON data:
{
"res": 1,
"message": "Common error"
}
I need to extract the "res" value from this data, so I've tried to use the following code:
QJsonDocument d = QJsonDocument::fromJson(some_json_data.toUtf8());
QJsonObject root_object = d.object();
QJsonValue res = root_object.value("res");
But I've found that QJsonValue class doesn't have a member function toInt or something like this (there's toDouble, toString, etc only). What can I do in such a situation? What is the best way to extract integer values via QjsonValue class?
(tl;dr: a one-liner solution at the end of answer.)
First a comprehensive way giving you full control. Below code assumes int is enough range for your integers, but could be expanded to work for most of the range of int64_t (but better test the boundary cases to get it fully correct):
QJsonValue res = root_object.value("res");
int result = 0;
double tmp = res.toDouble();
if (tmp >= std::numeric_limits<int>::min() && // value is not too small
tmp <= std::numeric_limits<int>::max() && // value is not too big
std::floor(tmp) == tmp // value does not have decimals, if it's not ok
) {
result = std::floor(tmp); // let's be specific about rounding, if decimals are ok
} else {
// error handling if you are not fine with 0 as default value
}
A shorter way using QVariant, and as an example also getting result into a larger integer type, if you just want to let Qt do it's thing. I'm not sure how it it handles integer values which are too big for a double to handle accurately, so again if that is important, better test.
QJsonValue res = root_object.value("res");
QVariant tmp = res.toVariant();
bool ok = false;
qlonglong result = tmp.toLongLong(&ok);
if (!ok) {
// error handling if you are not fine with 0 as default value
}
or same as error-ignoring one-liner, change integer type as appropriate:
qlonglong result = root_object.value("res").toVariant().toLongLong();
int QJsonValue::toInt(int defaultValue = 0) const
Converts the value to an int and returns it.
If type() is not Double or the value is not a whole number, the
defaultValue will be returned.
QByteArray JSettings::read_file(QString path)
{
QFile file;
file.setFileName(path);
if (!file.open(QIODevice::ReadOnly | QIODevice::Text))
return QByteArray();
QByteArray barray = file.readAll();
file.close();
return barray;
}
QVariantMap JSettings::to_map(QByteArray json)
{
QJsonDocument jdoc = QJsonDocument::fromJson(json);
QVariantMap vmap = jdoc.toVariant().toMap();
// qDebug() << vmap["res"].toInt();
return vmap;
}
QJsonObject JSettings::to_object(QByteArray json)
{
QJsonDocument jdoc = QJsonDocument::fromJson(json);
QJsonObject obj = jdoc.object();
// qDebug() << obj["res"].toInt();
return obj;
}
Qt JSON portal: http://doc.qt.io/qt-5/json.html
QJsonDocument d = QJsonDocument::fromJson(some_json_data.toUtf8());
QJsonObject root_object = d.object();
QJsonValue res = root_object.value("res");
int i = res.toString().toInt();
this is working, QString to Int

Loading Local Content in Awesomium

I have written a local DataSource because from what I know there are none included in Awesomium, but the thing is that it request everything from the data source html, images etc
And I have no clue on how I should load all types of mime formats.
My current code only supports html/text, where I load the file into binary and send as a response. This does not work for images.
Does anybody know where I should go on from here?
class LocalDataSource :
public Awesomium::DataSource
{
public:
LocalDataSource() { }
virtual ~LocalDataSource() { }
virtual void OnRequest(int request_id, const Awesomium::WebString& path)
{
std::string filepath = Awesomium::ToString(path).insert(0, "./");
std::basic_ifstream<char> is(filepath, std::ios_base::in | std::ios_base::binary);
if (is)
{
is.seekg(0, is.end);
int length = is.tellg();
is.seekg(0, is.beg);
char *buffer = new char[length + 1];
is.read(buffer, length);
buffer[length] = '\0';
is.close();
SendResponse(request_id, strlen(buffer), (unsigned char*)buffer, Awesomium::WSLit("text/html"));
delete[] buffer;
}
else
{
// Error
}
}
};
EDIT:
for now I will load the file relative to the executable and not use DataSource's.
I know this is old, but it was relevant to me, I fixed this the same way as Steven did, I will post the C++ code I used:
bool ResInterceptor::OnFilterNavigation(int origin_process, int origin_routing_id, const Awesomium::WebString& method, const Awesomium::WebURL& url, bool is_main_frame)
{
return false;
}
Awesomium::ResourceResponse* ResInterceptor::OnRequest(Awesomium::ResourceRequest* request)
{
bool isAsset = std::strcmp(ToString(request->url().scheme()).c_str(), "asset")==0;
bool isFile = std::strcmp(ToString(request->url().scheme()).c_str(), "file")==0;
if(!isAsset && !isFile)
{
//if it is neither of these we "may" still intercept the call, this allows for offline-online versions to work
return Awesomium::ResourceInterceptor::OnRequest(request);
}
if(isAsset)
{
//Blah blah, do whatever
}
else if(isFile)
{
//Blah blah, same
}
//As you can see this isn't very, but it worked for my purposes
std::string contentpath = "E:/Location/of/files" + ToString(request->url().path());
Awesomium::WebString datatype;
std::string filename = Awesomium::ToString(request->url().filename());
//I still want to check for the correct mime type
if (has_suffix(filename, ".html")) datatype = Awesomium::WSLit("text/html");
else if(has_suffix(filename, ".js")) datatype = Awesomium::WSLit("text/javascript");
else if(has_suffix(filename, ".css")) datatype = Awesomium::WSLit("text/css");
else if(has_suffix(filename, ".swf")) datatype = Awesomium::WSLit("application/x-shockwave-flash");
else if(has_suffix(filename, ".zip")) datatype = Awesomium::WSLit("application/zip");
else if(has_suffix(filename, ".txt")) datatype = Awesomium::WSLit("text/plain");
else if(has_suffix(filename, ".text")) datatype = Awesomium::WSLit("text/plain");
else if(has_suffix(filename, ".png")) datatype = Awesomium::WSLit("image/png");
else if(has_suffix(filename, ".jpeg")) datatype = Awesomium::WSLit("image/jpeg");
else if(has_suffix(filename, ".jpg")) datatype = Awesomium::WSLit("image/jpeg");
else if(has_suffix(filename, ".webm")) datatype = Awesomium::WSLit("video/webm");
else if(has_suffix(filename, ".mp4")) datatype = Awesomium::WSLit("video/mp4");
else if(has_suffix(filename, ".ogv")) datatype = Awesomium::WSLit("video/ogg");
else if(has_suffix(filename, ".flv")) datatype = Awesomium::WSLit("video/flv");
if(!datatype.IsEmpty())
{
FILE * pFile;
long lSize;
unsigned char * buffer;
size_t result;
pFile = fopen ( contentpath.c_str() , "rb" );
if (pFile!=NULL)
{
// obtain file size:
fseek (pFile , 0 , SEEK_END);
lSize = ftell (pFile);
rewind (pFile);
// allocate memory to contain the whole file:
buffer = (unsigned char*) malloc (sizeof(unsigned char)*lSize);
if (buffer == NULL) {fputs ("Memory error",stderr); exit (2);}
// copy the file into the buffer:
result = fread (buffer,1,lSize,pFile);
if (result != lSize) {fputs ("Reading error",stderr); exit (3);}
//This is where the magic happens!!
return Awesomium::ResourceResponse::Create(lSize, buffer, datatype);
// terminate
fclose (pFile);
free (buffer);
}
else
{
//send this off to the default request handler instead of it being a local file
return Awesomium::ResourceInterceptor::OnRequest(request);
}
}else
{
//send this off to the default request handler instead of it being a local file
return Awesomium::ResourceInterceptor::OnRequest(request);
}
}
//Support function
bool ResInterceptor::has_suffix(const std::string &str, const std::string &suffix)
{
return str.size() >= suffix.size() &&
str.compare(str.size() - suffix.size(), suffix.size(), suffix) == 0;
}
as for how I hooked it up, I simply added this line of code:
_web_core = WebCore::Initialize(config);
_web_core->set_resource_interceptor(new ResInterceptor());
This took me a whole night to nail down all because I was passing in a pointer with a variable and not using the "new" keyword directly! I got it now at least!
also note, I tried the exact same code above inside the LocalDataSource and it didn't work for anything except the text files, so I think there is a bug in there, good news is, this works the exact same way, but you get more control over every file request.
Thank you Steven for all the great reference code!
The easy way is to send the contents of a file without sweating mime type detection is to use the static method static ResourceResponse* Awesomium::ResourceResponse::Create.
From the Awesomium docs:
Create a ResourceResponse from a file on disk.
I couldn't figure out a way to map ResourceResponse::Create to DataSource::SendResponse.
As a workaround, you could rewrite your data source as an IResourceInterceptor instead of a DataSource. I wrote up a detailed example in C# on how to use http:// scheme instead of the custom asset:// scheme for embedded resources It should be pretty straightforward to translate the C# to C++. Below is an edited down version of my post (not tested).
using System;
using System.IO;
using System.Reflection;
using Awesomium.Core;
namespace MyApp
{
public class ResourceInterceptor : IResourceInterceptor
{
/// <summary>
/// Intercepts any requests for the EmbeddedResourceDomain base Uri,
/// and returns a response using the embedded resource in this app's assembly/DLL file
/// </summary>
public virtual ResourceResponse OnRequest(ResourceRequest request)
{
ResourceResponse response = null;
string resourceName;
string filePath;
filePath = String.Concat("./", request.Url.AbsolutePath);
filePath = Path.GetFullPath(resourceName.Replace('/', Path.DirectorySeparatorChar));
// cache the resource to a temp file if
if (File.Exists(filePath))
{
response = ResourceResponse.Create(filePath);
}
return response;
}
/// <summary>
/// Optionally blocks any web browser requests by returning true. Not used.
/// </summary>
/// <remarks>
/// This method can implement a whitelist of allowed URLs here by
/// returning true to block any whitelist misses
/// </remarks>
public virtual bool OnFilterNavigation(NavigationRequest request)
{
return false;
}
}
}
Another option might be to hack the HTML content to inject a <base href="file:///c:/my/bin/path" /> element inside the <head> of the document. You would need to modify the href attribute value before loading the content. This may be more work than it is worth.

Why is this encrypted message damaged?

I use the following code to encrypt a string with a key, using the 3-DES algorithm:
private bool Encode(string input, out string output, byte[] k, bool isDOS7)
{
try
{
if (k.Length != 16)
{
throw new Exception("Wrong key size exception");
}
int length = input.Length % 8;
if (length != 0)
{
length = 8 - length;
for (int i = 0; i < length; i++)
{
input += " ";
}
}
TripleDESCryptoServiceProvider des = new TripleDESCryptoServiceProvider();
des.Mode = CipherMode.ECB;
des.Padding = PaddingMode.Zeros;
des.Key = k;
ICryptoTransform ic = des.CreateEncryptor();
byte[] bytePlainText = Encoding.Default.GetBytes(input);
MemoryStream ms = new MemoryStream();
CryptoStream cStream = new CryptoStream(ms,
ic,
CryptoStreamMode.Write);
cStream.Write(bytePlainText, 0, bytePlainText.Length);
cStream.FlushFinalBlock();
byte[] cipherTextBytes = ms.ToArray();
cStream.Close();
ms.Close();
output = Encoding.Default.GetString(cipherTextBytes);
}
catch (ArgumentException e)
{
output = e.Message;
//Log.Instance.WriteToEvent("Problem encoding, terminalID= "+objTerminalSecurity.TerminalID+" ,Error" + output, "Security", EventLogEntryType.Error);
return false;
}
return true;
}
I send the output parameter as is over to a WCF http-binding webservice, and I noticed that the actual encoded string looks different, it looks like there are some \t and \n but the charachters are about the same.
What is going on, why does the server get a different encoded string?
Usually cipher text is base64 encoded in an effort to be binary safe during transmission.
Also I would not use 3DES with ECB. That is awful, you must have copy pasted this from somewhere. Use AES with cbc mode and think about adding a cmac or hmac.

RapidXML, reading and saving values

I've worked myself through the rapidXML sources and managed to read some values. Now I want to change them and save them to my XML file:
Parsing file and set a pointer
void SettingsHandler::getConfigFile() {
pcSourceConfig = parsing->readFileInChar(CONF);
cfg.parse<0>(pcSourceConfig);
}
Reading values from XML
void SettingsHandler::getDefinitions() {
SettingsHandler::getConfigFile();
stGeneral = cfg.first_node("settings")->value();
/* stGeneral = 60 */
}
Changing values and saving to file
void SettingsHandler::setDefinitions() {
SettingsHandler::getConfigFile();
stGeneral = "10";
cfg.first_node("settings")->value(stGeneral.c_str());
std::stringstream sStream;
sStream << *cfg.first_node();
std::ofstream ofFileToWrite;
ofFileToWrite.open(CONF, std::ios::trunc);
ofFileToWrite << "<?xml version=\"1.0\"?>\n" << sStream.str() << '\0';
ofFileToWrite.close();
}
Reading file into buffer
char* Parser::readFileInChar(const char* p_pccFile) {
char* cpBuffer;
size_t sSize;
std::ifstream ifFileToRead;
ifFileToRead.open(p_pccFile, std::ios::binary);
sSize = Parser::getFileLength(&ifFileToRead);
cpBuffer = new char[sSize];
ifFileToRead.read( cpBuffer, sSize);
ifFileToRead.close();
return cpBuffer;
}
However, it's not possible to save the new value. My code is just saving the original file with a value of "60" where it should be "10".
Rgds
Layne
I think this is a RapidXML Gotcha
Try adding the parse_no_data_nodes flag to cfg.parse<0>(pcSourceConfig)
You should definitely be testing that the output file opened correctly and that your write succeeded. At the simplest, you need something like:
if ( ! ofFileToWrite << "<?xml version=\"1.0\"?>\n"
<< sStream.str() << '\0' ) {
throw "write failed";
}
Note that you don't need the '\0' terminator, but it shouldn't do any harm.
Use the following method to add an attribute to a node. The method uses the allocation of memory for strings from rapidxml. So rapidxml takes care of the strings as long as the document is alive. See http://rapidxml.sourceforge.net/manual.html#namespacerapidxml_1modifying_dom_tree for further information.
void setStringAttribute(
xml_document<>& doc, xml_node<>* node,
const string& attributeName, const string& attributeValue)
{
// allocate memory assigned to document for attribute value
char* rapidAttributeValue = doc.allocate_string(attributeValue.c_str());
// search for the attribute at the given node
xml_attribute<>* attr = node->first_attribute(attributeName.c_str());
if (attr != 0) { // attribute already exists
// only change value of existing attribute
attr->value(rapidAttributeValue);
} else { // attribute does not exist
// allocate memory assigned to document for attribute name
char* rapidAttributeName = doc.allocate_string(attributeName.c_str());
// create new a new attribute with the given name and value
attr = doc.allocate_attribute(rapidAttributeName, rapidAttributeValue);
// append attribute to node
node->append_attribute(attr);
}
}