Casting QByteArray to `long` outputs different result for same input - c++

EDIT: Added full MCV example project.
I have a strange problem where the same code and same input produce different output values.
The purpose of the code is to test a function that takes a value packed into 4 bytes, and unpack it into a single 32bit value. The expected value of value1, value2 and value3 in test_unpack() is 2018915346 (i.e. 0x78563412 because of little-endian unpacking). I got this method of unpacking from another answer. Below is an MCV example that you can easily build and see the problem for yourself. Note that if you comment out the body of test1() test_unpack() magically passes with the correct value.
test_canserialcomm.cpp
#include "test_canserialcomm.h"
#include <QtTest/QtTest>
#include <QByteArray>
long unpack() noexcept
{
quint8 a_bytes[] = {0x12, 0x34, 0x56, 0x78};
QByteArray a = QByteArray(reinterpret_cast<char*>(a_bytes), 4);
long value1 = *((long*)a.data());
qDebug() << value1; // outputs "32651099317351442" (incorrect value)
quint8 b_bytes[] = {0x12, 0x34, 0x56, 0x78};
QByteArray b = QByteArray(reinterpret_cast<char*>(b_bytes), 4);
long value2 = *((long*)b.data());
qDebug() << value2; // outputs "2018915346" (correct value)
quint8 c_bytes[] = {0x12, 0x34, 0x56, 0x78};
QByteArray c = QByteArray(reinterpret_cast<char*>(c_bytes), 4);
long value3 = *((long*)c.data());
qDebug() << value3; // outputs "2018915346" (correct value)
return value1;
}
void TestCanSerialComm::test1()
{
QCOMPARE("aoeu", "aoeu"); // If you comment this line, the next test will pass, as expected.
}
void TestCanSerialComm::test_unpack()
{
long expected {0x78563412};
QCOMPARE(unpack(), expected);
}
test_canserialcomm.h
#ifndef TEST_CANSERIALCOMM_H
#define TEST_CANSERIALCOMM_H
#include <QtTest>
class TestCanSerialComm: public QObject
{
Q_OBJECT
private slots:
void test1();
void test_unpack();
};
#endif // TEST_CANSERIALCOMM_H
test_main.cpp
#include <QtTest>
#include "test_canserialcomm.h"
#include <QCoreApplication>
int main(int argc, char** argv) {
QCoreApplication app(argc, argv);
TestCanSerialComm testCanSerialComm;
// Execute test-runner.
return QTest::qExec(&testCanSerialComm, argc, argv); }
tmp.pro
QT += core \
testlib
QT -= gui
CONFIG += c++11
TARGET = tmp
CONFIG += console
CONFIG -= app_bundle
TEMPLATE = app
TARGET = UnitTests
HEADERS += test_canserialcomm.h
SOURCES += test_canserialcomm.cpp \
test_main.cpp
The output of value1 in test_unpack() is wrong, despite the same code and same inputs. Strangely, if I remove the qDebug() calls and set a breakpoint, the debugger expression evaluator now shows that value2 has the wrong value.
Any idea why this is happening? Or even how to troubleshoot this further?
Additional Notes: If I add a line qDebug() << "garbage"; at the top of my function, all 3 values produced are correct.

You're compiling and running this program on a system where long is 8 bytes, but your QByteArray has only 4 bytes. That means that when you alias the array as a long (using *((long*)a.data())) you're reading 4 bytes past the end of the array, into uninitialized heap storage.
The fix is to use a type that is guaranteed to be 4 bytes in size, e.g. std::int32_t.
As an aside, using *((long*)[...]) to alias memory is not guaranteed to work, primarily because of alignment issues but also (in the general case) because aliasing is only supported for types equivalent to char or a signed or unsigned variant. The safer technique is to use memcpy:
std::uint32_t value1;
assert(a.size() == sizeof(value1));
memcpy(&value1, a.data(), a.size());

Related

QString(const char* p) constructor misrepresents non ASCII characters

I am trying to upgrade a bigger C++ program from Qt4 to Qt5 or higher and have some problems with the legacy code that was written in ISO-LATIN1. From Qt5, the code is expected to be present in UTF8 and that is what we tried to do. We use a own String-class (let's call it myQString here), that was basically a char* under Qt4 and went to a QString-derived class in Qt5. So far so good.
The cases where I still have some problems is when I try to pass char* variables to the myQString class, that includes non-ASCII characters (like the letter characters with diaeresis for example, 'ä', 'Ä', 'ö', 'Ö', etc.).
I tried to write a mini program that reproduces/illustrates the problem. To make it clearer I could post some code but a picture would be better in this case:
Zoom Debugger output
Here we can see via the Debugger-View, that the desired end-products (cyan and yellow color: "mstr2, mstr4, qstr2, qstr4), that should be something that store "Äa", misrepresent the first byte "Ä". All of them use the green marked constructor, myQString(const char* p).
The function that illustrates the problem (in the Debugger) is charPointerToQstring(). It is part of the file main.cpp (see last code block).
If you want to run that "mini" program, I will also post all the files (four) needed to do so. I am using QtCreator and have a project file, which you can call how you like, let's say "testingQString.pro"
QT -= gui
CONFIG += c++17 console
SOURCES += \
main.cpp \
myqstring.cpp
HEADERS += \
myqstring.h
Then we have a "stripped" myQString class, with the two files:
myqstring.h:
#ifndef MYQSTRING_H
#define MYQSTRING_H
#include <QString>
class myQString : public QString
{
public:
myQString();
myQString(const QString& str);
myQString(char c);
myQString(const char* p);
myQString(const QByteArray& ba);
};
#endif // MYQSTRING_H
and "stripped" myqstring.cpp:
#include "myqstring.h"
#include <QDebug>
#define ENTER_FUNCTION qDebug() << "========== Entering:" << Q_FUNC_INFO
myQString::myQString()
{
ENTER_FUNCTION;
}
myQString::myQString(const QString& str) : QString(str)
{
ENTER_FUNCTION;
}
myQString::myQString(char c) : QString(QChar(c))
{
ENTER_FUNCTION;
}
myQString::myQString(const char* p) : QString(p)
{
ENTER_FUNCTION;
}
myQString::myQString(const QByteArray& ba)
{
ENTER_FUNCTION;
foreach (auto c, ba) {
#if QT_VERSION_MAJOR == 5
append(QChar(c));
#endif
#if QT_VERSION_MAJOR == 6
append(char(c));
#endif
}
}
The file main.cpp is also "stripped" here and only shows that one specific problem:
#include "myqstring.h"
#include <QDebug>
#include <array>
// -----------------------------------------------------------------------------
// -----------------------------------------------------------------------------
void charPointerToQstring() {
// case 1 - const char* with string as initialiser
const char* buf1("Äa");
myQString mstr1(buf1);
QString qstr1(buf1);
// case 2 - char* with char assignment
const int len = 2;
char* buf2 = new char[len+1];
buf2[0] = char(0xC4); // 0xC4 == 196 == AE (umlaut)
buf2[1] = 'a';
buf2[len] = '\0';
myQString mstr2(buf2);
QString qstr2(buf2);
// case 3 - str
myQString mstr3("Äa");
QString qstr3("Äa");
// case 4 - std::array<char>
std::array<char, len+1> stda1;
stda1[0] = char(0xC4);
stda1[1] = 'a';
stda1[len] = '\0';
myQString mstr4(stda1.data());
QString qstr4(stda1.data());
qDebug() << "Set a breakpoint exactly on ME (nr 3) and check the results via Debugger!!!";
}
// -----------------------------------------------------------------------------
// -----------------------------------------------------------------------------
int main(int argc, char *argv[])
{
Q_UNUSED(argc)
Q_UNUSED(argv)
// missing code with more tests here...
charPointerToQstring();
}
The big question is: Why isn't Qt handling a single char of a char* argument right but a string as argument (with the same info) goes well? If we have a char* as an argument then we can only go for each char from 0x00 to 0xFF (unsinged). Why not make 0x0000 to 0x00FF out of it?
Edit:
The answer of Artyer explains the behavior for buf1 but not for buf2. buf2 is a char[3] { 0xC4, 0x61, '\0' } which get's converted (with Artyers help) to a QString with elements QChar{ 0x00C4, 0x0061 }. So Qt can easily convert those 0xC4 characters to 0x00C4. In fact qstr1 shows that it can convert the two chars { 0xC3, 0x84 } from 'Ä' to one correct QChar {0x00C4}. If we have a char* as an argument then we can only go for each char from 0x00 to 0xFF (unsinged). Why not make 0x0000 to 0x00FF out of it?
And btw, I can't accept that approach yet because it now breaks mstr1 and mstr3. They then get exactly the "same" elements as buf1 but in QChar (so, without the closing '\0', from char[3] { 0xC3, 0x84, 0x61 } to QChar { 0x00C3, 0x0084, 0x0061 } but it should get QChar { 0x00C4, 0x0061 })
What is probably the case is that "Äa" is three UTF-8 encoded bytes in the source file (Equivalent to char[4]{ 0xC3, 0x84, 'a', '\0' }), and the QString constructor expects UTF-8 encoded data.
The 65533 character (U+FFFD) is the replacement character for the invalid UTF-8 data.
Use QString::fromLatin1:
myQString::myQString(const char* p) : QString(QString::fromLatin1(p, std::strlen(p)))
{
ENTER_FUNCTION;
}
myQString::myQString(const QByteArray& ba) : QString(QString::fromLatin1(ba))
{
ENTER_FUNCTION;
}
Also consider using QLatin1StringView instead of char* to avoid getting confused about encoding (might be called QLatin1String in older QT versions)

Qt Creator linking error: undefined reference to `KFUNC'

) I'm currently developing a project under Qt Creator 4.3.1 which needs an external library to use a key-lock hardware device.
The device comes with a CD with the libraries libkfunc64.so and libkfunc64.a (and the 32 bits versions) presumably compiled for C with gcc. I already tried the libraries in a dummy program to check if they work, and they do.
But when I add the important chunk of code into the Qt project it spits the following error
[...] error: undefined reference to `KFUNC(unsigned int, unsigned int, unsigned int, unsigned int)'
I've already tried to include the library
by coping the libraries to the system directories and simply including the following line in the .pro file
LIBS += -lkfunc64
by using relative paths to the libs in the .pro file
LIBS += -L$$PWD/../build-project/ -lkfunc64
INCLUDEPATH += $$PWD/../build-Qmetrics-Desktop-Debug
DEPENDPATH += $$PWD/../build-Qmetrics-Desktop-Debug
# PRE_TARGETDEPS += $$PWD/../build-Qmetrics-Desktop-Debug/libkfunc64.a
but it keeps showing this error when trying to (re)build it. I've read almost every subject about this issue on the internet but non of them fix this problem.
This is the code for main.cpp
#include "mainwindow.h"
#include <QApplication>
#include <QMessageBox>
// Key-lock stuff ----------
#define ValidateCode1 0X488B
#define ValidateCode2 0XFEE2
#define ValidateCode3 0XEF90
#define ClientIDCode1 0xB862
#define ClientIDCode2 0x54CB
#define ReadCode1 0X1772
#define ReadCode2 0XC4E6
#define ReadCode3 0XBCF8
extern unsigned long KFUNC(unsigned, unsigned, unsigned, unsigned);
unsigned RotateLeft(unsigned, int);
void KTASK(unsigned, unsigned, unsigned, unsigned);
unsigned short ReturnValue1, ReturnValue2;
// -------------------------
int main(int argc, char *argv[])
{
KTASK(1, ValidateCode1, ValidateCode2, ValidateCode3);
KTASK(RotateLeft(ReturnValue1, ReturnValue2 & 7) ^ ReadCode3 ^ ReturnValue2, RotateLeft(ReturnValue2, ReturnValue1 & 15), ReturnValue1 ^ ReturnValue2, 0);
if ((ReturnValue1 == ClientIDCode1) && (ReturnValue2 == ClientIDCode2))
{
QApplication app(argc, argv);
MainWindow w;
w.showFullScreen();
return app.exec();
}
else
{
QMessageBox msgBox;
msgBox.setText("Wrong or missing key-lock!");
return msgBox.exec();
}
}
unsigned RotateLeft(unsigned Target, int Counts)
{
int i;
static unsigned LocalTarget, HighBit;
LocalTarget = Target;
for (i=0; i<Counts; i++)
{
HighBit = LocalTarget & 0X8000;
LocalTarget = (LocalTarget << 1) + (HighBit >> 15);
}
LocalTarget = LocalTarget & 0XFFFF; /* For 32 bit integers */
return (LocalTarget);
}
void KTASK(unsigned CommandCode, unsigned Argument2, unsigned Argument3, unsigned Argument4)
{
unsigned long int ReturnValue;
ReturnValue = KFUNC(CommandCode, Argument2, Argument3, Argument4); // <--- this is the only function that is used from the external library
ReturnValue1 = (unsigned) (ReturnValue & 0XFFFF);
ReturnValue2 = (unsigned) (ReturnValue >> 16);
}
And the (relevant part of the) .pro file is as follow
# Adds OpenCV, DC1394 and Keylok libraries
unix {
CONFIG += link_pkgconfig
PKGCONFIG += opencv
LIBS += -ldc1394 -lkfunc64
}
I really don't know what to do :'(
Have you tried declaring the function with extern "C"?
e.G.:
#ifdef __cplusplus
extern "C"
{
#endif
extern unsigned long KFUNC(unsigned, unsigned, unsigned, unsigned);
#ifdef __cplusplus
};
#endif
This is nessecary to call a external c function from c++.

Qt Lang environment?

I Have got an issue concerning Qt Locale environment when I execute the following code
QApplication(argc,argv) ;
float f = 42.5f
std::cout << std::to_string(f) ; // prints 42,5
Even if my computer got its locale to french I'd like my program to be compiled with us standard printing format (i.e. 42.5 ). Is there a way to do that with a compiler option ?
This works fine for me:
main.cpp
#include <QString>
#include <QDebug>
#include <QCoreApplication>
#include <QLocale>
int main(int argc, char **argv)
{
QCoreApplication coreApplication(argc, argv);
float f = 42.5f;
qDebug() << QString::number(f, 'f', 1);
QLocale locale;
qDebug() << locale.toString(f, 'f', 1);
return coreApplication.exec();
}
main.pro
TEMPLATE = app
TARGET = main
QT = core
SOURCES += main.cpp
Build and Run
qmake && make && ./main
Output
"42.5"
"42.5"
This may be interesting for you:
QString QString::number(double n, char format = 'g', int precision = 6) [static]
Returns a string equivalent of the number n, formatted according to the specified format and precision. See Argument Formats for details.
Unlike QLocale::toString(), this function does not honor the user's locale settings.

UTF-8 error in GtkTextView while decoding base64

I have been trying to figure this out for a few days now. All I am trying to do is decode a base64 string and add it to a Gtk::TextView. Below is the code:
txtbuffer_ = Gtk::TextBuffer::create();
txtview_.set_buffer(txtbuffer_);
const Glib::ustring str = Glib::Base64::decode("YmJi3A==");
txtbuffer_->set_text(str);
When I run the program I get the error:
Gtk-CRITICAL **: gtk_text_buffer_emit_insert: assertion 'g_utf8_validate (text, len, NULL)' failed
This error only occurs with Unicode characters. When the text is ASCII it all works fine.
I have tried three different base64 decoders, I tried using std::string and Glib::ustring with all the different decoders. I also tried using the function Glib::locale_to_utf8(), but that gives me the error terminate called after throwing an instance of 'Glib::ConvertError'. And I tried using Glib::convert with the same error.
I know that Gtk::TextView can display Unicode because if I set the text to a string with Unicode it will display the text.
I read that Gtk::TextView displays text in UTF-8, so I think my problem is that the decoded string is not coded in UTF-8, but I am not sure. So my question is how can I get Gtk::TextView to display the decoded base64?
Added note: I am using version 3.8 of Gtkmm
Tested using version 3.12, same error message
Minimal program:
//test.h
#ifndef TEST_H_
#define TEST_H_
#include <gtkmm.h>
class MainWindow : public Gtk::Window
{
public:
MainWindow();
virtual ~MainWindow();
protected:
Gtk::Box box_main;
Gtk::TextView txtview_;
Glib::RefPtr<Gtk::TextBuffer> txtbuffer_;
};
#endif /* TEST_H_ */
//test.cpp
#include "test.h"
MainWindow::MainWindow()
{
Gtk::Window::add(box_main);
box_main.pack_start(txtview_);
txtbuffer_ = Gtk::TextBuffer::create();
txtview_.set_buffer(txtbuffer_);
const Glib::ustring str = Glib::Base64::decode("YmJi3A==");
txtbuffer_->set_text(str);
Gtk::Window::show_all_children();
}
MainWindow::~MainWindow()
{
}
//main.cpp
#include "test.h"
int main(int argc, char* argv[])
{
Glib::RefPtr<Gtk::Application> app = Gtk::Application::create(argc, argv, "test.program");
MainWindow mw;
return app->run(mw);
}
The reason why it was not working was because the string that I encoded was not UTF-8. Thanks to: https://mail.gnome.org/archives/gtk-list/2014-April/msg00016.html. I found out that the encoding was ISO-8859-1. So there are 2 fixes kind of, first, first encode the string to utf8:
const Glib::ustring str2 = Glib::Base64::encode("bbbÜ");
or you have to figure out the original encoding of the string, so for me this worked:
Glib::convert(base64_str, "UTF-8", "ISO-8859-1");
From documentation:
Note that the returned binary data is not necessarily zero-terminated,
so it should not be used as a character string.
That means utf8 validate will read beyond bounds with a likelyhood near 1 get a sequence of bytes which fail to be valid utf8 characters.
But even that did not fix it. It seems that the length is one too long and the last value is just garbage.
So you can either use (which I'd recommend)
std::string stdstr = Glib::Base64::decode (x);
const Glib::ustring str(stdstr.c_str(), stdstr.length()-1);
or
gsize len = 0;
const gchar *ret = (gchar*)g_base64_decode (x, &len);
len --;
const Glib::ustring str(ret, len);
g_free (ret);
So I guess this a bug in gtk+ (which gtkmm encapsulates)

Wrong output of qDebug() (UTF - 8)

I'm trying to store a string with special chars::
qDebug() << "ÑABCgÓ";
Outputs: (here i can't even type the correct output some garbage is missing after à & Ã)
ÃABCgÃ
I suspect some UTF-8 / Latin1 / ASCII, but can't find the setting to output to console / file. What i have written in my code : "ÑABCgÓ".
(Qt:4.8.5 / Ubunto 12.04 / C++98)
You could use the QString QString::fromUtf8(const char * str, int size = -1) [static] as the sample code presents that below. This is one of the main reasons why QString exists.
See the documentation for details:
http://qt-project.org/doc/qt-5.1/qtcore/qstring.html#fromUtf8
main.cpp
#include <QString>
#include <QDebug>
int main()
{
qDebug() << QString::fromUtf8("ÑABCgÓ");
return 0;
}
Building (customize for your scenario)
g++ -fPIC -I/usr/include/qt -I/usr/include/qt/QtCore -lQt5Core main1000.cpp && ./a.out
Output
"ÑABCgÓ"
That being said, depending on your locale, simply qDebug() << "ÑABCgÓ"; could work as well like in here, but it is recommended to make sure by explicitly asking the UTF-8 handling.
Try this:
QTextCodec *codec = QTextCodec::codecForName("UTF-8");
QTextCodec::setCodecForCStrings(codec);
qDebug() << "ÑABCgÓ";