Qt and GSM encoding - c++

Hi i am developping an open source sms application and i am using Qt. But now i have various problem with the encodings.
I am using this library for various pdu encoding and decoding https://sourceforge.net/projects/pdulib/
In gsm there 3 main encodings Default 7bit alphabet, UCS2 and 8bit data. So when For 7bit encoding the library expect std::string argument so i am passing it like this
QString message = lineEdit->toPlainText();
std::string msg_to_pass = message.toStdString();
The sms is correctly sended but when there is accent characters the sms arrives with bad characters like 'C?' the sign of bad charset/encoding. Note that it is the phone that show it. So decoding error is very low.
But when i am using the UCS2 encoding
QString message = lineEdit->toPlainText();
std::wstring msg_to_pass = message.toStdWString();
because the library requires std::wstring for UCS2 the characters with accent got printed well. But the problem is when i am decoding a sms received. I have modified function of this library to suit my needs but again characters with accent have a problem. Here is the function
template<class MsgT>
void PrintUserData(const MsgT& msg, QString &message)
{
if (MsgT::DefaultAlphabet == msg.alphabet())
{
message = QString::fromStdString(msg.template userData<typename MsgT::DefaultUserData>().userData());
std::cout << "User Data: " << msg.template userData<typename MsgT::DefaultUserData>().userData() << std::endl;
}
else if(MsgT::UCS2Alphabet == msg.alphabet())
{
QTextStream ts(stdout);
message = QString::fromStdWString( msg.template userData<typename MsgT::UCS2UserData>().userData());
qInfo()<< message;
std::wcout << L"User Data: " << msg.template userData<typename MsgT::UCS2UserData>().userData() << std::endl;
}
else if(MsgT::EightBitDataAlphabet == msg.alphabet())
{
qInfo()<< "3";
const std::vector<unsigned char>& data = msg.template userData<typename MsgT::EightBitUserData>().userData();
std::cout << "User Data: ";
PrintBinary(data.begin(), data.end());
std::cout << std::endl;
}
}
I have tested various pdu lib for C++ and this seems to be the best for me.
My think is that i am doing something bad with the std::string and std::wstring but i can't figure what.
Also i know that UCS2 is UTF-16 but i have read that there is small differences.
My locale is FR (French)
I am using Qt 5.9
Sorry for my english.

Related

How to send string in utf-8 to irc server?

I have an irc bot written in c++ with the use of Qt library. I store console text input in std::string , and then i'm using QSocket to post it on irc chat. But the problem is im want to use special signs (polish letters), which dont appear properly on chat. What is the problem?
The way i use QSocketis:
void Socket::poster(const QByteArray send)
{
mSocket->write(send);
mSocket->flush();
mSocket->reset();
}
QByteArray i create from std::string and std::cin
he code's long so i only post the parts crucial for the specific functonality which fails
Socket class (which is the main class in the program, providing data to other classes):
#############################################################
protected:
QSslSocket *mSocket;
--------------------
connect(mSocket, SIGNAL(readyRead()),
this, SLOT(readyReady())
--------------------
//console input:
QThread *thread = new QThread;
consoleInput = new ConsoleInput();
consoleInput->startConsole(thread, mSocket);
consoleInput->moveToThread(thread);
thread->start();
-------------------
void Socket::readyReady()
{
QString data;
data2 = data;
mSocket->ReadOnly;
while(mSocket->canReadLine())
{
data = mSocket->readLine();
}
mSocket->reset();
}
---------------------
void Socket::poster(const QByteArray send) //sending to irc from many classes news, console itd
{
mSocket->write(send);
mSocket->flush();
mSocket->reset();
}
-------------------
ConsoleInput class (which takes console input, which is later sent to irc chat):
###############################
void ConsoleInput::run()
{
std::cout << "!ConsoleInput::run()" << "\n";
while(1){
std::string input;
std::getline(std::cin, input);
determineOption(input);
if(input[0] != '/' || input[0] != '\\')
postInput(input);
input.clear();
}
}
----------------------------------
void ConsoleInput::postInput(std::string &input)
{
if(input[0]=='/')
return; //this prevents bot poting "/command" to channel
std::string lineToPost;
std::cout << "!lineToPost - input " << input << "\n";
ColourManipulation c;
lineToPost = "PRIVMSG #grunge " + c.addColours(input) + "\r\n";
emit mySignal(QByteArray::fromStdString(lineToPost)); // problem
}
Make sure std::cin/cout can accept & show non-ascii characters
Check the code can accept & show non-ascii characters:
std::string input;
std::getline(std::cin, input);
std::cout << input;
If you don't have problems with non-ascii characters in console itself
You need:
Know in which encoding the data originally comes from console to std::string &input.
std::string type per se uses no encoding -- it will return the bytes
you put in it -
What encoding does std::string.c_str() use?.
Import the bytes into QString using necessary encoding convertion
Export the resulting QString to UTF-8 encoded QByteArray (QByteArray itself is just an array of bytes too).
Write the QByteArray to a socket.
You can write something like the following:
/*
From doc: QTextCodec::codecForLocale()
Returns a pointer to the codec most suitable for this locale.
The codec will be retrieved from ICU where that backend is in use,
otherwise it may be obtained from an OS-specific API.
In the latter case, the codec's name may be "System".
*/
QTextCodec *codec = QTextCodec::codecForLocale(); // In most cases, it is not UTF-8
// Or set the encoding explicitly:
//QTextCodec *codec = QTextCodec::codecForName("Shift-JIS"); // put your input encoding here
QTextDecoder *decoder = codec->makeDecoder();
QByteArray chunk = QByteArray::fromStdString(input);
QString string = decoder->toUnicode(chunk);
delete decoder;
emit mySignal(string.toUtf8());
Be note that you can avoid std::string and use QString only:
QString is more comfortable to use, and, once received the data correctly, it always stores data in the same known format internally, despite of std::string, which has no idea what data it stores.
How to read from console to QString directly:
QTextStream in(stdin);
in.setCodec(<your console codec>);
QString input = in.readLine();
See QTextCodec and QTextStream.
Read also The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)

How to get datas send by Protobuf with a QTcpSocket [duplicate]

This question already has answers here:
Correct way to losslessly convert to and from std::string and QByteArray
(3 answers)
Closed 3 years ago.
I'm new on protobuf and QTcpServer/Socket and I want to read my .proto data send by my client, but when i'm reading the data, the QString done is empty
For now, I just want to send a message that say "hello" when my client is connected. The QByteArray returning by QTcpSocket::readAll is NOT empty, but the QString created with the bytes is empty.
Here is my .proto ultra basic one :
syntax = "proto3";
package protobuf;
message Message
{
string content = 2;
}
write functions :
// When i'm connecting to the server i create a PlayerManager and i call this function with message = "hello"
void Server::sendMessageToPlayer(const PlayerManager& playerManager, const QString& message)
{
auto messageProto = new protobuf::Message;
messageProto->set_content(message.toStdString());
playerManager.socketManager()->sendData(*messageProto);
}
// I serialize the protobuf
template <typename protobufType>
void sendData(const protobufType& protobuf)
{
std::string dataToSend;
if (!protobuf.SerializeToString(&dataToSend))
{
// This never pass -> the protobuf is well serialize
qDebug() << "The protobuf send cannot be serialized, please, make sure that you used protobufs correctly";
}
// Before i write it
write(dataToSend);
}
void SocketManager::write(const std::string& data)
{
// I tried this but it's not working either
// QTextCodec* codec = QTextCodec::codecForName("CP1251");
// QString codecData = codec->toUnicode(data.c_str());
QByteArray block;
QDataStream out(&block, QIODevice::WriteOnly);
out.setVersion(QDataStream::Qt_5_10);
out << data.c_str();
_tcpSocket->write(block);
qDebug() << block;
}
read functions :
void SocketManager::read()
{
QByteArray bytes = _tcpSocket->readAll();
qDebug() << bytes;
// Doesn't work either
// QTextCodec* codec = QTextCodec::codecForName("CP1251");
// QString data = codec->toUnicode(bytes);
QString data(bytes);
qDebug() << data;
emitMessageType(data.toStdString());
}
void SocketManager::emitMessageType(const std::string& data)
{
// Protobufs can parse an empty string (and so, emit signal), to avoid that, the function will tell you if data
// are empty, then return
if (data.empty())
{
qDebug() << "Datas are empty";
return;
}
protobuf::Message message;
if (message.ParseFromString(data))
{
emit messageProtoReceived(message);
return;
}
qDebug() << "The data send cannot be translate, please, make sure that you used protobufs correctly";
}
So, I would like that my client receive "hello" when he is connected but my debug are :
Server side :
"\x00\x00\x00\b\x12\x05hello\x00"
Client side
"\x00\x00\x00\b\x12\x05hello\x00"
""
Datas are empty
When I use QTextCodec (commented lines of the code) the debug are :
Server side :
"\x00\x00\x00\x0E\x00\x12\x00\x05\x00h\x00""e\x00l\x00l\x00o"
Client side
"\x00\x00\x00\x0E\x00\x12\x00\x05\x00h\x00""e\x00l\x00l\x00o"
"\u0000\u0000\u0000\u000E\u0000\u0012\u0000\u0005\u0000h\u0000e\u0000l\u0000l\u0000o"
The data send cannot be translate, please, make sure that you used protobufs correctly
So the QByteArea is parse, but protobuf don't succeed to parse the given string.
Thanks for reading, I hope youc ould help me
Not sure if it's the case but the documentation says that the default constructor of a QString that takes a QByteArray:
The given byte array is converted to Unicode using fromUtf8(). Stops copying at the first 0 character, otherwise copies the entire byte array.
So probably, you are having some troubles whit the conversion.
As alternative, you can try ParseFromArray method, instead of converting a QByteArray into a std::string.
const auto byteArray = _tcpSocket->readAll();
protobuf::Message message;
if (!message.ParseFromArray(byteArray.data(), byteArray.size())) {
qDebug() << "Failed to parse person.pb.";
}
out << data.c_str() inside SocketManager::write might discard at first \0. Try converting to QByteArray as described at Correct way to losslessly convert to and from std::string and QByteArray to ensure the whole string is sent.

Working with files, cp1251 and utf-8 encodings

Rigth now i have something like
QCoreApplication a(argc, argv);
QStringList argList = a.arguments();
if (argList.length() < 2 || argList.length() > 3)
{
QTextStream(stdout) << "Usage: toGift input_file [output_dir]\n";
return - 1;
}
For decoding Windows-1251 strings, but i still get wrong filename, which is passed through argv. The filename is in cyrillic. Can't firgure out what am i missing
Text encodings in Windows console applications is a hairy matter. Tip: use the arguments() member of your QApplication (QCoreApplication if it's a console executable), it retrieves the arguments directly in UTF-16 from Windows, without trying to decode it from the lossy local encoding used for argc/argv.
in the end it came to this:
QTextCodec *codec = QTextCodec::codecForName("Windows-1251");
if(codec == 0)
{
QTextStream(stdout) << "No codec was found" << endl;
}
QTextDecoder *decoder = codec->makeDecoder();
QString inp_fileName = QString(decoder->toUnicode(argv[1]));
................
QFile inp_file(inp_fileName);
if(!inp_file.open(inp_file.ReadOnly))
{
QTextStream(stdout) << "Could not open file for reading.\n" << inp_file.errorString() << endl;
return -3;
}
I still needed the decoder for further use.

Wrong/ too few output from Qt serial write function

I am trying to send 7 variables that I read before from the GUI with the serial->write... function to my microcontroller.
I wrote a littile program on my microcontroller that blinks a led if there was an input. But it only shows 5 inputs.
I thougt it might be that my microcontroller is not fast enough and I stalled the Qt programm but that didn´t work.
So I don´t understand why there are too few inputs.
Further I used the qDebug() << function to print out in the console what I send to my microcontroller but this shows only the number of characters I send ( f.e. if I send 500 qDebug() shows "3").
So i guess something with my conversion isn´t correct too.
here is my code:
//conversion from QString into const char*
q_bauteillaenge = (ui->Bauteillaenge_e->text());
q_messintervall_vert = (ui->Messintervall_vert_e->text());
q_anz_inkrem_vert = (ui->Anzahl_Inkremente_vert_e->text());
q_inkrem_laenge_vert = (ui->Inkrementlaenge_vert_e->text());
q_anz_messungen_vert = (ui->Anzahl_Messungen_vert_e->text());
q_abs_messungen_vert = (ui->Abstand_Messungen_vert_e->text());
a_bauteillaenge = q_bauteillaenge.toUtf8();
a_messintervall_vert = q_messintervall_vert.toUtf8() ;
a_anz_inkrem_vert = q_anz_inkrem_vert.toUtf8() ;
a_inkrem_laenge_vert = q_inkrem_laenge_vert.toUtf8() ;
a_anz_messungen_vert = q_anz_messungen_vert.toUtf8();
a_abs_messungen_vert = q_abs_messungen_vert.toUtf8();
bauteillaenge = a_bauteillaenge.constData();
messintervall_vert = a_messintervall_vert.constData() ;
anz_inkrem_vert = a_anz_inkrem_vert.constData() ;
inkrem_laenge_vert = a_inkrem_laenge_vert.constData() ;
anz_messungen_vert = a_anz_messungen_vert.constData();
abs_messungen_vert = a_abs_messungen_vert.constData();
//Sending
qDebug() << serial->write("1");
serial->write("\n");
qDebug() << serial->write(bauteillaenge);
serial->write("\n");
qDebug() << serial->write(messintervall_vert);
serial->write("\n");
qDebug() << serial->write(anz_inkrem_vert);
serial->write("\n");
qDebug() << serial->write(inkrem_laenge_vert);
serial->write("\n");
qDebug() << serial->write(anz_messungen_vert);
serial->write("\n");
qDebug() << serial->write(abs_messungen_vert);
serial->write("\n");
Where are my faults ?
Regards
I would look into the UTF-8 encoding. Are you sure your microcontroller code excepts UTF-8 bytes?
How the led is responding to the incoming data depends on the code of your microcontroller. Maybe you can add some microcontroller code.
The serial->write function returns the count of the actual written bytes. So it works as it should, but you expected another return value.
Read the docs about this one:
http://doc.qt.io/qt-5/qiodevice.html#write

win32 C++ print string to printer

After a few days of searching the net on how exactly I can go about printing an arbitrary string to an arbitrary printer on windows, I finally came up with this code.
LPBYTE pPrinterEnum;
DWORD pcbNeeded, pcbReturned;
PRINTER_INFO_2 *piTwo = NULL;
HDC printer;
EnumPrinters(PRINTER_ENUM_LOCAL,NULL,2,NULL,0,&pcbNeeded,&pcbReturned);
pPrinterEnum = new BYTE[pcbNeeded];
if (!EnumPrinters(PRINTER_ENUM_LOCAL,NULL,2,pPrinterEnum,pcbNeeded,&pcbNeeded,&pcbReturned)) {
qDebug() << "In Print, could not enumerate printers";
} else {
piTwo = ((PRINTER_INFO_2*)pPrinterEnum);
for (int i = 0; i < pcbReturned; i++) {
QString name = QString::fromWCharArray(piTwo[i].pPrinterName);
if (this->m_printer_path == name) {
const WCHAR * driver = L"WINSPOOL\0";
printer = CreateDC(NULL,piTwo[i].pPrinterName,NULL,NULL);
}
}
}
if (printer == 0) {
qDebug() << "No Printer HDC";
return;
} else {
qDebug() << "Printer seems okay!";
}
qDebug() << "Starting Document";
DOCINFO di;
memset( &di, 0, sizeof( di ) );
di.cbSize = sizeof( di );
WCHAR * text = new WCHAR[ba.length()];
QString(ba).toWCharArray(text);
StartDoc(printer,&di);
qDebug() << "Writing text";
TextOut(printer,0, 0, text, ba.length());
qDebug() << "Text Written";
EndPage(printer);
qDebug() << "Page ended";
DeleteDC(printer);
qDebug() << "DC Deleted";
Some basic caveats:
1) I cannot use QPrinter. I need to write raw text, no postscript.
2) I do not know the name of the printer until the user sets it, and I do not know the size of the string to print until the user creates it.
Additional information:
a) The printer works, I can print from Notepad, Chrome, just about everything to the printer that I want.
b) I am willing to implement just about any hack. Ones like write it to a text file and issue the copy command don't seem to work, that is, I get a failed to initialize device error.
This works:
notepad /P Documents/test_print.txt
This does not work:
copy Documents\test_print.txt /D:EPSON_TM_T20
copy Documents\test_print.txt /D \MYCOMPUTER\epson_tm_t20 (leads to access denied, printer is shared)
print Documents\test_print.txt (Unable to initialize device)
I have tried just about every recommended way to print a text file from the command line, just doesn't work. I have installed, reinstalled driver, added printer, mucked with ports and done it all again.
Obviously there is something simple about windows printing that I am missing due to inexperience.
What I want to accomplish is:
1) Best Scenario( Directly write text to the printer)
2) Second best scenario (Write text to a file, then execute some program to print it for me) Notepad adds an annoying amount of space to the bottom of the printout wasting paper.
Since the program is for end users, I have to find a way to do this automagically for them, so I can't expect them to click checkbox a in tab 36 after running command obscure_configuration from a powershell.
Any help would be greatly appreciated.
/Jason
UPDATE
This is the working code, before I go through an spruce it up a bit, which prints the contents of a QByteArray to a thermal printer.
qDebug() << "Executing windows code";
BOOL bStatus = FALSE;
DOC_INFO_1 DocInfo;
DWORD dwJob = 0L;
DWORD dwBytesWritten = 0L;
HANDLE hPrinter;
wchar_t * name = new wchar_t[this->m_printer_path.length()+1];
this->m_printer_path.toWCharArray(name);
name[this->m_printer_path.length() + 1] = 0;
qDebug() << "opening printer";
bStatus = OpenPrinter(name,&hPrinter, NULL);
if (bStatus) {
qDebug() << "Printer opened";
DocInfo.pDocName = L"My Document";
DocInfo.pOutputFile = NULL;
DocInfo.pDatatype = L"RAW";
dwJob = StartDocPrinter( hPrinter, 1, (LPBYTE)&DocInfo );
if (dwJob > 0) {
qDebug() << "Job is set.";
bStatus = StartPagePrinter(hPrinter);
if (bStatus) {
qDebug() << "Writing text to printer";
bStatus = WritePrinter(hPrinter,ba.data(),ba.length(),&dwBytesWritten);
EndPagePrinter(hPrinter);
} else {
qDebug() << "could not start printer";
}
EndDocPrinter(hPrinter);
qDebug() << "closing doc";
} else {
qDebug() << "Couldn't create job";
}
ClosePrinter(hPrinter);
qDebug() << "closing printer";
} else {
qDebug() << "Could not open printer";
}
if (dwBytesWritten != ba.length()) {
qDebug() << "Wrong number of bytes";
} else {
qDebug() << "bytes written is correct " << QString::number(ba.length()) ;
}
Note: I do owe an apology to Skizz, what he wrote was actually helpful in debugging the fundamental issue. The characters in the QByteArray are preformatted specifically for the printer, the problem is, they contain several NULL bytes. When trying to send them to the printer, this causes TextOut to truncate the text, only printing the first few lines. Using WritePrinter, as suggested in the answer ignores null bytes and accepts a void * and a length, and just puts it all there.
Further, his response recommending the use of PrintDlg did work to fectch the correct printer HDC, the issus is that, the user first chooses a printer once, and then doesn't need to choose it each time they print, because they will be printing alot (It's a Point of Sale).
The problem with getting the printer HDC from the string name was due to not adding the all important NULL byte to wchar_* which was solved this way:
wchar_t * name = new wchar_t[this->m_printer_path.length()+1];
this->m_printer_path.toWCharArray(name);
name[this->m_printer_path.length() + 1] = 0;
In the above, m_printer_path is a string representation of the name of the printer taken from Print Manager.
Because the string has all the formatting necessary for the printer, there's no need to worry about new lines, or any formatting.
All three answers to this question were actually very helpful in implementing the final working solution, and I have voted up each answer, and I appreciate the time each person took in responding.
Most modern printers don't perform any form of layout processing of the data they are given. Thus, sending a sequence of characters to the printer would, at best, just print a line of text running off the side of the page in some default font. Carriage returns may work too.
What modern printers usually do is print pages using preprocessed data that the printer understands and defines what to print where and how to print it. All this preprocessing is done on the host PC and the results sent to the printer. This is why you usually install printer drivers - these drivers take the user data (whether it's a simple text file or a DTP page) and converts it into a language the printer understands.
The upshot of this is that sending raw text to the printer probably won't work.
Then you've got the problem of having multiple printers with different properties and languages.
So, in Windows, all this is abstracted into the printer device context object. This has the same interface as a graphics device context but you create it differently.
The Win32 API has a common dialog to let the user choose the printer. Use the PrintDlgEx function to allow the user to choose a printer. Then use the returned DC to draw text to the page.
There are a couple of MSDN articles describing how to send raw data (printer control codes, etc.) to a printer.
How To: Send Data Directly to a GDI Printer
How To: Send Data Directly to an XPS Printer
You have the right idea (though you should have StartPage and EndDoc calls to match up). The problem is that TextOut draws only a line of text. It won't break long strings into multiple lines, etc. You need to do that (or find code to do it).
If you know that the text will always fit on a single page, you could probably replace your TextOut with a DrawTextEx call, which can do basic line breaking, tab expansion, etc.
Why not try QPrint.. it prints raw text using a Generic Text Only driver
QString prn("^XA^FO121,41^A0N,19,15^FDABC DEFGHIJKLMNOPQRSTUVWXYZ^FS^XZ");
QPrinter printer(QPrinterInfo::defaultPrinter()); //the default printer is "Generic / Text Only"
QTextDocument doc(prn);
doc.print(&printer);
MTry the following code in C++:
#include<fstream>
Class PrinterDriver{
Private:
fstream print("PRN")
Public:
Void Print(char a[]){
print >>a;}
Char GetPrinterStatus[](){
char c[];
print<<c;
return c;}};
understand it(key)