Qt - Sending a large amount of data via QDataStream TCP - c++

I want to benchmark a TCP Connection to an ECU (Transferrate)
For that, i want so send some Data. The amout of data can be set in my GUI from x KB to x GB.
The transferSize is calculated from the values set in the GUI. Buffer Size is 1024:
#define BENCHMARK_TRANSFER_BUFFER_SIZE 1024
void logic::TransferAndLatencyEthernetBenchmark::transferDataTcp(TCPClient& tcpclient)
{
uint32_t toSend = transferSize;
uint32_t currentSendSize = 0;
while (toSend > 0)
{
if (toSend / BENCHMARK_TRANSFER_BUFFER_SIZE >= 1)
{
currentSendSize = BENCHMARK_TRANSFER_BUFFER_SIZE;
}
else if ((toSend / BENCHMARK_TRANSFER_BUFFER_SIZE) == 0)
{
currentSendSize = toSend % BENCHMARK_TRANSFER_BUFFER_SIZE;
}
toSend -= currentSendSize;
tcpclient.sendData(transferSendBuffer, currentSendSize);
}
}
So iam calling the tcpclient.sendData(transferSendBuffer, currentSendSize) function als long as toSend > 0:
...
QDataStream out;
....
void TCPClient::sendData(uint8_t* buffer, uint32_t size)
{
qDebug() << "current send size: " << size << " - Transfer send buffer: " << (const char*)buffer;
for (int i=0; i<size; i++)
{
buffer[i] = 'a';
}
out.setDevice(socket_aurix);
out.writeBytes((const char*)buffer, size);
}
Iam pretty new on Qt (C++ in generel) and iam sure iam doing it wrong ;)
It actually works but crashed when i try so send bigger amouts of data.
What would be a better way to initialize the fill the array instead of the for loop?
What what be a better approach in generel to use QDataStream?
Many thanks.

Related

Qt c++ Issue receiving data weighing scale Ohaus aviator 7000

I am trying to establisch a serial port connection to my Aviator 7000 weighing scale using Qt c++. The expected result would be a succesfull communication through the use of a byte command.
Sadly I don't receive any bytes back from the scale. below you can find what I tried so far:
const int Max_attempts = 5;
const int Max_sleep = 125;
int attemps;
attemps = 0;
while (true)
{
int enq {5};
QByteArray bytes;
bytes.setNum(enq);
m_serial->write(bytes);
m_serial->waitForReadyRead(Max_sleep);
if (m_serial->bytesAvailable() !=0)
{
qDebug() << m_serial->bytesAvailable() ;
qDebug() << "connected" << m_serial->readAll();
break;
}
attemps += 1;
if (attemps == Max_attempts)
{
qDebug() << "no connection established";
break;
}
}
Kind regards,
Tibo
According to this manual you are supposed to send a byte 0x05 but you are sending 0x35 (the character "5").
Use
bytes.append('\X05');

RTMP Broadcast packet body structure for Twitch

I'm currently working on a project similar to OBS, where I'm capturing screen data, encoding it with the x264 library, and then broadcasting it to a twitch server.
Currently, the servers are accepting the data, but no video is being played - it buffers for a moment, then returns an error code "2000: network error"
Like OBS Classic, I'm dividing each NAL provided by x264 by its type, and then making changes to each
int frame_size = x264_encoder_encode(encoder, &nals, &num_nals, &pic_in, &pic_out);
//sort the NAL's into their types and make necessary adjustments
int timeOffset = int(pic_out.i_pts - pic_out.i_dts);
timeOffset = htonl(timeOffset);//host to network translation, ensure the bytes are in the right format
BYTE *timeOffsetAddr = ((BYTE*)&timeOffset) + 1;
videoSection sect;
bool foundFrame = false;
uint8_t * spsPayload = NULL;
int spsSize = 0;
for (int i = 0; i<num_nals; i++) {
//std::cout << "VideoEncoder: EncodedImages Size: " << encodedImages->size() << std::endl;
x264_nal_t &nal = nals[i];
//std::cout << "NAL is:" << nal.i_type << std::endl;
//need to account for pps/sps, seems to always be the first frame sent
if (nal.i_type == NAL_SPS) {
spsSize = nal.i_payload;
spsPayload = (uint8_t*)malloc(spsSize);
memcpy(spsPayload, nal.p_payload, spsSize);
} else if (nal.i_type == NAL_PPS){
//pps always happens after sps
if (spsPayload == NULL) {
std::cout << "VideoEncoder: critical error, sps not set" << std::endl;
}
uint8_t * payload = (uint8_t*)malloc(nal.i_payload + spsSize);
memcpy(payload, spsPayload, spsSize);
memcpy(payload, nal.p_payload + spsSize, nal.i_payload);
sect = { nal.i_payload + spsSize, payload, nal.i_type };
encodedImages->push(sect);
} else if (nal.i_type == NAL_SEI || nal.i_type == NAL_FILLER) {
//these need some bytes at the start removed
BYTE *skip = nal.p_payload;
while (*(skip++) != 0x1);
int skipBytes = (int)(skip - nal.p_payload);
int newPayloadSize = (nal.i_payload - skipBytes);
uint8_t * payload = (uint8_t*)malloc(newPayloadSize);
memcpy(payload, nal.p_payload + skipBytes, newPayloadSize);
sect = { newPayloadSize, payload, nal.i_type };
encodedImages->push(sect);
} else if (nal.i_type == NAL_SLICE_IDR || nal.i_type == NAL_SLICE) {
//these packets need an additional section at the start
BYTE *skip = nal.p_payload;
while (*(skip++) != 0x1);
int skipBytes = (int)(skip - nal.p_payload);
std::vector<BYTE> bodyData;
if (!foundFrame) {
if (nal.i_type == NAL_SLICE_IDR) { bodyData.push_back(0x17); } else { bodyData.push_back(0x27); } //add a 17 or a 27 as appropriate
bodyData.push_back(1);
bodyData.push_back(*timeOffsetAddr);
foundFrame = true;
}
//put into the payload the bodyData followed by the nal payload
uint8_t * bodyDataPayload = (uint8_t*)malloc(bodyData.size());
memcpy(bodyDataPayload, bodyData.data(), bodyData.size() * sizeof(BYTE));
int newPayloadSize = (nal.i_payload - skipBytes);
uint8_t * payload = (uint8_t*)malloc(newPayloadSize + sizeof(bodyDataPayload));
memcpy(payload, bodyDataPayload, sizeof(bodyDataPayload));
memcpy(payload + sizeof(bodyDataPayload), nal.p_payload + skipBytes, newPayloadSize);
int totalSize = newPayloadSize + sizeof(bodyDataPayload);
sect = { totalSize, payload, nal.i_type };
encodedImages->push(sect);
} else {
std::cout << "VideoEncoder: Nal type did not match expected" << std::endl;
continue;
}
}
The NAL payload data is then put into a struct, VideoSection, in a queue buffer
//used to transfer encoded data
struct videoSection {
int frameSize;
uint8_t* payload;
int type;
};
After which it is picked up by the broadcaster, a few more changes are made, and then I call rtmp_send()
videoSection sect = encodedImages->front();
encodedImages->pop();
//std::cout << "Broadcaster: Frame Size: " << sect.frameSize << std::endl;
//two methods of sending RTMP data, _sendpacket and _write. Using sendpacket for greater control
RTMPPacket * packet;
unsigned char* buf = (unsigned char*)sect.payload;
int type = buf[0]&0x1f; //I believe &0x1f sets a 32bit limit
int len = sect.frameSize;
long timeOffset = GetTickCount() - rtmp_start_time;
//assign space packet will need
packet = (RTMPPacket *)malloc(sizeof(RTMPPacket)+RTMP_MAX_HEADER_SIZE + len + 9);
memset(packet, 0, sizeof(RTMPPacket) + RTMP_MAX_HEADER_SIZE);
packet->m_body = (char *)packet + sizeof(RTMPPacket) + RTMP_MAX_HEADER_SIZE;
packet->m_nBodySize = len + 9;
//std::cout << "Broadcaster: Packet Size: " << sizeof(RTMPPacket) + RTMP_MAX_HEADER_SIZE + len + 9 << std::endl;
//std::cout << "Broadcaster: Packet Body Size: " << len + 9 << std::endl;
//set body to point to the packetbody
unsigned char *body = (unsigned char *)packet->m_body;
memset(body, 0, len + 9);
//NAL_SLICE_IDR represents keyframe
//first element determines packet type
body[0] = 0x27;//inter-frame h.264
if (sect.type == NAL_SLICE_IDR) {
body[0] = 0x17; //h.264 codec id
}
//-------------------------------------------------------------------------------
//this section taken from https://stackoverflow.com/questions/25031759/using-x264-and-librtmp-to-send-live-camera-frame-but-the-flash-cant-show
//in an effort to understand packet format. it does not resolve my previous issues formatting the data for twitch to play it
//sets body to be NAL unit
body[1] = 0x01;
body[2] = 0x00;
body[3] = 0x00;
body[4] = 0x00;
//>> is a shift right
//shift len to the right, and AND it
/*body[5] = (len >> 24) & 0xff;
body[6] = (len >> 16) & 0xff;
body[7] = (len >> 8) & 0xff;
body[8] = (len) & 0xff;*/
//end code sourced from https://stackoverflow.com/questions/25031759/using-x264-and-librtmp-to-send-live-camera-frame-but-the-flash-cant-show
//-------------------------------------------------------------------------------
//copy from buffer into rest of body
memcpy(&body[9], buf, len);
//DEBUG
//save individual packet body to a file with name rtmp[packetnum]
//determine why some packets do not have 0x27 or 0x17 at the start
//still happening, makes no sense given the above code
/*std::string fileLocation = "rtmp" + std::to_string(packCount++);
std::cout << fileLocation << std::endl;
const char * charConversion = fileLocation.c_str();
FILE* saveFile = NULL;
saveFile = fopen(charConversion, "w+b");//open as write and binary
if (!fwrite(body, len + 9, 1, saveFile)) {
std::cout << "VideoEncoder: Error while trying to write to file" << std::endl;
}
fclose(saveFile);*/
//END DEBUG
//other packet details
packet->m_hasAbsTimestamp = 0;
packet->m_packetType = RTMP_PACKET_TYPE_VIDEO;
if (rtmp != NULL) {
packet->m_nInfoField2 = rtmp->m_stream_id;
}
packet->m_nChannel = 0x04;
packet->m_headerType = RTMP_PACKET_SIZE_LARGE;
packet->m_nTimeStamp = timeOffset;
//send the packet
if (rtmp != NULL) {
RTMP_SendPacket(rtmp, packet, TRUE);
}
I can see that Twitch is receiving the data in the inspector, at a steady 3kbps. so I'm sure something is wrong with how I'm adjusting the data before sending it. Can anyone advise me on what I'm doing wrong here?
The problems start before the code you included even. When you configure x264 be sure to set:
b_aud = 0;
b_repeat_headers = 0;
b_annexb = 0;
This will tell x264 to generate the format needed by rtmp, Then you can skip all the per-nal preprocessing.
For sps/pps use x264_encoder_headers to retrieve them after x264_encoder_open. Encode them into an "extradata" buffer as documented here Possible Locations for Sequence/Picture Parameter Set(s) for H.264 Stream. This extradata goes into an rtmp "sequence header" packet before any frames are sent. Set the frame the AVCPacketType accordingly body[1] in your case, 0 for sequence header 1 for everything else,
body[0] = 0x27;
body[1] = 0;
body[2] = 0;
body[3] = 0;
body[4] = 0;
memcpy(&body[5], extradata, extradata_size);
body[2] through body[4] MUST be set to the frame cts (pts - dts) if you have b frames. If you want to set it to zero, configure x264 for baseline profile, but this will result in reduced image quality. Use the return code from x264_encoder_encode as the frame size, and write the whole frame in one go.
int frame_size = x264_encoder_encode(encoder, &nals, &num_nals, &pic_in, &pic_out);
if(frame_size) {
int cts = pic_out->i_pts - pic_out->i_dts;
body[0] = pic_out->b_keyframe ? 0x27 : 0x17;
body[1] = 1;
body[2] = cts>>16;
body[3] = cts>>8;
body[4] = cts;
memcpy(&body[5], nals->p_payload, frame_size);
}
Finally, Twitch requires you also send an AAC audio stream. and be sure to set the keyframe interval to 2 seconds.

How can I send all data over a socket?

I am trying to send large amounts of data over a socket, sometimes when I call send (on Windows) it won't send all the data I requested, as expected. So, I wrote a little function that should have solved my problems- but it's causing problems where the data isn't being sent correctly and causing the images to be corrupted. I'm making a simple chat room where you can send images (screenshots) to each other.
Why is my function not working?
How can I make it work?
void _internal_SendFile_alignment_512(SOCKET sock, BYTE *data, DWORD datasize)
{
Sock::Packet packet;
packet.DataSize = datasize;
packet.PacketType = PACKET_FILETRANSFER_INITIATE;
DWORD until = datasize / 512;
send(sock, (const char*)&packet, sizeof(packet), 0);
unsigned int pos = 0;
while( pos != datasize )
{
pos += send(sock, (char *)(data + pos), datasize - pos, 0);
}
}
My receive side is:
public override void OnReceiveData(TcpLib.ConnectionState state)
{
if (state.fileTransfer == true && state.waitingFor > 0)
{
byte[] buffer = new byte[state.AvailableData];
int readBytes = state.Read(buffer, 0, state.AvailableData);
state.waitingFor -= readBytes;
state.bw.Write(buffer);
state.bw.Flush();
if (state.waitingFor == 0)
{
state.bw.Close();
state.hFile.Close();
state.fileTransfer = false;
IPEndPoint ip = state.RemoteEndPoint as IPEndPoint;
Program.MainForm.log("Ended file transfer with " + ip);
}
}
else if( state.AvailableData > 7)
{
byte[] buffer = new byte[8];
int readBytes = state.Read(buffer, 0, 8);
if (readBytes == 8)
{
Packet packet = ByteArrayToStructure<Packet>(buffer);
if (packet.PacketType == PACKET_FILETRANSFER_INITIATE)
{
IPEndPoint ip = state.RemoteEndPoint as IPEndPoint;
String filename = getUniqueFileName("" + ip.Address);
if (filename == null)
{
Program.MainForm.log("Error getting filename for " + ip);
state.EndConnection();
return;
}
byte[] data = new byte[state.AvailableData];
readBytes = state.Read(data, 0, state.AvailableData);
state.waitingFor = packet.DataSize - readBytes;
state.hFile = new FileStream(filename, FileMode.Append);
state.bw = new BinaryWriter(state.hFile);
state.bw.Write(data);
state.bw.Flush();
state.fileTransfer = true;
Program.MainForm.log("Initiated file transfer with " + ip);
}
}
}
}
It receives all the data, when I debug my code and see that send() does not return the total data size (i.e. it has to be called more than once) and the image gets yellow lines or purple lines in it — I suspect there's something wrong with sending the data.
I mis-understood the question and solution intent. Thanks #Remy Lebeau for the comment to clarify that. Based on that, you can write a sendall() function as given in section 7.3 of http://beej.us/guide/bgnet/output/print/bgnet_USLetter.pdf
int sendall(int s, char *buf, int *len)
{
int total = 0; // how many bytes we've sent
int bytesleft = *len; // how many we have left to send
int n = 0;
while(total < *len) {
n = send(s, buf+total, bytesleft, 0);
if (n == -1) {
/* print/log error details */
break;
}
total += n;
bytesleft -= n;
}
*len = total; // return number actually sent here
return n==-1?-1:0; // return -1 on failure, 0 on success
}
You need to check the returnvalue of send(). In particular, you can't simply assume that it is the number of bytes sent, there is also the case that there was an error. Try this instead:
while(datasize != 0)
{
n = send(...);
if(n == SOCKET_ERROR)
throw exception("send() failed with errorcode #" + to_string(WSAGetLastEror()));
// adjust pointer and remaining number of bytes
datasize -= n;
data += n;
}
BTW:
Make that BYTE const* data, you're not going to modify what it points to.
The rest of your code seems too complicated, in particular you don't solve things by aligning to magic numbers like 512.

Artema Hybrid communication with c++ on Linux

I am currently trying to integrate a POS system with an Artema Hybrid CC handheld. I am wondering if anyone else has worked on this or something similar.
I can read from the device, that is I receive the ENQ, and send back an ACK in a thread, and I keep it open for reading/writing, but everything I try to write from it simply does nothing.
Here is the code for the function to write the data:
void PayLife::sendPayLifeData(QString data) {
int len = data.length();
int i = 0;
char lrc = 0;
char stx = 0x02;
char etx = 0x03;
char ack = 0x06;
char * bytes;
int ret;
char buffer[132];
bytes = (char *) malloc(sizeof(char) * len + 10);
strcpy(bytes,data.toLatin1().data());
qDebug() << "PayLife Sending data: " << data << " of len " << QString::number(len) <<
" " << " Bytes is: " << bytes ;
while (i < len) {
lrc ^= bytes[i];
i++;
}
/* sprintf(buffer,"%c%c%s%c%c",ack,stx,bytes,etx,lrc);
for (i = 0; i < strlen(buffer); i++) {
printf("c: %X ", buffer[i]);
}
printf(" [[ %s ]] \n", buffer); */
qDebug() << "Starting";
write(this->descriptor,&ack,1);
usleep(100000);
write(this->descriptor,&stx,1);
usleep(100000);
ret = write(this->descriptor,bytes,132);
usleep(100000);
write(this->descriptor,&etx,1);
usleep(100000);
write(this->descriptor,&lrc,1);
qDebug() << "Done";
free(bytes);
}
The data argument is: E11U000008507000099VZ000000
Of course, the documentation is in German, which I don't speak, so this is as far as I have gotten. I've basically got 1 month to implement this then I have to give the device back.
If anyone has any pointers, or some example code that would be awesome.
/jason
The transport protocol looks like something standard, so maybe you should only send len bytes of the data not 132 and include the ETX character in the lrc summing ?
It might be easier and clearer to use QByteArray instead of malloc arrays or QString (QString are for user displayed strings which is not the case here):
void PayLife::sendPayLifeData(const QByteArray & data) {
static const char stx = 0x02;
static const char etx = 0x03;
static const char ack = 0x06;
QByteArray buffer = stx + data + etx;
// Calculate the xor sum on data + etx
char lrc = 0;
for(int i = 1; i < buffer.size(); ++i) {
lrc ^= buffer[i];
}
buffer += lrc;
qDebug() << "Starting";
write(this->descriptor, &ack, 1);
write(this->descriptor, buffer.data(), buffer.size());
qDebug() << "Done";
}
You should also have a look at QSerialDevice, it could allow you to implement the protocol in a more event driven way with signal and slots.

problem with sending bytes from client to server using TCP

My send() and recv() looks like this:
int Send(const char* buffer, int size)
{
cout << "SIZE: " << size << endl;
int offset;
while(offset < size)
{
int n = ::send(getSocket(), buffer + offset, size - offset, 0);
if(n == SOCKET_ERROR)
{
break;
}
offset += n;
if(offset != size)
{
Sleep(1);
}
}
return offset;
}
int Recv(char* buffer, int size)
{
int n = ::recv(getSocket(), buffer, size, 0);
if(n == SOCKET_ERROR)
{
cout << "Error receiving data" << endl;
}
if(n == 0)
{
cout << "Remote host closed connection" << endl;
}
return n;
}
But my output show kind of many bytes sent that seems strange to me:
Received from client: 669
Sent to web server: 3990336
So it should supose to sent 669 bytes, so from where did it get 3990336 ? It is some kind of error or ?
Thanks.
Did you notice that int offset; is not initialize ?
You have to initialize offset with zero. Otherwise it could be any random value.
You do not need Sleep as send call is blocking.
Buffer that you are sending could be split. So if you send, for example, 2K buffer, you could get it in two parts - 1.5K and 0.5K, so you have to perform multiple reads on a client side. MTU is usually set to 1500 bytes.
Maybe it's just your (stripped down?) example code, but you never actually initialize offset. It might have any value, e.g. -5000 and will cause the loop to send 5669 bytes.