How to copy Buffer bytes block in Poco C++? - c++

Hi i am trying to write a TCP connection in poco. the client sends a packet with this fields :
packetSize : int
date : int
ID : int
so the first 4 bytes contains the packet size. in the receive side i have this code :
int packetSize = 0;
char *bufferHeader = new char[4];
// receive 4 bytes that contains packetsize here
socket().receiveBytes(bufferHeader, sizeof(bufferHeader), MSG_WAITALL);
Poco::MemoryInputStream *inStreamHeader = new Poco::MemoryInputStream(bufferHeader, sizeof(bufferHeader));
Poco::BinaryReader *BinaryReaderHeader = new Poco::BinaryReader(*inStreamHeader);
(*BinaryReaderHeader) >> packetSize; // now we have the full packet size
Now I am trying to store all remaining incoming bytes into one array for future binary reading :
int ID = 0;
int date = 0;
int availableBytes = 0;
int readedBytes = 0;
char *body = new char[packetSize - 4];
do
{
char *bytes = new char[packetSize - 4];
availableBytes = socket().receiveBytes(bytes, sizeof(bytes), MSG_WAITALL);
if (availableBytes == 0)
break;
memcpy(body + readedBytes, bytes, availableBytes);
readedBytes += availableBytes;
} while (availableBytes > 0);
Poco::MemoryInputStream *inStream = new Poco::MemoryInputStream(body, sizeof(body));
Poco::BinaryReader *BinaryReader = new Poco::BinaryReader(*inStream);
(*BinaryReader) >> date;
(*BinaryReader) >> ID;
cout << "date :" << date << endl;
cout << "ID :" << ID << endl;
the problem is the byte block of body is not storing the remaining bytes , it has always only the first 4 bytes (date). so in the out put the date is correct but the ID is not as expected. I tried to Stream it without Block copy and manually receive the each field without loop, it was just fine and had expected data. but when i try to store the incoming bytes into one array and then pass that array to a memorystream to read it, i have only the first block correct and expected!!
I really need to store all incoming bytes into one array and then read whole that array, how should i change my code?
thanks alot

I see two errors in your code. Or, more precisely, an error you do twice.
You confuse sizeof of char[] with sizeof of char *; the first is the number of characters in the array, the second is the size of the pointer: typically 4 or 8 bytes, depending on the memory model.
So, when you write
availableBytes = socket().receiveBytes(bytes, sizeof(bytes), MSG_WAITALL);
you are asking for 4 (I suppose) bytes. This is not serious as you continue to ask other bytes until the message is finished.
The real problem is the following instruction
Poco::MemoryInputStream *inStream = new Poco::MemoryInputStream(body, sizeof(body));
where you transfer only sizeof(char *) bytes in inStream
You should substitute sizeof(body) and sizeof(bytes) with packetSize - 4.
P.s.: sorry for my bad english
Edit: I've seen another error. In this instruction
char *bytes = new char[packetSize - 4];
you allocate packetSize - 4 chars. This memory in never deleted and in allocated in the do ... while() cycle.
You can allocate bytes outside of the cycle (togheter with body).
Edit 2016.03.17
Proposed solution (caution: non tested)
size_t toRead = packetSize - 4U;
size_t totRead = 0U;
size_t nowRead;
char * body = new char[toRead];
do
{
nowRead += socket().receiveBytes(body+totRead, toRead-totRead,
MSG_WAITALL);
if ( 0 == nowRead )
throw std::runtime_error("shutdown from receiveBytes()");
totRead += nowRead;
} while ( totRead < toRead );
Poco::MemoryInputStream *inStream = new Poco::MemoryInputStream(body,
toRead);
delete[] body;
body = NULL;

Related

ESP8266 getting garbage data from char array

I'm honestly at a loss here. I'm trying to store an SSID and password that the user sends through a post request in the flash EEPROM section. To do that I convert the data sent from the post request to a char array and index it to EEPROM. The SSID runs without any problems, but the password always ends up with junk data before it even gets to EEPROM.
Here is the bit of code in question:
// Recieve data from the HTTP server
void changeConfig(String parameter, String value){
int memoffset = 0;
if(parameter == "ssid")
memoffset = 0;
else if(parameter == "pass")
memoffset = 32;
else
return;
#ifdef DEBUG
Serial.println("Updating Data");
Serial.print("Param: ");
Serial.println(parameter);
Serial.print("Value: ");
Serial.println(value);
#endif
EEPROM.begin(64);
char _data[sizeof(value)];
value.toCharArray(_data, sizeof(value));
for(int i = memoffset; i < memoffset + sizeof(value); i++)
{
#ifdef DEBUG
Serial.print("addr ");
Serial.print(i);
Serial.print(" data ");
Serial.println(_data[i]);
#endif
EEPROM.write(i,_data[i]);
}
EEPROM.end();
}
And the Serial monitor output:
Post parameter: ssid, Value: NetworkName
Updating Data
Param: ssid
Value: NetworkName
addr 0 data N
addr 1 data e
addr 2 data t
addr 3 data w
addr 4 data o
addr 5 data r
addr 6 data k
addr 7 data N
addr 8 data a
addr 9 data m
addr 10 data e
addr 11 data ␀
Post parameter: pass, Value: Networkpass
Updating Data
Param: pass
Value: Networkpass
addr 32 data |
addr 33 data (
addr 34 data �
addr 35 data ?
addr 36 data L
addr 37 data ␛
addr 38 data �
addr 39 data ?
addr 40 data ␁
addr 41 data ␀
addr 42 data ␀
addr 43 data ␀
As you can see, when the name of the POST parameter is ssid, it works alright. With pass on the other hand, the char array is just filled with gibberish. Any insight would be helpful. I'm using platformio in the arduino environment. Generic ESP01 with 1M of flash.
Thanks in advance.
You have two problems with your code.
First, you are using sizeof incorrectly. Sizeof returns the size of the String object, but you are trying to get the length of the contained string. Sizeof is not the right tool for that, instead you should use whatever API String offers to read the size of the string.
The next problem is your usage of offsets. The following code snippet is all wrong:
char _data[sizeof(value)];
value.toCharArray(_data, sizeof(value));
for(int i = memoffset; i < memoffset + sizeof(value); i++)
{
...
EEPROM.write(i,_data[i]);
Your i starts with offset of 32, so you are trying to access element with index 32 in your _data array. But _data stores characters starting from the index 0, and since the length of array is actually 12 (sizeof of String is always 12) by accessing element with index 32 you are going beyond it's bounds, and obviously find garbage there (in C++ parlance, it is called undefined behavior).
Last, but not the least, C++ is an extremely complicated language, which can't be learned by 'trial and error'. Instead, you need to methodically study, preferably using one of the good C++ books. The list of those can be found here: The Definitive C++ Book Guide and List
You're using sizeof() incorrectly.
sizeof() tells you the size of the object, at compile time.
Try this experiment - run this code:
#include <Arduino.h>
void setup() {
String x("");
String y("abc");
String z("abcdef");
Serial.begin(115200);
delay(1000);
Serial.println(sizeof(x));
Serial.println(sizeof(y));
Serial.println(sizeof(z));
}
void loop() {
}
On my ESP8266 this outputs:
12
12
12
That's because it takes 12 bytes using this development environment to represent a String object (it might be different on a different CPU and compiler). The String class dynamically allocates storage, so sizeof can tell you nothing about how long the string itself is, only the compile-time size of the object.
For the String class, you should use its length() method. Your lines:
char _data[sizeof(value)];
value.toCharArray(_data, sizeof(value));
for(int i = memoffset; i < memoffset + sizeof(value); i++)
should be written as
char _data[value.length()];
value.toCharArray(_data, value.length());
for(int i = memoffset; i < memoffset + value.length(); i++)
For more information see the documentation on the String class.
You'll likely still have issues with string terminators. C and C++ terminate char array strings with the null character '\0', adding an extra byte to the length of the strings. So your code should more likely be:
void changeConfig(String parameter, String value){
int memoffset = 0;
if(parameter == "ssid")
memoffset = 0;
else if(parameter == "pass")
memoffset = 33;
else
return;
#ifdef DEBUG
Serial.println("Updating Data");
Serial.print("Param: ");
Serial.println(parameter);
Serial.print("Value: ");
Serial.println(value);
#endif
EEPROM.begin(66);
char _data[value.length() + 1];
value.toCharArray(_data, value.length() + 1);
for(int i = memoffset; i < memoffset + value.length() + 1; i++)
{
#ifdef DEBUG
Serial.print("addr ");
Serial.print(i);
Serial.print(" data ");
Serial.println(_data[i]);
#endif
EEPROM.write(i,_data[i]);
}
EEPROM.end();
}
to allow the string terminators to work correctly for 32 character SSIDs and passwords. But the fundamental issue that's breaking your code is the incorrect use of sizeof.

Modbus Registers beyond 255 cannot be accessed SimpleModbus

I'm french student and I have been playing with SimpleModbus for 1 week, and it worked fine... Until I tried to implement a slave with a large amount of registers (1000 needed). Registers beyond 255 using slave 1 cannot be accessed. Whatever I do I get a timeout error as soon as the registers reading go beyond 255 :
control.modbus.CommunicationException: While reading holding registers 255 on slave 1
The documentation says nothing about such a limitation. Reading
SimpleModbusSlave.cpp didn't help ... maybe I need to change the starting address in the "modbus update" function but I am too beginner to understand...
unsigned int modbus_update() {
if (*(ModbusPort).available())
{
unsigned char buffer = 0;
unsigned char overflow = 0;
while ((*ModbusPort).available())
{
// The maximum number of bytes is limited to the serial buffer size of 128 bytes
// If more bytes is received than the BUFFER_SIZE the overflow flag will be set and the
// serial buffer will be red untill all the data is cleared from the receive buffer.
if (overflow)
(*ModbusPort).read();
else
{
if (buffer == BUFFER_SIZE)
overflow = 1;
frame[buffer] = (*ModbusPort).read();
buffer++;
}
delayMicroseconds(T1_5); // inter character time out
}
// If an overflow occurred increment the errorCount
// variable and return to the main sketch without
// responding to the request i.e. force a timeout
if (overflow)
return errorCount++;
// The minimum request packet is 8 bytes for function 3 & 16
if (buffer > 7)
{
unsigned char id = frame[0];
broadcastFlag = 0;
if (id == 0)
broadcastFlag = 1;
if (id == slaveID || broadcastFlag) // if the recieved ID matches the slaveID or broadcasting id (0), continue
{
unsigned int crc = ((frame[buffer - 2] << 8) | frame[buffer - 1]); // combine the crc Low & High bytes
if (calculateCRC(buffer - 2) == crc) // if the calculated crc matches the recieved crc continue
{
function = frame[1];
unsigned int startingAddress = ((frame[2] << 8) | frame[3]); // combine the starting address bytes
unsigned int no_of_registers = ((frame[4] << 8) | frame[5]); // combine the number of register bytes
unsigned int maxData = startingAddress + no_of_registers;
unsigned char index;
unsigned char address;
unsigned int crc16;
// broadcasting is not supported for function 3
if (!broadcastFlag && (function == 3))
{
if (startingAddress < holdingRegsSize) // check exception 2 ILLEGAL DATA ADDRESS
{
if (maxData <= holdingRegsSize) // check exception 3 ILLEGAL DATA VALUE
{
unsigned char noOfBytes = no_of_registers * 2;
// ID, function, noOfBytes, (dataLo + dataHi)*number of registers,
// crcLo, crcHi
unsigned char responseFrameSize = 5 + noOfBytes;
frame[0] = slaveID;
frame[1] = function;
frame[2] = noOfBytes;
address = 3; // PDU starts at the 4th byte
unsigned int temp;
for (index = startingAddress; index < maxData; index++)
{
temp = regs[index];
frame[address] = temp >> 8; // split the register into 2 bytes
address++;
frame[address] = temp & 0xFF;
address++;
}
crc16 = calculateCRC(responseFrameSize - 2);
frame[responseFrameSize - 2] = crc16 >> 8; // split crc into 2 bytes
frame[responseFrameSize - 1] = crc16 & 0xFF;
sendPacket(responseFrameSize);
}
else
exceptionResponse(3); // exception 3 ILLEGAL DATA VALUE
}
else
exceptionResponse(2); // exception 2 ILLEGAL DATA ADDRESS
}
else if (function == 16)
{
// Check if the recieved number of bytes matches the calculated bytes
// minus the request bytes.
// id + function + (2 * address bytes) + (2 * no of register bytes) +
// byte count + (2 * CRC bytes) = 9 bytes
if (frame[6] == (buffer - 9))
{
if (startingAddress < holdingRegsSize) // check exception 2 ILLEGAL DATA ADDRESS
{
if (maxData <= holdingRegsSize) // check exception 3 ILLEGAL DATA VALUE
{
address = 7; // start at the 8th byte in the frame
for (index = startingAddress; index < maxData; index++)
{
regs[index] = ((frame[address] << 8) | frame[address + 1]);
address += 2;
}
// only the first 6 bytes are used for CRC calculation
crc16 = calculateCRC(6);
frame[6] = crc16 >> 8; // split crc into 2 bytes
frame[7] = crc16 & 0xFF;
// a function 16 response is an echo of the first 6 bytes from
// the request + 2 crc bytes
if (!broadcastFlag) // don't respond if it's a broadcast message
sendPacket(8);
}
else
exceptionResponse(3); // exception 3 ILLEGAL DATA VALUE
}
else
exceptionResponse(2); // exception 2 ILLEGAL DATA ADDRESS
}
else
errorCount++; // corrupted packet
}
else
exceptionResponse(1); // exception 1 ILLEGAL FUNCTION
}
else // checksum failed
errorCount++;
} // incorrect id
}
else if (buffer > 0 && buffer < 8)
errorCount++; // corrupted packet
}
return errorCount;
}
Slave: SimpleModbusSlave
Any thoughts ?
Edit : I think someone had the same problem but I really did not understand what he changed :( here
Thanks in advance to those who can enlighten me!
Thank you very much #Marker it works now! I changed my librarie for the slave: github.com/smarmengol/Modbus-Master-Slave-for-Arduino –

c++ insane memory consumption on large file

I am loading a 10GB file into memory and I find that even if I strip away any extra overhead and store the data in nothing but an array it still takes up 53 GB of ram. This seems crazy to me since I am converting some of the text data to longs which take up less room and convert the rest to char * which should take up the same amount of room as a text file. I have about 150M rows of data in the file I am trying to load. Is there any reason why this should take up so much ram when I load it the way I do below?
There are three files here a fileLoader class and its header file and a main that simply runs them.
To answer some questions:
OS is UBUNTU 12.04 64bit
This is on a machien with 64GB of RAM and an SSD hd that I have providing 64GB of swap space for RAM
I am loading all of the data at once becuase of the need for speed. It is critical for the application. All sorting, indexing, and lots of the data intensive work runs on the GPU.
The other reason is that loading all of the data at once made it much simpler for me to write the code. I dont have to worry about indexed files, and mappings to locations in another file for example.
Here is the header file:
#ifndef FILELOADER_H_
#define FILELOADER_H_
#include <iostream>
#include <fstream>
#include <fcntl.h>
#include <stdlib.h>
#include <string.h>
#include <string>
class fileLoader {
public:
fileLoader();
virtual ~fileLoader();
void loadFile();
private:
long long ** longs;
char *** chars;
long count;
long countLines(std::string inFile);
};
#endif /* FILELOADER_H_ */
Here is the CPP file
#include "fileLoader.h"
fileLoader::fileLoader() {
// TODO Auto-generated constructor stub
this->longs = NULL;
this->chars = NULL;
}
char ** split(char * line,const char * delim,int size){
char ** val = new char * [size];
int i = 0;
bool parse = true;
char * curVal = strsep(&line,delim);
while(parse){
if(curVal != NULL){
val[i] = curVal;
i++;
curVal = strsep(&line,delim);
}else{
parse = false;
}
}
return val;
}
void fileLoader::loadFile(){
const char * fileName = "/blazing/final/tasteslikevictory";
std::string fileString(fileName);
//-1 since theres a header row and we are skipinig it
this->count = countLines(fileString) -1;
this->longs = new long long*[this->count];
this->chars = new char **[this->count];
std::ifstream inFile;
inFile.open(fileName);
if(inFile.is_open()){
std::string line;
int i =0;
getline(inFile,line);
while(getline(inFile,line)){
this->longs[i] = new long long[6];
this->chars[i] = new char *[7];
char * copy = strdup(line.c_str());
char ** splitValues = split(copy,"|",13);
this->longs[i][0] = atoll(splitValues[4]);
this->longs[i][1] = atoll(splitValues[5]);
this->longs[i][2] = atoll(splitValues[6]);
this->longs[i][3] = atoll(splitValues[7]);
this->longs[i][4] = atoll(splitValues[11]);
this->longs[i][5] = atoll(splitValues[12]);
this->chars[i][0] = strdup(splitValues[0]);
this->chars[i][1] = strdup(splitValues[1]);
this->chars[i][2] = strdup(splitValues[2]);
this->chars[i][3] = strdup(splitValues[3]);
this->chars[i][4] = strdup(splitValues[8]);
this->chars[i][5] = strdup(splitValues[9]);
this->chars[i][6] = strdup(splitValues[10]);
i++;
delete[] splitValues;
free(copy);
}
}
}
fileLoader::~fileLoader() {
// TODO Auto-generated destructor stub
if(this->longs != NULL){
delete[] this->longs;
}
if(this->chars != NULL){
for(int i =0; i <this->count;i++ ){
free(this->chars[i]);
}
delete[] this->chars;
}
}
long fileLoader::countLines(std::string inFile){
int BUFFER_SIZE = 16*1024;
int fd = open(inFile.c_str(), O_RDONLY);
if(fd == -1)
return 0;
/* Advise the kernel of our access pattern. */
posix_fadvise(fd, 0, 0, 1); // FDADVICE_SEQUENTIAL
char buf[BUFFER_SIZE + 1];
long lines = 0;
while(size_t bytes_read = read(fd, buf, BUFFER_SIZE))
{
if(bytes_read == (size_t)-1)
return 0;
if (!bytes_read)
break;
for(char *p = buf; (p = (char*) memchr(p, '\n', (buf + bytes_read) - p)); ++p)
++lines;
}
return lines;
}
Here is the file with my main function:
#include "fileLoader.h"
int main()
{
fileLoader loader;
loader.loadFile();
return 0;
}
Here is an example of the data that I am loading:
13|0|1|1997|113|1|4|12408012|C9FF921CA04ADA3D606BF6DAC4A0B092|SEMANAL|66C5E828DC69F857ADE060B8062C923E|113|1
14|0|1|1997|113|1|5|12408012|C9FF921CA04ADA3D606BF6DAC4A0B092|SEMANAL|66C5E828DC69F857ADE060B8062C923E|113|1
15|0|1|1997|113|1|6|12408012|C9FF921CA04ADA3D606BF6DAC4A0B092|SEMANAL|66C5E828DC69F857ADE060B8062C923E|113|1
16|0|1|1997|113|1|7|12408012|C9FF921CA04ADA3D606BF6DAC4A0B092|SEMANAL|66C5E828DC69F857ADE060B8062C923E|113|1
17|0|1|1997|113|1|8|12408012|C9FF921CA04ADA3D606BF6DAC4A0B092|SEMANAL|66C5E828DC69F857ADE060B8062C923E|113|1
18|0|1|1997|113|1|9|12408012|C9FF921CA04ADA3D606BF6DAC4A0B092|SEMANAL|66C5E828DC69F857ADE060B8062C923E|113|1
19|0|1|1997|113|1|10|12408012|C9FF921CA04ADA3D606BF6DAC4A0B092|SEMANAL|66C5E828DC69F857ADE060B8062C923E|113|1
20|0|1|1997|113|1|11|12408012|C9FF921CA04ADA3D606BF6DAC4A0B092|SEMANAL|66C5E828DC69F857ADE060B8062C923E|113|1
21|0|1|1997|113|1|12|12408012|C9FF921CA04ADA3D606BF6DAC4A0B092|SEMANAL|66C5E828DC69F857ADE060B8062C923E|113|1
9|0|1|1997|113|1|13|12408012|C9FF921CA04ADA3D606BF6DAC4A0B092|SEMANAL|66C5E828DC69F857ADE060B8062C923E|113|1
27|0|1|1992|125|1|1|10183|9EF534D2CF74B24AC28CBD9BE937A412|SEMANAL|375CCE505F5353CCDE85D4E84A9888D8|125|1
28|0|1|1992|125|1|2|10183|9EF534D2CF74B24AC28CBD9BE937A412|SEMANAL|375CCE505F5353CCDE85D4E84A9888D8|125|1
29|0|1|1992|125|1|3|10183|9EF534D2CF74B24AC28CBD9BE937A412|SEMANAL|375CCE505F5353CCDE85D4E84A9888D8|125|1
30|0|1|1992|125|1|4|10183|9EF534D2CF74B24AC28CBD9BE937A412|SEMANAL|375CCE505F5353CCDE85D4E84A9888D8|125|1
31|0|1|1992|125|1|5|10183|9EF534D2CF74B24AC28CBD9BE937A412|SEMANAL|375CCE505F5353CCDE85D4E84A9888D8|125|1
32|0|1|1992|125|1|6|10183|9EF534D2CF74B24AC28CBD9BE937A412|SEMANAL|375CCE505F5353CCDE85D4E84A9888D8|125|1
33|0|1|1992|125|1|7|10183|9EF534D2CF74B24AC28CBD9BE937A412|SEMANAL|375CCE505F5353CCDE85D4E84A9888D8|125|1
34|0|1|1992|125|1|8|10183|9EF534D2CF74B24AC28CBD9BE937A412|SEMANAL|375CCE505F5353CCDE85D4E84A9888D8|125|1
35|0|1|1992|125|1|9|10183|9EF534D2CF74B24AC28CBD9BE937A412|SEMANAL|375CCE505F5353CCDE85D4E84A9888D8|125|1
36|0|1|1992|125|1|10|10183|9EF534D2CF74B24AC28CBD9BE937A412|SEMANAL|375CCE505F5353CCDE85D4E84A9888D8|125|1
37|0|1|1992|125|1|11|10183|9EF534D2CF74B24AC28CBD9BE937A412|SEMANAL|375CCE505F5353CCDE85D4E84A9888D8|125|1
38|0|1|1992|125|1|12|10183|9EF534D2CF74B24AC28CBD9BE937A412|SEMANAL|375CCE505F5353CCDE85D4E84A9888D8|125|1
39|0|1|1992|125|1|13|10183|9EF534D2CF74B24AC28CBD9BE937A412|SEMANAL|375CCE505F5353CCDE85D4E84A9888D8|125|1
40|0|1|1992|125|1|14|10183|9EF534D2CF74B24AC28CBD9BE937A412|SEMANAL|375CCE505F5353CCDE85D4E84A9888D8|125|1
41|0|1|1992|125|1|15|10183|9EF534D2CF74B24AC28CBD9BE937A412|SEMANAL|375CCE505F5353CCDE85D4E84A9888D8|125|1
10|0|1|1996|126|1|1||||||
You are allocating nine chunks of memory for each line, so you are allocating a total of 1350 million pieces of memory. These allocations have a certain overhead, usually at least twice the size of a pointer, possibly even more. On a 64 bit machine, that is already 16 bytes, so you get 21.6 GB of overhead.
In addition to that, you get the overhead of heap fragmentation and alignment: Even if you only ever store a string in it, the allocator has to align the memory allocations so that you can store the largest possible values in it without triggering misalignment. Alignment may depend on the vector unit of your CPU, which can require very significant alignments, 16 byte alignment not being uncommon.
Doing the calculation with 16 bytes allocation overhead and 16 bytes alignment, we get allocations of 43.2 GB without the original data. With the original data this calculation is already very close to your measurement.
Each of those objects and strings you create has individual memory management overhead. So you load the string "0" from column 2, depending on your memory manager, it probably takes between two and four full words (could be more). Call it 16 to 32 bytes of storage to hold a one byte string. Then you load the "1" from column 3. And so on.

C++ What's the proper way to receive data over tcp until I get as many as I want [duplicate]

I am trying to send a message over Socket in c++. I have read many questions on stack overflow related to this but couldn't still figure out how it works. lets say i am sending following characters(M,a,r,t,i,n) to a local host server, people suggest that you can use 4 bytes as the length(i.e 32 bits, so that it can handle a message up to 4GB length).
I did the same thing at my client side but still dont know how can i figure out this thing at server side whether i want to receive only starting 3 bytes(M,a,r) or last 3 bytes(t,i,n) of my data.
I am posting my code please help me mainly in the server side, will be thankfull if can write few lines with relevance to code.
Client side code
std::vector<char> userbuffer(20);
std::cout<<"\nclient:"<<std::endl;
char* p = userbuffer.data();
*p = 'M';
++p; *p = 'a';
++p; *p = 'r';
++p; *p = 't';
++p; *p = 'i';
++p; *p = 'n';
size_t length = strlen(userbuffer.data());
uint32_t nlength = htonl(length);
//line containg message length information
int header_info = send(socketFD, (char*)&nlength, 4, 0);
// Data bytes send to the server
int bytes_sent = send(socketFD, userbuffer.data(), length, 0);
if(bytes_sent == SOCKET_ERROR){ //some errror handling}
Server Side Code
char receivebuffer[MAX_DATA] = { '\0' };
int bytesReceivedFromClientMsg = 1;
int length_bytes = 0;
uint32_t length, nlength;
//code to check length if we have received whole data length
while(length_bytes < 4){
int read = recv(clientSocket, ((char*)&nlength)+length_bytes, (4-length_bytes), 0);
if (read == -1) { //error handling}
length_bytes += read;}
// Most painfull section to understand.
// I implemented this code from some ideas on internet
//but still cant find how its extracting length and what i am reading :(
while(bytesReceivedFromClientMsg > 0){
int msgheader = recv(clientSocket,(char*)&nlength,6, 0);
length = ntohl(nlength);//leng value here is in severel thousand size
char *receivebuffer = new char(length+1);
bytesReceivedFromClientMsg = recv(clientSocket, receivebuffer, msgheader, 0);
receivebuffer[length] = 0 ;
std::cout<<"msg header is :"<<msgheader<<std::endl;
std::cout<<"msg data is :"<<bytesReceivedFromClientMsg<<std::endl;
if(bytesReceivedFromClientMsg == SOCKET_ERROR){//some error handling}
You need a design for your network protocol. There are protocols like SMTP that are text-like protocols. You have to read characters until you find a termination character like the new-line in a text-like protocol.
With a message based protocol you have better chances for high performance protocol. You define a header (that is used in your code but not defined). In the header you put information about the length and probably about the type of the next message. Then you send the header in front of the message body. The body is "Martin" in your example.
The receiver has a state "header received". When the header is not received complete (or nothting at all) it will use the size of the header as chunk size. It receives chunksize bytes into the header variable. When the header is received complete the receiver sets the chunksize to the sized the is set in the header and receives so many bytes to the payload buffer. When this has been complete the state "header received" is false again.
int receive(socket sock, char * buffer, int chunk_size)
{
int offset = 0;
while (chunk_size > 0)
{
// add select() here when you have a non-blocking socket.
int n = recv(sock, buffer+offset, chunk_size);
// TODO: error handling
offset += n;
chunk_size -= n;
}
// return amount of received bytes
return offset;
}
void do_receive(void)
{
struct {
int size;
// other message information
} header;
while (true)
{
receive(sock, &header, sizeof(header);
receive(sock, buffer, header.size);
process_message(buffer, header.size);
}
}
The code above will not pass any compiler. But it shows the idea..

Error when trying to deallocate pointer of char array: _BLOCK_TYPE_IS_VALID(pHead->nBlockUse)

I'm writing a C++ program that sends and receives images using Boost.Asio.
When compiling I don't get errors, but when executing and having sent an image the program that receives the image crashes giving the following error message (in Visual Studio 2012, Windows 7 32bit):
Debug Assertion Failed:
Program: […]\DataSender.exe
File: f:\dd\vctools\crt_bld\self_x86\crt\src\dbgdel.cpp
Line: 52
Expression: _BLOCK_TYPE_IS_VALID(pHead->nBlockUse)
I read packages the size of 4096 bytes into a pointer to a char array while there are still incoming bytes to read. In the final looping—if there are less than 4096 bytes to read—I delete the pointer and create a pointer to a char array the size of the remaining bytes. Until here it still works.
But when I try to delete the char pointer array again at the end of the loop (in order to create a new char pointer array with standard size 4096 for the next incoming images), the program crashes.
Here is my code's excerpt in question:
char* buffer = new char[4096];
[…]
int remainingBytes = imageSize;
[…]
// read data
while( remainingBytes > 0 )
{
boost::system::error_code error;
// use smaller buffer if remaining bytes don't fill the tcp package
// fully
if( remainingBytes < 4096 )
{
delete[] buffer; // this one doesn't give an error
bufferSize = remainingBytes;
char* buffer = new char[bufferSize];
}
// read from socket into buffer
size_t receivedBytes = socket.read_some(
boost::asio::buffer(buffer, bufferSize), error);
remainingBytes -= receivedBytes;
// count total length
totalReceivedBytes += receivedBytes;
// add current buffer to totalBuffer
for( int i = 0; i < bufferSize; i++)
{
totalBuffer.push_back(buffer[i]);
}
// if smaller buffer has been used delete it and
// create usual tcp buffer again
if( receivedBytes < 4096 )
{
delete[] buffer; // here the error occurs
bufferSize = 4096;
char* buffer = new char[bufferSize];
}
}
I ran the same code also on a Debian GNU/Linux 7.2 64bit machine, which returned the following error, at the same position in code:
*** glibc detected *** ./datasender: double free or corruption (!prev): 0x0000000002503970 ***
I assume I'm doing something wrong when deallocating the char pointer array but I haven't figured it out yet.
Can someone point me in the right direction?
You're actually deleting twice the buffer when remainingBytes and receivedBytes are less than 4096.
Indeed, you're deleting buffer once, then allocate memory into a local buffer, not the outer one.
Then, when you delete buffer in the second if block, you're deleting a second time the same buffer. The allocation you've made in the if scopes are memory leaks. These aren't the same variables.
When you do
char* buffer = new char[bufferSize];
in your if scopes, you're creating a new variable, not allocating memory into the outer buffer variable. Thus, you're leaking, and not allocating memory into the buffer you just deleted.
Without looking further, you should remove the char* in front of buffer in both if blocks and then continue debugging.
I would use std::vector instead:
#include <vector>
//...
std::vector<char> buffer(remainingBytes);
bufferSize = remainingBytes;
//...
while( remainingBytes > 0 )
{
boost::system::error_code error;
// use smaller buffer if remaining bytes don't fill the tcp package
// fully
if( remainingBytes < 4096 )
{
buffer.resize(remainingBytes);
bufferSize = remainingBytes;
}
// read from socket into buffer
size_t receivedBytes = socket.read_some(
boost::asio::buffer(&buffer[0], bufferSize), error);
remainingBytes -= receivedBytes;
// count total length
totalReceivedBytes += receivedBytes;
// add current buffer to totalBuffer
totalBuffer.insert(totalBuffer.end(), buffer.begin(),
buffer.begin() + receivedBytes);
// if smaller buffer has been used delete it and
// create usual tcp buffer again
if( receivedBytes < 4096 )
{
buffer.resize(4096);
bufferSize = 4096;
}
}
There will be no memory leaks.
Also, I think your code has a bug in that you are supposed to copy only the number of received bytes (the return value of the read_some() function). Instead you assumed that bufferSize characters were returned.