I have C++ service which sends whar_t strings to nodejs script. To send messages it uses IOCP WSASend. The problem is that pretty often nodejs can't receive (decode) those messages correctly. the message itself is a serialized json with SOH (\u0001) in front and "\r" at the end.
Message is pretty big and nodejs script receives at in chunks. It keeps concatenating the chunks until it finds "\r"
var pool = "";
socket.on('data', function (data) {
pool += Buffer.from(data.toString(), 'utf16le').toString();
var list = pool.split('\u0001');
for (var i in list) {
pool = '';
if (!list[i].length) {
continue;
}
if (list[i].substr(0, 1) != '{') {
continue;
}
if (list[i].substr(-1, 1) != '\r') {
pool = list[i];
break;
}
var cmd = JSON.parse(list[i]);
}
}
So it receives the chunk, append to the existing string and checks if the end reached. Works well until the broken chunk comes. I wrote the chunks to the file to track and found out that the broken chunk looks like Chinese. I suspect that the chunk could be started from the odd byte and full message shifted one byte. Needless to say that the message end "\r" will never be found. And even if found never parsed properly.
OVERLAPPED structure I send with WSASend looks like that (need SmartBuffer because same message could be sent to many sockets):
typedef std::shared_ptr<wchar_t> SmartBuffer;
class OVERLAPPED_EX : public OVERLAPPED
{
public:
WSABUF m_wsabuf;
UCHAR m_type;
SOCKET m_socket;
DWORD bytes;
SmartBuffer m_buffer;
OVERLAPPED_EX(SOCKET s, UCHAR t, SmartBuffer buf) : OVERLAPPED(),
m_socket(s), m_type(t), bytes(0)
{
hEvent = NULL;
m_buffer = buf;
m_wsabuf.buf = (char*)m_buffer.get();
m_wsabuf.len = wcslen((wchar_t*)buf.get()) * sizeof(wchar_t);
};
~OVERLAPPED_EX() {};
};
How to send widechar messages from c++ to nodejs or receive in nodejs properly please?
Related
I am sending udp packages from server to client. At the server side I split data into packages by 500 bytes, and sent to client. The client receives the packages and accumulate received data and deserializes an object.
The problem is that client receive 133 packages maximum and stops like nothing else was sent to socket, but server send whole object (1238 packages). And this problem exists in Windows only, but works perfectly under OSX.
Here is a server code sending packages:
// sends #buffer of size #length to #endpoint
// #buffer already contains a header, and the method splits #buffer into chunks and send it one by one
void server::send_package(char* buffer, int length, udp::endpoint endpoint){
if (length > BUFFER){
protocol::header header;
int dataLength = length - sizeof (header);
// copy header from buffer
memcpy(&header, buffer, sizeof(header));
header.isEnd = false;
int position = 0;
// allocate memory to collect data to send
char* data_to_send = new char[dataLength];
// copy data
memcpy(data_to_send, &buffer[sizeof(header)], dataLength);
header.totalPackages = dataLength/(BUFFER-sizeof (header));
// create chucks of data and send
while (position < dataLength){
int frame_size = BUFFER;
header.currentPackage++;
if (dataLength-position+sizeof (header) <= BUFFER) {
header.isEnd = true;
frame_size = dataLength-position+sizeof (header);
}
char* temp_buffer = new char[frame_size];
header.length = frame_size-sizeof(header);
// set the header of a chunk
memcpy(temp_buffer, &header, sizeof(header));
// set data to chunk
memcpy(&temp_buffer[sizeof (header)], &data_to_send[position], frame_size-sizeof(header));
// send chunk
socket->send_to(boost::asio::buffer(temp_buffer, frame_size), endpoint);
socket->wait(boost::asio::ip::tcp::socket::wait_write);
position += frame_size-sizeof(header);
}
} else {
socket->async_send_to(boost::asio::buffer(buffer, length), endpoint,
boost::bind(&server::release_sent_buffer,
this,
buffer, length)
);
}
}
Here is the client receives packages:
void connectionManager::handle_receive( const boost::system::error_code &error,
std::size_t size,
udp::endpoint* ep) {
if (size > 0) {
// _lock.try_lock();
protocol::header header;
memcpy(&header, &recv_buffer, sizeof(header));
logg("response from server received " + boost::asio::ip::address_v4(header.ip).to_string());
logg("received header:");
logg(protocol::getHeaderInfo(header));
std::stringstream ss;
ss << "header.length = " << header.length;
logg(ss.str().c_str());
udp::endpoint endpoint(boost::asio::ip::address_v4(header.ip), _server_port);
switch (header.command) {
case protocol::commands::server_instance_instruments_state_response: {
package_chain chain(size-sizeof(header));
memcpy(chain.data, &recv_buffer[sizeof(header)], size-sizeof(header));
packages[header.id].push_back(chain);
// at Windows machine the last package is #133. But 1248 packages expected.
// WHY????...
int packs = (packages[header.id].size());
if (header.isEnd) {
char* buf = getDataFromPackages(header.id, header.length);
std::stringstream str;
str << buf;
boost::archive::text_iarchive ar(str);
instance_plugin_information* inst_inf;
inst_inf = new instance_plugin_information();
try {
ar & inst_inf;
if (onPluginStateResponse != nullptr) {
onPluginStateResponse(*inst_inf);
}
} catch (const std::exception& e) {
}
}
break;
}
}
// We will hang on this line when package #133 received.
socket->wait(boost::asio::ip::tcp::socket::wait_read);
connectionManager::start_receive();
}
I just don't understand what I am missing? Why client receives exactly 133 packages (133 x 500 bytes) and then drops?
I have changed the code in many ways, but with no luck. The last thing I added is
socket->wait(boost::asio::ip::tcp::socket::wait_read);
before I call start_receive() again, and the program hands on this line exactly when package #133 is received.
Please help. I am close to give up and become a pizza delivery guy.
I wrote two programs, one as server and another as client. The server is written in standard C++ using WinSock2.h. It is a simple echo server which means the server responds what it receives back to the client. I used a new thread for every client's connection, and in each thread:
Socket* s = (Socket*) a;
while (1) {
std::string r = s->ReceiveLine()
if (r.empty()) {
break;
}
s->SendLine(r);
}
delete s;
return 0;
Socket is a class from here. The server runs properly and I've tested it using telnet, it works well.
Then I wrote the client using C++.NET (or C++/CLI), TcpClient is used to send and receive message from the server. The code is like:
String^ request = "test";
TcpClient ^ client = gcnew TcpClient(server, port);
array<Byte> ^ data = Encoding::ASCII->GetBytes(request);
NetworkStream ^ stream = client->GetStream();
stream->Write(data, 0, data->Length);
data = gcnew array<Byte>(256);
String ^ response = String::Empty;
int bytes = stream->Read(data, 0, data->Length);
response = Encoding::ASCII->GetString(data, 0, bytes);
client->Close();
When I run the client and tries to show the response message onto my form, the program halted at the line int bytes = stream->Read(data, 0, data->Length); and cannot fetch the response. The server is running and there's nothing to do with the network as they are all running on the same computer.
I guess the reason is that the data server responds with is less than data->Length, so the Read method is waiting for more data. Is that right? How should I solve this problem?
Edit
I think I've solved the problem... There are another two methods in the Socket class, ReceiveBytes and SendBytes, and these two methods will not delete the unused space in the bytes array. So the length of data back from the server will match that from the client, thus the Read method will not wait for more data to come.
std::string Socket::ReceiveLine() {
std::string ret;
while (1) {
char r;
switch(recv(s_, &r, 1, 0)) {
case 0: // not connected anymore;
// ... but last line sent
// might not end in \n,
// so return ret anyway.
return ret;
case -1:
return "";
// if (errno == EAGAIN) {
// return ret;
// } else {
// // not connected anymore
// return "";
// }
}
ret += r;
if (r == '\n') return ret;
}
}
i guess receiveline function of the server is waiting for an enter key '\n'.
just try with "test\n" string.
String^ request = "test\n";
// other codes....
I have written a client/server application where the server spawns multiple threads depending upon the request from client.
These threads are expected to send some data to the client(string).
The problem is, data gets overwritten on the client side. How do I tackle this issue ?
I have already read some other threads on similar issue but unable to find the exact solution.
Here is my client code to receive data.
while(1)
{
char buff[MAX_BUFF];
int bytes_read = read(sd,buff,MAX_BUFF);
if(bytes_read == 0)
{
break;
}
else if(bytes_read > 0)
{
if(buff[bytes_read-1]=='$')
{
buff[bytes_read-1]='\0';
cout<<buff;
}
else
{
cout<<buff;
}
}
}
Server Thread code :
void send_data(int sd,char *data)
{
write(sd,data,strlen(data));
cout<<data;
}
void *calcWordCount(void *arg)
{
tdata *tmp = (tdata *)arg;
string line = tmp->line;
string s = tmp->arg;
int sd = tmp->sd_c;
int line_no = tmp->line_no;
int startpos = 0;
int finds = 0;
while ((startpos = line.find(s, startpos)) != std::string::npos)
{
++finds;
startpos+=1;
pthread_mutex_lock(&myMux);
tcount++;
pthread_mutex_unlock(&myMux);
}
pthread_mutex_lock(&mapMux);
int t=wcount[s];
wcount[s]=t+finds;
pthread_mutex_unlock(&mapMux);
char buff[MAX_BUFF];
sprintf(buff,"%s",s.c_str());
sprintf(buff+strlen(buff),"%s"," occured ");
sprintf(buff+strlen(buff),"%d",finds);
sprintf(buff+strlen(buff),"%s"," times on line ");
sprintf(buff+strlen(buff),"%d",line_no);
sprintf(buff+strlen(buff),"\n",strlen("\n"));
send_data(sd,buff);
delete (tdata*)arg;
}
On the server side make sure the shared resource (the socket, along with its associated internal buffer) is protected against the concurrent access.
Define and implement an application level protocol used by the server to make it possible for the client to distinguish what the different threads sent.
As an additional note: One cannot rely on read()/write() reading/writing as much bytes as those two functions were told to read/write. It is an essential necessity to check their return value to learn how much bytes those functions actually read/wrote and loop around them until all data that was intended to be read/written had been read/written.
You should put some mutex to your socket.
When a thread use the socket it should block the socket.
Some mutex example.
I can't help you more without the server code. Because the problem is probably in the server.
I've been stuck on this issue for awhile where I'm unable to send a file through a socket. I've sent other information just fine using this method, but the problem seems to appear when I try to send a PNG file as a string.
These are the methods I use to to send and receive information:
// Sends a Message to the specified Socket
void Server::sendMessage(int socket, string message)
{
// Write the Message Size to the Socket
send(socket, itoa((message.length() + 1)), sizeof(size_t));
// Wait for Write Confirmation
bool response;
receive(socket, &response, 2);
// Write the Message to the Socket
send(socket, (char*) message.c_str(), message.length() + 1);
// Wait for Write Confirmation
receive(socket, &response, 2);
}
// Receives Message from the specified Socket
string Server::receiveMessage(int socket)
{
// Read the Message Size from the Socket
int size;
receive(socket, &size, sizeof(size_t));
// Send Write Confirmation
send(socket, itoa(true), 2);
// Receive the Message from the Socket
char message[size];
receive(socket, message, size);
// Send Write Confirmation
send(socket, itoa(true), 2);
// Return the Message as a String
string msg(message);
return msg;
}
The send and receive methods are just relays for write and read respectively. I'm only doing error checking in those methods, and it's the send method that's telling me that the write isn't working. In case it matters, this is my send method:
// Sends a Data Packet to the specified Socket
int Server::send(int socket, void* data, int size)
{
// Write the Data to the Socket
int count = write(socket, data, size);
// Make sure the Write Succeeded
if(count == -1)
{
print("$f1Error: $f0Unable to Write to Socket $t1%i$t0\n", socket);
exit(1);
}
return count;
}
I should note that the Server operates as a Thread, therefore the above three functions are static. The Client also contains the same four networking functions.
The command line breaking this happens in a separate static function which I use to handle Clients. Here is the relevant portion of said method:
// Handles each Client with a Thread
void* Server::server_handleClient(void* arg)
{
// Determine the Socket Descriptor
int socket = *((int*) arg);
free(arg);
// Create the Rover
Rover* rover = new Rover();
// Loop Indefinitely
while(true)
{
...
// Take a Picture and Send it
sendMessage(socket, rover -> takePicture());
...
}
// Delete the Rover
delete rover;
// Close the Socket
close(socket);
// Return a Successful Status
return (void*) new int(0);
}
Here you can see that I make use of a method from another class I've created. Here is the takePicture method from the Rover class, which is where I actually grab the picture:
// Takes a Picture and Returns the Photo as a String
inline string Rover::takePicture()
{
// Open the Picture File
ifstream picture;
string filepath = "./Server/Pictures/" + getDirection() + ".png";
picture.open(filepath.c_str());
// Make sure the File Opened
if(!picture.is_open())
return "";
// Read the File into a String Buffer
stringstream buffer;
buffer << picture.rdbuf();
return buffer.str();
}
So in short, the Server gets a picture from the Rover which it then sends to a Client. When I check the contents of the string for the photo, it's all there. All possible photos are reasonable in size (the photo used for testing is 674,962 bytes, and the buffer size sent is 674,963 which is expected).
I've used these methods for sending various messages, and all of that worked fine. I'm able to send strings (Like "Hello World!") and integers just fine.
Is there something that I'm doing wrong? Is the file that I'm trying to send simply too large? Is there some information that I'm missing? I need help...
Edit:
I've made a few changes with a little progress. I made one small change to the sendMessage command. The current problem is that the picture isn't being sent properly.
New sendMessage function:
// Sends a Message to the specified Socket
void Server::sendMessage(int socket, string message, bool data = false)
{
// Write the Message Size to the Socket
send(socket, itoa((message.length() + 1)), sizeof(size_t));
// Wait for Write Confirmation
bool response;
receive(socket, &response, 2);
// Determine the Type of Data to Send
if(data)
{
// Write the Message Data to the Socket
send(socket, (char*) message.data(), message.length() + 1);
}
else
{
// Write the Message to the Socket
send(socket, (char*) message.c_str(), message.length() + 1);
}
// Wait for Write Confirmation
receive(socket, &response, 2);
}
The Client's copy of this function has been updated to match as well.
Now that we're working on getting the PNG file saved, here's the function that deals with that as well:
// Handles each Client with a Thread
void* Client::client_handleServer(void* arg)
{
// Define Socket Variables
int socket = *((int*) arg);
free(arg);
...
// Export the Picture to the Client's Directory
message = receiveMessage(socket);
ofstream picture;
picture.open("./Client/Pictures/Picture.png", std::ifstream::binary);
picture << message;
picture.close();
...
}
Currently you are opening the file in textmode. that means any characters in the files which contain newlines "\n" are converted to new line + carriage returns "\r\n".
Open your file in binary mode, like so
picture.open(filepath.c_str(), std::ifstream::binary);
then it may work.
void Server::sendMessage(int socket, string message)
The problem is right here. Don't use string as a container for binary data. Pass the image around as a byte array. Same applies to this:
string Server::receiveMessage(int socket)
I eventually figured everything out in the long run.
Pictures are binary files, and I was using Strings which use ASCII Characters. The issue with this is that binary data does not always translate to ASCII, and Strings are terminated by null characters, whereas binary data can contain null data within it. Long story short, strings do not work.
To preserve the message handling I had in place, I ended up just converting the binary data to hexadecimal data (0-F) which could be displayed in a String.
I'm using Winsock to send commands through Telnet ; but for some reason when I try to send a string, a few characters get dropped occasionally. I use send:
int SendData(const string & text)
{
send(hSocket,text.c_str(),static_cast<int>(text.size()),0);
Sleep(100);
send(hSocket,"\r",1,0);
Sleep(100);
return 0;
}
Any suggestions?
Update:
I checked and the error still occurs even if all the characters are sent. So I decided to change the Send function so that it sends individual characters and checks if they have been sent:
void SafeSend(const string &text)
{
char char_text[1];
for(size_t i = 0; i <text.size(); ++i)
{
char_text[0] = text[i];
while(send(hSocket,char_text,1,0) != 1);
}
}
Also, it drops characters in a peculiar way ; i.e. in the middle of the sentence. E.g.
set variable [fp]exit_flag = true
is sent as
ariable [fp]exit_flag = true
Or
set variable [fp]app_flag = true
is sent as
setrable [fp]app_flag = true
As mentioned in the comments you absolutely need to check the return value of send as it can return after sending only a part of your buffer.
You nearly always want to call send in a loop similar to the following (not tested as I don't have a Windows development environment available at the moment):
bool SendString(const std::string& text) {
int remaining = text.length();
const char* buf = text.data();
while (remaining > 0) {
int sent = send(hSocket, buf, remaining, 0);
if (sent == SOCKET_ERROR) {
/* Error occurred check WSAGetLastError() */
return false;
}
remaining -= sent;
buf += sent;
}
return true;
}
Update:
This is not relevant for the OP, but calls to recv should also structured in the same way as above.
To debug the problem further, Wireshark (or equivalent software) is excellent in tracking down the source of the problem.
Filter the packets you want to look at (it has lots of options) and check if they include what you think they include.
Also note that telnet is a protocol with numerous RFCs. Most of the time you can get away with just sending raw text, but it's not really guaranteed to work.
You mention that the windows telnet client sends different bytes from you, capture a minimal sequence from both clients and compare them. Use the RFCs to figure out what the other client does different and why. You can use "View -> Packet Bytes" to bring up the data of the packet and can easily inspect and copy/paste the hex dump.