I am trying to send a proto over a socket, but i am getting segmentation error. Could someone please help and tell me what is wrong with this example?
file.proto
message data{
required string x1 = 1;
required uint32 x2 = 2;
required float x3 = 3;
}
xxx.cpp
...
data data_snd, data_rec;
//sending data to the server
if (send(socket, &data_snd, sizeof(data_snd), 0) < 0) {
cerr << "send() failed" ;
exit(1);
}
//receiving data from the client
if (recv(socket, &data_rec, sizeof(data_rec), 0) < 0) {
cerr << "recv() failed";
exit(1);
}
Thanks for help and replies-
You're not supposed to write the protobuf object itself to the socket. Use the SerializeXXX family of methods to get a sequence of bytes which you can write to the socket.
std::string buf;
data.SerializeToString(&buf);
// now you can write buf.data() to the socket
For one thing, you're assuming that a single call to recv will retrieve all the data. More importantly though, you're not going through the serialization / deserialization code - you're just reading from the network and putting the bytes directly into the object. You should be using the stream-based APIs in Protocol Buffers to write and read the data.
See the Protocol Buffers C++ tutorial for more details - that gives an example of saving to disk, but I expect that the network version would be similar, just with a different stream.
Give more details about where it segfaults and how you manage the connection between server and client.
Use a debugger and print the backtrace, it will help ...
Related
I'm trying to use C/C++ to communicate with a CAN bus. I'm using sockets to read and write on the bus. I've created a write thread and a read thread. The read thread is constantly trying to read and the socket, and when a write request arrives, the write thread take control of the socket using a mutex and do the write.
I'm having a big issue with speed using this method, as a write request can sometimes take 500 ms to be completed (which is completely unfeasible for my application). I've tried to put a timeout on the read command to make it non-blocking when nothing comes on the bus, but if the timeout is too short, I have reliability issues with the read. On the other hand, if I make it too long, the speed increase is insufficient.
It's my first time working with CAN. Would you have some advices on implementation of fast, two-way CAN communication node in C/C++ ? Which library should I use to interface with the bus itself ? Which architecture would yield the lowest read and write latency ?
To give a few metrics for the application, the bus bitrate is 1MBits/sec, I'm using CAN-Open with 64 bits data packets (each message contains 32 bits for indexes and 32 bits of data). I would like a write frequency from 300 to 500hz, same for the read frequency.
Thanks a lot for your help !
EDIT :
Thanks a lot for all your comments. Here are some clarifications on my application and problems.
I'm working on a mobile robot project, and I'm using CAN-Open to communicate with motor drivers and sensors. The code will run on a Raspberry Pi CM4 running Raspbian OS mounted on a custom IO board integrating a MCP2515 CAN controller. I want to implement fast communication interface between the ROS architecture and the CAN bus. The language used could be either C or C++.
I'm currently using a homemade interface build around standard C sockets, but the speed is very low, and is a big bottleneck to the robot's performance. So I'm looking for a better solution, either:
An open-source library build for this purpose
Architecture suggestion to implement such a program
A combination of both
Here are the socket creation, the read and the write functions I use. Read and write being each called in a while loop in different threads (I'm using pthread):
bool connectCanBus(int* socketIDOut, std::string canInterfaceName){
// Socket and can variables
struct sockaddr_can addr;
struct ifreq ifr;
// Openning the socket to send commands over the can bus
if ((*socketIDOut = socket(PF_CAN, SOCK_RAW, CAN_RAW)) < 0) {
perror("[Can Controler] Unable to create socket.");
return false;
}
strcpy(ifr.ifr_name, canInterfaceName.c_str());
ioctl(*socketIDOut, SIOCGIFINDEX, &ifr);
memset(&addr, 0, sizeof(addr));
addr.can_family = AF_CAN;
addr.can_ifindex = ifr.ifr_ifindex;
// Setting option to gte errors as frames
can_err_mask_t err_mask = 0xFFFFFFFF;
setsockopt(*socketIDOut, SOL_CAN_RAW, CAN_RAW_ERR_FILTER, &err_mask, sizeof(err_mask));
struct timeval tv;
tv.tv_sec = 0;
tv.tv_usec = 10000;
setsockopt(*socketIDOut, SOL_SOCKET, SO_RCVTIMEO, (const char*)&tv, sizeof tv);
// Binding Socket
if (bind(*socketIDOut, (struct sockaddr *)&addr, sizeof(addr)) < 0)
{
perror("[Can Controler] Unable to bind socket.");
close(*socketIDOut);
return false;
}
ROS_INFO("CAN bus connected.");
return true;
}
int sendCommand(const char* id, const char* message, int socket, std::mutex& mutex)
{
struct canfd_frame frame;
int err = parseFrame(id, message, &frame);
if(err == 0){
ROS_ERROR_STREAM("[Can Utils] Unable to parse frame : " << id << ", " << message);
return 0;
}
mutex.lock();
int res = write(socket, &frame, sizeof(struct can_frame));
mutex.unlock();
if (res != sizeof(struct can_frame)) {
perror("[Can Utils] CAN bus Write error");
return 0;
}
return 1;
}
int readBus(CanFrame *outFrame, int socketID, std::mutex& mutex)
{
struct can_frame frame;
// Reading on bus
mutex.lock();
int nbytes = read(socketID, &frame, sizeof(struct can_frame));
mutex.unlock();
if (nbytes < 0) {
perror("[Can Utils] CAN bus Read error");
return 0;
}
// Converting frame to strings
sprint_canframe(outFrame, &frame);
return nbytes;
}
I hope this make the question clearer and better focused.
Thanks to all comments on the question, I was able to solve the speed issue. I was not aware that SocketCan supports multiple socket on the same bus. Being able to create per process sockets for read and write allowed to reduce talk to the bus in a distributed manner, greatly improving speed.
I'm trying to send data to the connected client, even when the client did not send me a message first.
This is my current code:
while (true) {
// open a new socket to transmit data per connection
int sock;
if ((sock = accept(listen_sock, (sockaddr *) &client_address, &client_address_len)) < 0) {
logger.log(TYPE::ERROR, "server::could not open a socket to accept data");
exit(0);
}
int n = 0, total_received_bytes = 0, max_len = 4096;
std::vector<char> buffer(max_len);
logger.log(TYPE::SUCCESS,
"server::client connected with ip address: " + std::string(inet_ntoa(client_address.sin_addr)));
// keep running as long as the client keeps the connection open
while (true) {
n = recv(sock, &buffer[0], buffer.size(), 0);
if (n > 0) {
total_received_bytes += n;
std::string str(buffer.begin(), buffer.end());
KV key_value = kv_from(vector_from(str));
messaging.set_command(key_value);
}
std::string message = "hmc::" + messaging.get_value("hmc") + "---" + "sonar::" + messaging.get_value("sonar") + "\n";
send(sock, message.c_str(), message.length(), 0);
}
logger.log(TYPE::INFO, "server::connection closed");
close(sock);
}
I thought by moving the n = recv(sock, &buffer[0], buffer.size(), 0); outside the while condition that it would send the data indefinitely, but that is not what happened.
Thanks in advance.
Solution
Adding MSG_DONTWAIT to the recv function enabled non-blocking operations which I was looking for.
First I will explain, why it does not work, then I will make a proposal for solutions. Basically you will find the answer in the man7.org > Linux > man-pages and for recv specifially here.
When the function "recv" is called, then it will not return, until data is available and can be read. This behavior of functions is called "blocking". Means, the current execution thread is blocked until data has been read.
So, calling the function
n = recv(sock, &buffer[0], buffer.size(), 0);
as you did, causes the trouble. You need also to check the return code. 0 means, connection closed, -1 means error and you must check errno for further information.
You can modify the socket to work in non-blocking mode with the function fnctl and the O_NONBLOCK flag, for the lifetime of the socket. You can also use the the flag MSG_DONTWAIT as 4th parameter (flags), to unblock the function on a per-function-call base.
In both cases, if no data is available, the functions returns a -1 and you need to check errno for EAGAIN or EWOULDBLOCK.
return value 0 indicates that the connection has been closed.
But from the architecture point of view, I would not recommend to use this approach. You could use multiple threads for receiving and sending data, or, using Linux, one of select, poll or similar functions. There is even a common design pattern for this. It is called "reactor", There are also related patterns like "Acceptor/Connector" and "Proactor"/"ACT" available. If you plan to write a more robust application, then you may consider those.
You will find an implementation of Acceptor, Connector, Reactor, Proactor, ACT here
Hope this helps
I'm sending and receiving info with a unix socket, but I do not completely understand how it works. Basically, I send a message like this:
int wr_bytes = write(sock, msg.c_str(), msg.length());
And receive message like this:
int rd_bytes = read(msgsock, buf, SOCKET_BUFFER_SIZE);
This code works perfectly with thousands of bytes, what I don't understand is, how does the read function knows when the other part is done sending the message? I tried to read the read documentation and, on my understanding read will return once it reaches EOF or the SOCKET_BUFFER_SIZE, is that correct?
So I'm guessing that when I give my string to the write function, it adds an EOF at the end of my content so the read function knows when to stop.
I'm asking this question because, I did not add any code that checks whether the other part finished sending the message, however, I'm receiving big messages (thousands of bytes) without any problem, why is that happening, why am I not getting only parts of the message?
Here is the full function I'm using to send a message to a unix socket server:
string sendSocketMessage(string msg) {
int sock;
struct sockaddr_un server;
char buf[1024];
sock = socket(AF_UNIX, SOCK_STREAM, 0);
if (sock < 0) {
throw runtime_error("opening stream socket");
}
server.sun_family = AF_UNIX;
strcpy(server.sun_path, "socket");
if (connect(sock, (struct sockaddr *) &server, sizeof(struct sockaddr_un)) < 0) {
close(sock);
throw runtime_error("connecting stream socket");
}
if (write(sock, msg.c_str(), msg.length()) < 0){
throw runtime_error("writing on stream socket");
close(sock);
}
bzero(buf, sizeof(buf));
int rval = read(sock, buf, 1024);
return string( reinterpret_cast< char const* >(buf), rval );
}
And here is my server function (a little bit more complicated, the type vSocketHandler represents a function that I call to handle requests):
void UnixSocketServer::listenRequests(vSocketHandler requestHandler){
int sock, msgsock, rval;
struct sockaddr_un server;
char buf[SOCKET_BUFFER_SIZE];
sock = socket(AF_UNIX, SOCK_STREAM, 0);
if (sock < 0) {
throw runtime_error("opening stream socket");
}
server.sun_family = AF_UNIX;
strcpy(server.sun_path, SOCKET_FILE_PATH);
if (bind(sock, (struct sockaddr *) &server, sizeof(struct sockaddr_un))) {
throw runtime_error("binding stream socket");
}
listen(sock, SOCKET_MAX_CONNECTIONS);
while(true) {
msgsock = accept(sock, 0, 0);
if (msgsock == -1){
throw runtime_error("accept socket");
} else {
bzero(buf, sizeof(buf));
if((rval = read(msgsock, buf, SOCKET_BUFFER_SIZE)) < 0)
throw runtime_error("reading stream message");
else if (rval == 0){
//do nothing, client closed socket
break;
} else {
string msg = requestHandler(string( reinterpret_cast< char const* >(buf), rval ));
if(write(msgsock, msg.c_str(), msg.length()) < 0)
throw runtime_error("sending stream message");
}
close(msgsock);
}
}
close(sock);
unlink(SOCKET_FILE_PATH);
}
what I don't understand is, how does the read function knows when the other part is done sending the message?
For a stream-type socket, such as you're using, it doesn't. For a datagram-type socket, communication is broken into distinct chunks, but if a message spans multiple datagrams then the answer is again "it doesn't". This is indeed one of the key things to understand about the read() and write() (and send() and recv()) functions in general, and about sockets more specifically.
For the rest of this answer I'll focus on stream oriented sockets, since that's what you're using. I'll also suppose that the socket is not in non-blocking mode. If you intend for your data transmitted over such a socket to be broken into distinct messages, then it is up to you to implement an application-level protocol by which the other end can recognize message boundaries.
I tried to read the read documentation and, on my understanding read will return once it reaches EOF or the SOCKET_BUFFER_SIZE, is that correct?
Not exactly. read() will return if it reaches the end of the file, which happens when the peer closes its socket (or at least shuts down the write side of it) so that it is certain that no more data will be sent. read() will also return in the event of any of a variety of error conditions. And read() may return under other unspecified circumstances, provided that it has transferred at least one byte. In practice, this last case is generally invoked if the socket buffer fills, but it may also be invoked under other circumstances, such as when the buffer empties.
So I'm guessing that when I give my string to the write function, it adds an EOF at the end of my content so the read function knows when to stop.
No, it does no such thing. On success, the write() function sends some or all of the bytes you asked it to send, and nothing else. Note that it is not guaranteed even to send all the requested bytes; its return value tells you how many of them it actually did send. If that's fewer than "all", then ordinarily you should simply perform another write() to transfer the rest. You may need to do this multiple times to send the whole message. In any event, only the bytes you specify are sent.
I'm asking this question because, I did not add any code that checks whether the other part finished sending the message, however, I'm receiving big messages (thousands of bytes) without any problem, why is that happening, why am I not getting only parts of the message?
More or less because you're getting lucky, but the fact that you're using UNIX-domain sockets (as opposed to network sockets) helps. Your data are transferred very efficiently from sending process to receiving process through the kernel, and it is not particularly surprising that large writes() are received by single read()s. You cannot safely rely on that always to happen, however.
In got the following problem:
I made a server which is able to handle multiple connection by using select(). But select returns a client(index of FD_SET) also if the socket just got an error like "client disconnect" or whatever.
Is it possible to check a socket without calling recv(). Because to receive I need to get a buffer out of my "BufferPool"
Sample code:
int ret = recv(client, buffer_pool->get(), BUFFER_SIZE, 0);
if(ret == -1) ... // something went wrong
Well then I have to release the buffer again, and it was pretty much a waste of one buffer in my pool. (for a short time)
So isn't it possible to check the socket without calling recv()
I am not sure about the Windows, but using getsockopt() works like a charm on POSIX-compliant systems. Though before you use it - make sure that getting your buffer from the pool is more expensive than making an extra system call. Here is a code snippet:
int my_get_socket_error(int fd)
{
int err_code;
socklen_t len = sizeof(err_code);
if (getsockopt(fd, SOL_SOCKET, SO_ERROR, &err_code, &len) != 0)
err_code = errno;
else
errno = err_code;
return err_code;
}
UPDATE:
According to this document, it seems like Windows supports it too.
No, there is no way to avoid the recv() call. If select() reports that the socket is readable, then you have to read from the socket to determine its new state. If the client disconnected gracefully, recv() will return 0, not -1. If you do not want to waste a pooled buffer, then you will have to read into a temporary local buffer first, and then if recv() returns any data, you can retrieve a pooled buffer and copy the read data into it.
Calling recv and similar function does not work directly with networking devices or something similar.
When you send or receive data, all you do is questioning OS for available data, or to put data in queue for sending. Then OS will do the other job when your code is already went further.
That is why you receive errors after next call of socket function that will "contact" OS networking layers.
It is normal to get errors on that point, and you have to deal with them.
But to prevent blocking sockets and wasting buffers, check out online techniques of implementing or ready libraries that gives you asynchronous way of working with sockets, that way you don't need to define anything before socket will trigger receive callback function where you have to do actual receiving.
As well, it is not good technique to receive big amount of data in one go, because you will face problems with merged or broken apart data through TCP layer, because it is stream based layer. It is recommended to have header in you packets (few bytes) and receive them, that way you don't need pull for header, but only after header you want to read rest of message based on length provided in header. This is just possible example.
After some minutes of work and your help I just receive 1byte before receiving the full amount:
SOCKET client = ...;
char temp = 0x00;
int len = recv(client, &temp, 1, 0);
if(len == 0)
{
// .. client error handling
return;
}
char* buffer = m_memory_pool->Get();
len = recv(client, buffer + 1, m_memory_pool->buffer_size() - 1, 0);
buffer[0] = temp;
// data handling
I tried also to set a timeout for recv() but seems that under Windows it does not work, this is my code:
...
long timeout_ms = 10;
struct timeval interval = {timeout_ms / 1000, (timeout_ms % 1000) * 1000};
if (interval.tv_sec < 0 || (interval.tv_sec == 0 && interval.tv_usec <= 0))
{
interval.tv_sec = 0;
interval.tv_usec = 10000;
}
setsockopt(s_sktIx, SOL_SOCKET, SO_RCVTIMEO, (char *)&interval, sizeof(struct timeval));
...
I am working on a small networking project using Winsock2. I am using a TCP connection and actually am working with IRC as an example since IRC is fairly simple. What I am doing is connecting to the server and sending an initial buffer so the server recognizes a connection. This works fine.
What concerns me is that I cannot write to the socket again. It seems my program hangs if I do not use shutdown() (on SD_SEND) after I send the initial buffer.
So the next data (based on RFC 1459) I want to send is the USER and NICK information, however, I feel like using shutdown() is what is causing my current issue. Is there a way to reinitialize the write socket?
Thanks!
ADDED CODE
Note that these are located within a class so it still may be slightly obscured. I am writing it into a simpler example using the elements I have. Everything is properly defined, so if I forget to define things, I apologize, but many of my recurring variables are defined for the scope of the class.
int main(int argc,char *argv[])
{
int iResult;
SOCKET Connection;
iResult = WSAStartup(MAKEWORD(2,2), &wsaData);
if(iResult != 0)
throw "Startup failed!";
// Prep stuff
ZeroMemory(&hints,sizeof(hints)); // This struct is defined addrinfo
hints.ai_family = AF_UNSPEC;
hints.ai_socktype = SOCK_STREAM;
hints.ai_protocol = IPPROTO_TCP;
// Now resolve server addr
iResult = getaddrinfo(argv[1],argv[2],&hints,&result);
if(iResult != 0)
throw "getaddrinfo() failed!";
// Now try to connect
for(ptr=result;ptr != NULL;ptr = ptr->ai_next)
{
Connection = socket(ptr->ai_family, ptr->ai_socktype, ptr->ai_protocol); // defined in that "hints" struct. argument number 2
if(Connection == INVALID_SOCKET)
{
freeaddrinfo(result);
WSACleanup();
throw "Error at socket();";
}
// Connect to server
iResult = connect(Connection, ptr->ai_addr, (int)ptr->ai_addrlen);
if(iResult != 0)
{
closesocket(Connection);
Connection = INVALID_SOCKET;
continue;
}
break;
}
freeaddrinfo(result);
// Send initial buffer so server know you're there :)
iResult = send(Connection, "", 1, 0);
if(iResult == SOCKET_ERROR)
{
close();
throw "Could not send initial buffer!";
}
// Close this connection for the inital buffer
iResult = shutdown(Connection, SD_SEND);
if(iResult == SOCKET_ERROR)
{
close();
throw "Could not close initial buffer socket!";
}
bool connected = true;
// This is taken from my read function within the class
// BEGIN READ FUNCTION
iResult = 0; // Reset
std::string data = ""; // Capture the output and send it all at once!
// This only works if we're connected sweet cakes <3
if(connected)
{
do
{
iResult = recv(socket, recvbuf, BUFLEN, 0);
if(iResult > 0)
{
// Working properly
// Save all data even if there is more than BUFLEN sent
continue;
}
else if(iResult == 0)
// Connection closed properly
break;
else
printf("ERROR!");
} while(iResult > 0);
}
data += recvbuf;
ZeroMemory(&recvbuf,sizeof(recvbuf));
// Function returns std::string but essentially this is what happens
printf("%s",data.c_str());
// END READ FUNCTION
// BEGIN WRITE FUNCTION
iResult = 0; // Reset
SOCKET socket = Connection; // Write function arg 1
char *data; // Write function arg 2
iResult = send(socket,data,(int)strlen(data),0);
if(iResult == SOCKET_ERROR)
{
close();
printf("Could not write data: %ld",WSAGetLastError());
return 1;
}
// Data sent, let's close the write socket
iResult = shutdown(socket, SD_SEND);
if(iResult != 0)
{
close();
printf("Could not close write socket!");
return 1;
}
//return iResult;
// END WRITE FUNCTION
// Now that will produce "Could not write data: 0" for any value of data
// So realistically I want to send the USER and NICK data, then read
// and probably process a PING string from the server and send my PONG response
return 0;
}
I hope that clarifies things!
EDIT
I think I have figured out what is going wrong. I made the corrections listed below to my code; thanks guys. However, it's my read loop which is messing with things. Even after it has all the information it seems that it is waiting for the connection to be closed before it sends the output. Any ideas? My output currently looks like this (the bytes written/total is something I added to make sure everything was going down the wire correctly)
Bytes Written: 41
Bytes Total: 41
Data: ERROR :Closing Link: raged123[127.0.0.1] 6667 (Ping timeout)
...
:irc.foonet.com NOTICE AUTH :*** Found your hostname (cached)
PING :2ED39CE5
[A bunch of funny characters]WinSock 2.0
So it appears to have timed out because the PING did not receive PONG in time, however, I cannot send the PONG without first processing the PING request which means I would need to be able to read the output before the connection is closed. Any ideas?
May I suggest a fun document on the subject? Chapter's 6 and 7 of Beej's Guide to Network Programming
It has several examples.
There shouldn't be any need to send an "initial buffer" like you've done. The server will receive notification when a client connects, it doesn't depend on the client actually sending anything. (And in particular, the IRC protocol says that the server will start sending you stuff as soon as you connect.)
The call to shutdown() is highly suspicious. Why did you expect to need to do this? Shutting down a socket is something you do when you're done with the connection, not when you're just starting. You should remove this completely.
I'm not sure what type recvbuf is, but it looks like you're using it incorrectly. Something that can be appended to a std::string probably can't also have ZeroMemory() called on it, without one or the other of those being wrong. You also aren't using iResult which is the actual number of bytes received from the server.
Your write function also contains a call to shutdown(), which you should remove.
According to man send(2)
On success, these calls return the
number of characters sent. On error,
-1 is returned, and errno is set appropriately.
What happens is probably that send does not send the full buffer at once, you must use a loop around it.
This might not be your actual problem however since you,re sending an empty string...
I'd highly recommend using Wireshark so you can check what goes down to the wire
data += recvbuf;
This can't work. There's no way string::operator+= to know how many bytes have been received. This function expects a C-style string, not an arbitrary chunk of bytes.
But you also have a very fundamental design problem. You're expecting your program to speak the IRC protocol, but it contains no implementation of that protocol whatsoever. For example, the IRC protocol specifies a particular way that messages are delimited, and you have no code whatsoever to parse those messages.
As a result, your transition from reading to writing occurs at essentially a random time determined by the vagaries of TCP timing and how the server chooses to segment its output. Since the server is permitted to segment its output however it pleases (the protocol is clear that the client cannot rely on segmentation to parse the protocol but instead must rely on the line-oriented nature), your program's behavior is unpredictable.