I need some help understanding a peculiar issue I'm having when using asio.
I've a client -server app, with a c++ client(using boost asio) sending 2 byte hearbeat (say, every second) to a server(written in java) (and receiving lots of data as well).
for a quite a few minutes the server correctly receives the 2 byte HeartBeat, but after that the server's 'read' complains abt a 0 byte read, and closes the connection (which I guess is correct for a blocking read). The client however always prints out that it's been transferring the correct amount.
I've experimented with almost all variants of the 'write' family of functions. are all of them implemented in terms of 'write_some' and does that mean that this behavior is expected?
I must be making some mistake in my usage, basically I'm looking for something within asio that guarantees a write ( at least a byte) . please help me figure out where I'm going wrong(and if any further info is reqd.)...
any advice, most appreciated!
thanks!
If it's sockets, you can't "guarantee a write"; what if the network is down, the cable yanked out, the switch is on fire, or the power is out worldwide and your computer happens to be the only one running on batteries?
That said, it sounds as if you have some kind of buffering/emptying issue perhaps, check over your read code to make sure it really consumes all data that appears.
A 0-byte read is not an error, look over that code again, check for any error status flags on the socket(s) and so on. A read can fail with a "AGAIN"-status, which really means you should try again.
strace the applications at both ends. It will show any error codes that are returned by read(), write() etc. Use strace -f if the application is multithreaded.
The advantage of this approach is that all applications - java, c++, python appear the same in an strace, so it's easy to spot bad behaviour.
In this case, it would probably show that the tcp connection ended (gracefully).
Related
I'm trying to create a method that determines access to the ntp server. I made a simple method, but if there is no connection, then it waits a long time for an answer - 5 seconds. I check 5 servers like this, for example - time.nist.gov. We have to wait a very long time.
Question: is there an easy way to check to avoid waiting so long, about 1-2 seconds?
bool is_connection(char* url)
{
// time.nist.gov
return gethostbyname(url) != NULL;
}
First, you should do all these checks in separated threads and join them to get the whole results in a single request.
Second, NTP uses UDP, so you can't check if the port (123 for NTP) is open or not, since UDP isn't a connected protocol - i.e. you don't have delivery results unless the server sends back another datagram to acknowledge your datagram. With TCP, you can "ping" a port to check if it's open, but not in UDP. You'll need to dive into RFC 1305 in order to be able to check that.
Resolving the name won't help you to check if it's a valid and working NTP server.
Anyway, your problem can be easily solved, but the solution is most likely dependent of your operating system (type, version, ...), your compiler (type, version, C++ standard used, ...), and the allowed C++ frameworks for your case (open bar, restricted, portable or not, ...).
I highly doubt that an EFFICIENT solution in pure portable C++ exists, in particular if you're stuck with old C++ standards. An efficient solution is more likely totally platform-dependent.
You should precise your working environment in order to get a more precise solution.
I have this 2-side(client and server) program written in c++, which is basically a "chat program".
So one side sends data to the other side which sends data to the other side .
I am using C-Sockets (TCP/SOCK_STREAM).
So my Problem is, that while the server is waiting for data ("recv()") it is unable to send data itself, as the recv queue is infinite, as far as my experience shows.
I have thought of two ways to solve this, but I am not sure if its worth the time. Because there maybe is a simple solution, which I am just too stupid too think of.
Make it a 6-sided connection, meaning I have a "physical Server", where my server is running on in four instances, twice for each client(sending and recieving). so if one server recieves data it stores the data in a file, for the sending server of the other client to read out and.. send it. Works same the other way around.
As this solution sounds rather shitty, I might not go with it.
Using Multithreading. Which is as I have heard pure cancer to use and you shouldn't use it, unless you have to.
So my question is, do I have to? or is there maybe a simple solution?
recv() on a blocking socket is used when you need to wait for incoming data and have nothing to do until data is arrived. If it's not your case you can use non-blocking socket, or select with timeout. select would be simpler, just check if anything arrived on your socket regularly (e.g. in a loop), and do what you want in between.
By the way, it's called "BSD socket", not "C socket", this can help you googling.
I'm really new to this whole socket and server development, I'm not yet familiar with how it all works.
I made a simple flash application that needs to communicate with a socket,
With that, I used a socket that supports AS3 and works on "Red Tamarin",
Well I'll get to the point:
I currently have a loop that always runs socket.receive()
It responds and even displays text that I send from my flash application.
My goal is to get a simple online flash game,
Probably use SQL / SQLite to save information and export it to players,
What I don't understand is how I can take it there..
What I thought I'll need to do is something like so:
On the server side:
Have a loop that runs as long as the server is alive, that loop should always check every connection it has with clients and wait for commands coming from them, such as log in, update player position, disconnect, request list of objects in given positions
Client side:
Send information to the server according to the action, like when a player moves, send the new position to the server in a similar way to this : "MovePlayer[name][x][y]"
Is my plan really how things should be?
And about the actual information being sent, I'm curious, will it be efficient to constantly send the server string data? (that's what I'm used to work with, not some weird bytes and stuff)
Thanks in advance!
You're on the right track. But I encourage you to first define a communication protocol. You can start by defining what a command looks like. For example:
COMMAND <space> PARAM1 <space> PARAM2 <line-break>
A few considerations on the protocol definition:
What if PARAM1 is a string and contains spaces? How can you tell the start and end of each parameter?
Your parameters could also contain a line-break.
If your client application is installed by your clients, they'll need to update it once in a while. To complicate even further, they may run an older version and expect it to work, even if you have changed your protocol. This imposes a need for protocol versioning. Keep that in mind if you require user interaction for updating the client part of your application.
These are the most fundamental considerations I can think for your scenario. There may be other important considerations, but most of them depend on how your game works. Feel free to amend my list if you think I forgot something OP should consider.
After defining what a command looks like, document all commands you believe your applications needs. Don't segregate definition of a command unless it becomes too complex or excessively long for some of your operations. Try to keep things simple.
Now back to your questions:
Is my plan really how things should be?
Yes. That's exactly how it should be.
And about the actual information being sent, I'm curious, will it be efficient to constantly send the server string data? (that's what I'm used to work with, not some weird bytes and stuff)
That depends on a number of factors:
Which protocol you're using (TCP, UDP, etc);
Number of concurrent clients;
Average time to process a command;
Do you broadcast updates to other players?
How you did implement your server application;
Physical contraints:
Hardware: CPU, memory, etc;
Network: bandwidth, latency, etc;
(source: it20.info)
look at this
https://code.google.com/p/spitfire-and-firedrop/
there you will see the basic of building a socket server with redtamarin
see in particular
https://code.google.com/p/spitfire-and-firedrop/source/browse/trunk/spitfire/src/spitfire/Server.as
the details is as follow, redtamarin basically use blocking sockets with select()
with a max hard coded FD_SETSIZE of 4096
see:
https://code.google.com/p/redtamarin/wiki/Socket#maxConcurrentConnection
so here what happen in your server loop
you basically have an array of sockets object
you loop every x milliseconds and for each socket
you ask if you can read it
if you can read on the socket, you then compare if this socket obj is the server
if it is the server that means you have a new connection
if not that means a client try to send you data and so you read this data
and then pass it to an "interpreter"
later in the same loop you check if the socket obj is still valid
and if you can write to it
if you can write and the socket object is not the server
then you can send data to the client
here the equivalent code in C for reference
http://martinbroadhurst.com/source/select-server.c.html
http://www.lowtek.com/sockets/select.html
for a very basic example look at socketpolicyd
https://code.google.com/p/spitfire-and-firedrop/wiki/socketpolicyd
https://code.google.com/p/spitfire-and-firedrop/source/browse/trunk/socketpolicyd/src/spitfire/SocketPolicyServer.as
and compare the implementation with Perl and PHP
http://www.adobe.com/devnet/flashplayer/articles/socket_policy_files.html
I have two applications running on my machine. One is supposed to hand in the work and other is supposed to do the work. How can I make sure that the first application/process is in wait state. I can verify via the resources its consuming, but that does not guarantee so. What tools should I use?
Your 2 applications shoud communicate. There are a lot of ways to do that:
Send messages through sockets. This way the 2 processes can run on different machines if you use normal network sockets instead of local ones.
If you are using C you can use semaphores with semget/semop/semctl. There should be interfaces for that in other languages.
Named pipes block until there is both a read and a write operation in progress. You can use that for synchronisation.
Signals are also good for this. In C it is called sendmsg/recvmsg.
DBUS can also be used and has bindings for variuos languages.
Update: If you can't modify the processing application then it is harder. You have to rely on some signs that indicate the progress. (I am assuming you processing application reads a file, does some processing then writes the result to an output file.) Do you know the final size the result should be? If so you need to check the size repeatedly (or whenever it changes).
If you don't know the size but you know how the processing works you may be able to use that. For example the processing is done when the output file is closed. You can use strace to see all the system calls including the close. You can replace the close() function with the LD_PRELOAD environment variable (on windows you have to replace dlls). This way you can sort of modify the processing program without actually recompiling or even having access to its source.
you can use named pipes - the first app will read from it but it will be blank and hence it will keep waiting (blocked). The second app will write into it when it wants the first one to continue.
Nothing can guarantee that your application is in waiting state. You have to pass it some work and get back a response. It might be transactions or not - application can confirm that it got the message to process before it starts to process it or after it was processed (successfully or not). If it does not wait, passing a piece of work should fail. Whether when trying to write to a TCP/IP socket or other means, or if timeout occurs. This depends on implementation, what kind of transport you are using and other requirements.
There is actually a way of figuring out if the process (thread) is in blocking state and waiting for data on a socket (or other source), but that means that client should be on the same computer and have access privileges required to do that, but that makes no sense other than debugging, which you can do using any debugger anyway.
Overall, the idea of making sure that application is waiting for data before trying to pass it that data smells bad. Not to mention the racing condition - what if you checked and it was OK, and when you actually tried to send the data, you found out that application is not waiting at that time (even if that is microseconds).
I wrote a simple server and client apps, where I can switch between TCP, DCCP and UDP protocols. The goal was to transfer a file from the one to the other and measure the traffic for each protocol, so I can compare them for different network setups (I know roughly what the result should be, but I need exact numbers/graphs). Anyway after starting both apps on different computers and starting tcpdump I only get in the tcpdump-log the first few MBs (~50MB) from my 4GB file. The apps are written in a standard C/C++ code, which could be found anywhere on the web.
What may be the problem or what could I be doing wrong here?
-- Edit
The command line I use is:
tcpdump -s 1500 -w mylog
tcpdump captures then packets only the first ~55 sec. That's the time the client needs to send the file to the socket. Afterwards it stops, even though the server continues receiving and writing the file to the hard drive.
-- Edit2
Source code:
client.cpp
server.cpp
common.hpp
common.cpp
-- Edit final
As many of you pointed out (and as I suspected) there were several misconceptions/bugs in the source code. After I cleaned it up (or almost rewrote it), it works as needed with tcpdump. I will accept the answer from #Laurent Parenteau but only for point 5. as it was the only relevant for the problem. If someone is interested in the correct code, here it is:
Source code edited
client.cpp
server.cpp
common.hpp
common.cpp
There are many things wrong in the code.
The file size / transfer size is hardcoded to 4294967295 bytes. So, if the file supplied isn't that many bytes, you'll have problems.
In the sender, you aren't checking if the file read is successful or not. So if the file is smaller than 4294967295 bytes, you won't know it and send junk data (or nothing at all) over the network.
When you use UDP and DDCP, the packets order isn't guarantee, so the data received may be out of order (ie. junk).
When you use UDP, there's no retransmission of lost packet, so some data may never be received.
In the receiver, you aren't check how many bytes you received, you always write MAX_LINE bytes to the file. So even if you receive 0 bytes, you'll still be writing to the file, which is wrong.
When you use UDP, since you're sending in a thigh loop, even if the write() call return the same amount of bytes sent that what you requested, a lot of data will probably be dropped by the network stack or the network interface, since there's no congestion control in place. So, you will need to put some congestion control in place yourself.
And this is just from a quick scan of the code, there is probably more problems in there...
My suggestion is :
Try the transfer with TCP, do a md5sum of the file you read/send, and a md5sum of the file you receive/save, and compare the 2 md5sum. Once you have this case working, you can move to testing (still using the md5sum comparison) with UDP and DCCP...
For the tcpdump command, you should change -s 1500 for -s 0, which means unlimited. With that tcpdump command, you can trust it that data not seen by it hasn't been sent/received. Another good thing to do is to compare the tcpdump output of the sender with the receiver. This way you'll know if some packet lost occurred between the two network stacks.
Do you have x term access? Switch to Wireshark instead and try with that - its free, open source, and probably more widely used than tcpdump today. (It was formerly known as Ethereal.)
Also, do try the following tcpdump options:
-xx print the link header and data of the packet as well (does -w write data?)
-C specify the max file size explicitly.
-U to write packet by packet to the file instead of flushing the buffer.
-p dont put the nic in promiscuous mode
-O dont use the packet matching optimizer as yours is a new app level protocol.
Are you using verbose output in tcpdump? This can make the buffers fill quickly so redirect stdout/err to a file when you run it.
Are these Gigabit ethernet card on both ends?
tcpdump is used as a diagnostic and forensics tool by 10s of thousands (at least) programmers and computer security professionals worldwide. When a tool like this seems to be mishandling a very common task the first thing to suspect is the code you wrote, and not the tool.
In this particular case your code has a wide variety of significant errors. In particular, with TCP, your server will continue to write data to the file regardless of whether or not the client is sending any.
This code has race conditions that will result in non-deterministic behavior in some situations, improperly treats '\0' as being a special value in network data, ignores error conditions, and ignores end-of-file conditions. And that's just a brief reading.
In this case I am nearly certain that tcpdump is functioning perfectly and telling you that your application does not do what you think it does.
"That's the time the client needs to
send the file to the socket.
Afterwards it stops, even though the
server continues receiving and writing
the file to the hard drive."
This sound really weird. The socket buffers are way too small to allow this to happen. I really think that your server code only seems to receive data, while the sender actually has already stopped sending data.
I know this might sound silly, but are you sure it is not a problem of flush() of the file? I.e. the data are still in memory and not yet written to disk (because they do not amount to a sufficient quantity).
Try sync or just wait a bit until you are certain that enough data have been transmitted.