I have a need to write a quick and dirty application to write some data over an ethernet connection to a remote machine.
The remote machine is sitting waiting for data, and I just want to blat some data at it to test the connection and bandwidth etc.
I'd like to be able to, say, send a known pattern of data (simple counting or repeating pattern) over the connection and be able to increase the bandwidth by x2, x10, x100 etc.
No need for handshaking, CRC, specific data format, framing etc. just plain old data.
Please... no third party libraries, just C++ (or C, or python)
If you can use netcat:
echo "BEEFCAKE" | nc remote.host port
I can recommend Beej's Guide to Network Programming. Helped me to understand all that network mumbo-jumbo.
However, if you need something really quick, why not use .NET? That has pretty nice classes for doing things like this. You could write your data in 10 lines.
P.S. Don't get thrown off by the fact that this is written for *nix. Winsock has all exactly the same functions.
When you say "IP:Port" then you must mean you need something higher layer than just an ethernet frame. You should read up on TCP/UDP/IP programming. I think the best resource online for this is Beej's Guide.. This is targeted toward berkeley or windows sockets.
Python sockets tutorial here.
Or just use teh googles and search for "socket programming in [language]".
This tutorial seems to cover what you want:
http://www.amk.ca/python/howto/sockets/
You can use Sockets.
Here is some basic tutorial.
Sounds like you want to be able to saturate the link for testing purposes without regard to whether the receiver can accept all the data. So, you will not want to use TCP since it is acknowledged, and flow controlled to avoid overrunning the receiver.
Probably easiest to go with UDP, although you could also consider a raw socket if you really want to write Ethernet frames directly (i.e. if you need to send data without IP/UDP headers, just raw Ethernet frames. In this case you'll need the destination MAC address to put in the Ethernet frame so it goes to the right place).
Regarding framing and CRC: Ethernet hardware will frame the packets and generate/check CRCs.
Edit: It might help if you could state what you are trying to achieve. Is it to test the speed of the Ethernet links/switches, or to see how fast the sending and receiving CPUs can exchange data? (It's likely that you can send data faster over Ethernet than the receiving CPU can handle it, although that does depend on the speed of the Ethernet and CPUs, as well as what OS the CPU is running, how busy it is, how it's network stack is tuned, etc..).
Related
I am writing a protocol to transfer gigabytes of data over a network using TCP, to try to teach myself a little bit about programming on protocols. I am unsure of how to design this transfer protocol, in order to transfer the data in the fastest and most efficient way.
I am using Qt on windows.
At the moment, my design of my application protocol (the part to transfer the data) is as follows:
First shoot the login details.
Write the first data packet (into the socket) of 4 kilobytes, and then wait for the server to confirm it has got the packet.
When the server confirms receiving the data packet (by writing int "1"), write the next 4 kilobytes.
When all data has been transferred, send the md5sum of the data transferred to the server.
If the server confirms again with an int 8, data transfer completes.
At the moment, I am not able to get speeds higher than 166KB/sec on the same computer when transferring over 127.0.0.1. I have been trying to read other protocol designs, but there is hardly any documentation on data transfer protocols that one can write for their application.
Is the protocol design that I've posted wrong or suffering from some serious issues?
Should the protocol wait for each packet to be confirmed by the server or should I write it continuously?
First, I would recommend spending some time on reading about TCP, and about Sliding Window Protocol.
I think there are 2 reasons why your implementation is so slow: first, you wait for acknowledgement of each packet - very slow, you should use sliding window.
Second, you use MD5 checksumming. There is nothing wrong with that, but TCP already implements some basic checksumming, and MD5 implementation you use can be very slow.
And finally, typical way to find out why something works very slow is to use profiling.
I was taking a look at using PF_RING for sending and receiving in my application.
If I plan to use PF_RING for maintaining a TCP connection, it looks like I'll need to manually "forge" the IP and TCP messages myself, as pfring_send sends raw packets. Does this mean I'll have to manually reimplement TCP on top of PF_RING?
I understand there is a clear advantage for receiving using PF_RING, has anyone tried sending data with PF_RING? Is there a clear advantage over normal send calls?
note: I am not using DNA (Direct NIC Access), I am just using the kernel partial bypass with NIC aware drivers.
To answer your first question, yes, you will have to manually build the TCP/IP messages from the ground up, MAC address and all. For an example take a look at pfsend.c from ntop.org.
ntop.org has also made a PF_RING user guide available that contains explanations.
As for sending data using PF_RING, it is absolutely possible, the idea is to bypass any and all notion of what is actually data on the wire and send as fast as possible, see wire speed traffic generation from ntop.org. The only advantage it has over normal sending calls using the kernel for TCP/IP is that you can send data 1. faster and 2. completely unformatted onto the wire. 2 can be handy for example when you want to play back a previously captured packet/multiple packets onto the network.
Unless you have a specific use case that requires you to get access to the raw underlying data without kernel intervention there is absolutely no good reason to use PF_RING in any way. Your best bet would be to use the standard socket()'s that are available, in most cases the performance you can achieve with that is more than adequate.
What specific use case did you have in mind?
I would like to write a program and run it on two machines, and send some data from one machine to another in an Ethernet frame.
Typically application data is at layer 7 of the OSI model, is there anything like a kernel restriction or API restriction, that would stop me from writing a program in which I can specify a destination MAC address and have some data sent to that MAC as the Ethernet payload? Then write a program to listen for incoming frames and grab the frames from a specified source MAC address, extracting the payload of data from the frame?
(So I don't want any other overhead like IP or TCP/UDP headers, I don't want to go higher than layer 2).
Can this be done in C++, or must all communication happen at the IP layer, and can this be done on Ubuntu? Extra love for pointing or providing examples! :D
My problem is obviously I'm new to network programming in c++ and as far as I know, if I want to communicate across a network I have to use a socket() call or similar, which works at an IP layer, so can I write a c++ program to work at OSI layer 2, are there APIs for this, does the Linux kernel even allow this?
As you already mentioned sockets, probably you would just like to use a raw socket. Maybe this page with C example code is of some help.
In case you are looking for an idea for a program only using Ethernet while still being useful:
Wake on LAN in it's original form is quite simple. Note however that most current implementations actually send UDP packets (exploiting that the receiver does not parse for packet headers etc. but just a string in the packet's payload).
Also the use of raw sockets is usually restricted to privileged users. You might need to either
call your program as root
or have it owned by root and setuid bit set
or set the capability for creating raw socket using setcap CAP_NET_RAW+ep /path/to/your/program-file
The last option gives more fine grained privileges (just raw sockets, not write access to your whole file system etc.) than the other two. It is still less widely known however, since it is "only" supported from kernel 2.6.24 on (which came with Ubuntu 8.04).
Yes, actually linux has a very nice feature that makes it easy to deal with layer 2 packets. You can use a TAP device, which allows your userspace program to read/write ethernet traffic through the kernel.
http://www.kernel.org/pub/linux/kernel/people/marcelo/linux-2.4/Documentation/networking/tuntap.txt
http://en.wikipedia.org/wiki/TUN/TAP
I'm working on an online turn based card game for the PC. It features a lobby which will automatically update the list of active games so I'll be sending many updates to many clients. I will have a game server for this. Lag is not too big a deal for me, if players have to wait an extra 1/4 second sometimes until their card is shown, it dosn't really concern me. What concerns me is reliability and stability. I want to be able to host many 4 player games, I also allow people to watch a specific game too.
I also will need to log them in, and remember their session if they disconnect so they can get back in the game if they get disconnected.
What I am debating is if I should go with Enet which is UDP based with reliability, or plain old TCP/IP.
I will eventually need to be able to send them additional content created such as extra decks which will be in the form of a zip file. But for that I'm sure there is a library that could help me get these from an HTTP source.
If anyone has experience with either or both of these, I'd appreciate your input.
Thnks
I'd stick with classic TCP, not some library that has hacked in correct ordering and packet delivery reliability into a protocol not designed for it.
Enet's claim is to have a few of the benefits of TCP, while delivering speed for "real-time" games. That phrase makes me think of FPS, which isn't your app at all.
I haven't worked with ENet, but I had a quick read of it's documentation, and it seems like it could be useful in certain circumstances.
First question: Do you need a reliable packet delivery mechanism, or is unreliable OK? ENet can do reliable, but at that point it's getting kind of similar to TCP, and unless you need it's multi-stream support, I'm not sure it's worth fooling with.
If unreliable is OK, then the next question is: do you need the packet fragmentation support that ENet has? If your packets are going to be small, then I'd say just use UDP directly.
I'm unclear on how ENet uses UDP sockets and how it manages connections, so I'm unclear whether you're better off with lots of open TCP sockets or lots of open ENet connections.
Lots of questions, I am sorry!
I am doing a voice-chat (VoIP) application and I was thinking of doing a custom implementation of the IP&UDP headers, along with small, extra information mainly seq number. Sounds alot like RTP yes, but I'm mainly just interested in the seq number or timestamp, and trying to implement my own whole RTP sounds like a nightmare with all the complexity involved and data im not likely to use.
Target OS for the application is windows xp and above. I have read http://msdn.microsoft.com/en-us/library/ms740548%28v=vs.85%29.aspx on the topic of Raw sockets in windows, and now I just want some confirmation.
I also have some general networking questions.
Here's the following questions;
1) According to MSDN, you cannot send custom IP packets with a source that is not on the network list. I understand it from a security PoV, but is there any way around this? My idea was to have for example two clients open UDP communication to a non-NAT protected server, and then have the clients spoof the source-header to make it look like packets come from the server instead of each other, thereby eliminating the need for a server as a relay of data to get through NAT, which would improve latency.
I have heard of winpcap but I don't want each client to have to install any 3rd party apps. Considering the number of DoS attacks surely there must be some way around this, like spoofing the network table the OS uses to check if source-header is legit? Will this trigger anti-virus systems?
I feel it would be really fun to actually toy with IP headers and above instead of just using predefined headers.
2) I've been having issues with free RTP libraries like JRTPLIB(which probably is very good anyway it just dosn't want to work for me) to make them work, more than I could almost tolerate, and am thinking of just writing my own interpretation ontop of UDP. Does application-level protcols like RTP simply build their header directly inside the UDP payload with the actual data afterwards? I suspect this considering the encapsulation process but just want to make sure.
If so, one does not need to create a RAW socket to implement application-level protocol, just an ordinary UDP socket and then your own payload interpretation above?
3) RTP does not give any performance boost compared to UDP since it adds more headers, all it does is making sure packets arrive in a sort-of correct manner based on timestamps and sequence numbers, right?
Is it -really- that usefull to use an RTP implementation for your basic VoIP project needs instead of adding basic sequencing yourself? I realise for video conferencing perhaps you reaally don't want frames to play out of order, but in audio conversations, would you really notice it?
4) If my solution in #1 is not applicable and I would have to use a server as a data relay between clients, would multicast be a good solution to reduce server loads? Is multicast supported enough in routing hardware?
5) It is related to question 1). Why do routers/firewalls allow things like UDP hole punching? For example, two clients first conenct to the server, then the server gives a client port / ip on to other clients, so the clients can talk to each other on those ports.
Why would firewalls allow data to be received from another IP than the one used in making the connection on that very port? Sounds like a big security hole that should easly be filtered? I understand that source IP spoofing would trick it, but this?
6) To set up a UDP session between two parties (the client which is behind NAT, server whic his non-NAT) does the client simply have to send a packet to the server and then the session is allowed through the firewall? Meaning the client can receive too from the server.
Based on article at wiki, http://en.wikipedia.org/wiki/UDP_hole_punching
7) Is SIP dependant on RTP? For some reason I got this impression but I cant find data to back it up. I may plan to add softphone functionality to my VoIP client in the future and want to make sure I have a good foundation (RTP if I really must, otherwise my own UDP interpretation)
Thanks in advance!
1, Raw sockets seems unnecessary for this application
2, Yes
3, RTP runs on top of UDP, of course it adds overhead. In many ways RTP (ignoring RTCP) is pretty much the bare minimum already and if you implemented a half-way decent alternative it would save you a few bytes at best and you wouldn't be able to use any of the many RTP test tools.
7, SIP is completely independent of RTP. SIP is used to Initiate Sessions. SDP is the protocol commonly transported by SIP, and it is SDP that negotiates and controls RTP video/voice voice.