I have a small project that I've been working on in C++, and due to the nature of what it does, I need to insert packets in to a live TCP stream. (The purpose is innocent enough, http://ee.forumify.com/viewtopic.php?id=3299 if you MUST know)
I'm creating a level editor for a game, and due to the nature of the handshakes, I can't simply establish a new connection with a high level library such as WinSock. Until now, it has relied on Winsock Packet Editor to do the dirty work, but if I were to let the application handle it all, it would make everyone happy.
So my question is this: Is there an API somewhere that will allow me to take control of a live TCP stream, and preferably one that keeps it valid after it finishes? And I would prefer to not have to inject any DLLs. Also, Detours is a no-no as I'm using GCC/Mingw.
I've toyed around with WinPCap and I have some working code (I can collect a packet, and from that generate a proper packet to send) but since it operates at such a low level, I cannot anticipate all of the potential protocols that the end user might use. Yes, chances are that they'll be using IPv4 over Ethernet, but what about those people who still use PPP, or some other obscure protocol? Also, the connection gets dropped by the client application after mine is done with it, as the latest ID values in the packets have changed and the client assumes that it has disconnected.
So, if anyone could provide a high-level TCP stream manipulator, I would be very happy. If not, I'll just continue tinkering with WinPCap and tell all the dial-up users to go get better internet.
Target platform: Microsoft Windows XP through Windows 7
Create a separate process to bind to a local port. When the initial tcp stream is created, proxy it through that process, which can then forward it on to the network. When you need to 'inject' into the stream you can have this proxy process do it. Just a thought.
you should look at the source code of ettercap http://ettercap.sourceforge.net/
or hunt, tcp hijacker http://packetstormsecurity.org/files/view/21967/hunt-1.5.tgz
Those 2 softs do what you're after.
I don't think there's any sensible API that will allow you to hijack a TCP stream. Such a thing would, inherently, be a security problem.
Can you insert your program as a proxy for the connection in question? That is, get the program that opens the connection to open it to your program, then have your program open the connection to the real target.
The idea is that if all the packets pass through your program anyway, then modifying the TCP stream becomes relatively trivial.
Related
I've written a socket client server program in c++. The program works perfectly and what I want to do now is to check periodically whether the connected clients are available. I know that it can be done using a while(true) loop in the server program. But it will use more cpu resources. Is there any other efficient way to check the availability of the clients? I've heard something called heartbeat. Does that help me? Is there any other way to do this?
By definition, is a client is "connected" it is "available" otherwise it will not be "connected".
If you need a persistent connection just use a transport protocol that provides for persistence (like TCP) and let TCP to do its job. Its own keep-alive and retransmission timers are already set up to satisfy network characteristics (and if they are not, this is a system problem, not something a specific application should manage alone).
Just send and receive data and manage the connection errors that eventually arise if one of the party goes away or becomes unreachable.
In any case:
Don't do dumb infinite loops sending/receiving empy messages just to test: there are billions on node in the internet: if anyone will behave as such, the entire internet will be fullfilled by "ping pong-s". Bendwidth has a cost. Much more than CPU and memory. Don't waste it.
Don't attempt to timeout yourself o recover a missing packet. There are hundredth of good reason you can even never imagine why a network can decide to reroute or discard a packet. Let TCP and IP to play their role consistently. All application trying to fix themselves netwotking issues do nothing more to adding mess to network managers.
In very short terms: if a socket is opened and is not in error, than the computer is connected. If it is in error, just close it and attempt to reopen it (and wait for the transport protocol timeouts. It ca be up to minutes, but don't try to escape from networking protocol rules: you are mot alone in the network, and it's not you who can make those rules).
Why not firing the main event in a timely manner instead of an endless loop ?
I heard from someone that modern MB architecture and memory designs can eliminate the need of allocating and reallocating mem-chunks by applying caching methods with some specialties to implement re-usability which helps also to avoid creating too many fragments both externally and internally. I don't mind checking this out again as I am listening to my online professor's online interview right now...:D. Good luck searching!
[EDIT] I read up on the other reply by ongard using ping method timely, which is very nice but please also consider that clients can block pings from other computers as well.
I'm looking for a way to add network emulation to a socket.
The basic solution would be some way to add bandwidth limitation to a connection.
The ideal solution for me would:
Support advanced network properties (latency, packet-loss)
Open-source
Have a similar API as standard sockets (or wraps around them)
Work on both Windows and Linux
Support IPv4 and IPv6
I saw a few options that work on the system level, or even as proxy (Dummynet, WANem, neten, etc.), but that won't work for me, because I want to be able to emulate each socket manually (for example, open one socket with modem emulation and one with 3G emulation. Basically I want to know how these tools do it.
EDIT: I need to embed this functionality in my own product, therefore using an extra box or a third-party tool that needs manual configuration is not acceptable. I want to write code that does the same thing as those tools do, and my question is how to do it.
Epilogue: In hindsight, my question was a bit misleading. Apparently, there is no way to do what I wanted directly on the socket. There are two options:
Add delays to send/receive operation (Based on #PaulCoccoli's answer):
by adding a delay before sending and receiving, you can get a very crude network simulation (constant delay for latency, delay sending, as to not send more than X bytes per second, for bandwidth).
Paul's answer and comment were great inspiration for me, so I award him the bounty.
Add the network simulation logic as a proxy (Based on #m0she and others answer):
Either send the request through the proxy, or use the proxy to intercept the requests, then add the desired simulation. However, it makes more sense to use a ready solution instead of writing your own proxy implementation - from what I've seen Dummynet is probably the best choice (this is what webpagetest.org does). Other options are in the answers below, I'll also add DonsProxy
This is the better way to do it, so I'm accepting this answer.
You can compile a proxy into your software that would do that.
It can be some implementation of full fledged socks proxy (like this) or probably better, something simpler that would only serve your purpose (and doesn't require prefixing your communication with the destination and other socks overhead).
That code could run as a separate process or a thread within your process.
Adding throttling to a proxy shouldn't be too hard. You can:
delay forwarding of data if it passes some bandwidth limit
add latency by adding timer before read/write operations on buffers.
If you're working with connection based protocol (like TCP), it would be senseless to drop packets, but with a datagram based protocol (UDP) it would also be simple to implement.
The connection creation API would be a bit different from normal posix/winsock (unless you do some macro or other magic), but everything else (send/recv/select/close/etc..) is the same.
If you're building this into your product, then you should implement a layer of abstraction over the sockets API so you can select your own implementation at run time. Alternatively, you can implement wrappers of each socket function and select whether to call your own version or the system's version.
As for adding latency, you could have your implementation of the sockets API spin off a thread. In that thread, have a priority queue ordered by time (i.e. this background thread does a very basic discrete event simulation). Each "packet" you send or receive could be enqueued along with a delivery time. Each delivery time should have some amount of delay added. I would use some kind of random number generator with a Gaussian distribution.
The background thread would also have to simulate the other side of the connection, though it sounds like you may have already implemented that part?
I know only Network Link Conditioner for Mac OS X Lion. You should be mac developer to download it, so i cannot put download link there. Only description from 9to5mac.com: http://9to5mac.com/2011/08/10/new-in-os-x-lion-network-link-conditioner-utility-lets-you-simulate-internet-and-bandwidth-conditions/
This answer might be a partial solution for you when using linux:
Simulate delayed and dropped packets on Linux. It refers to a kernel module called netem, which can simulate all kinds of network problems.
If you want to work with TCP connections, having "packet loss" could be problematic since a lot of error-handling (like recovering lost packages) is done in the kernel. Simulating this in a cross-platform way could be hard.
you usually add a network device to your network that throttles the bandwidth or latency, on a port by port basis, you can then achieve what you want just by connecting to the port allocated to the particular type of crappy network you want to test, with no code changes or modifications required.
The easiest ways to do this is just add iptables rules to a Linux server acting as a proxy.
If you want it to work without the separate device, try trickle that is a software package that throttles your network on your client PC. (or for Windows)
You may would like to check WANem http://wanem.sourceforge.net/ . WANEM is Open Source and licensed under the GNU General Public License.
WANem allows the application development team to setup a transparent application gateway which can be used to simulate WAN characteristics like Network delay, Packet loss, Packet corruption, Disconnections, Packet re-ordering, Jitter, etc.
I think you could use a tool like Network Simulator. It's free, for Windows.
The only thing to do is to setup your program to use the right ports (and the settings for the network, of course).
If you want a software only solution that you control, you will have to implement it yourself. I know of no such existing package.
While a wrapper layer over a socket may give you the ability to introduce delay, it won't be sufficient to introduce loss or out of order delivery. In order to simulate those activities, you actually need intercept the data in transit between the two TCP stacks.
The approach I would recommend is to use a tunneling device (say tunX). Routes should be set so the client believes the way to the server is through tunX. Additional code (perhaps running in a different thread) would promiscuously intercept traffic on tunX, and perform your augmented behavior, before forwarding packets over the true physical interface that will get the traffic to your server. The reverse would happen for packets arriving from the server on the physical interface. Those packets would be intercepted by the client code, behavior augmented, before forwarding through tunX.
However, since you are testing client software, I am unclear as to why you would want to embed this code in your released software, unless the software itself is a WAN simulating client.
I would like to write a program and run it on two machines, and send some data from one machine to another in an Ethernet frame.
Typically application data is at layer 7 of the OSI model, is there anything like a kernel restriction or API restriction, that would stop me from writing a program in which I can specify a destination MAC address and have some data sent to that MAC as the Ethernet payload? Then write a program to listen for incoming frames and grab the frames from a specified source MAC address, extracting the payload of data from the frame?
(So I don't want any other overhead like IP or TCP/UDP headers, I don't want to go higher than layer 2).
Can this be done in C++, or must all communication happen at the IP layer, and can this be done on Ubuntu? Extra love for pointing or providing examples! :D
My problem is obviously I'm new to network programming in c++ and as far as I know, if I want to communicate across a network I have to use a socket() call or similar, which works at an IP layer, so can I write a c++ program to work at OSI layer 2, are there APIs for this, does the Linux kernel even allow this?
As you already mentioned sockets, probably you would just like to use a raw socket. Maybe this page with C example code is of some help.
In case you are looking for an idea for a program only using Ethernet while still being useful:
Wake on LAN in it's original form is quite simple. Note however that most current implementations actually send UDP packets (exploiting that the receiver does not parse for packet headers etc. but just a string in the packet's payload).
Also the use of raw sockets is usually restricted to privileged users. You might need to either
call your program as root
or have it owned by root and setuid bit set
or set the capability for creating raw socket using setcap CAP_NET_RAW+ep /path/to/your/program-file
The last option gives more fine grained privileges (just raw sockets, not write access to your whole file system etc.) than the other two. It is still less widely known however, since it is "only" supported from kernel 2.6.24 on (which came with Ubuntu 8.04).
Yes, actually linux has a very nice feature that makes it easy to deal with layer 2 packets. You can use a TAP device, which allows your userspace program to read/write ethernet traffic through the kernel.
http://www.kernel.org/pub/linux/kernel/people/marcelo/linux-2.4/Documentation/networking/tuntap.txt
http://en.wikipedia.org/wiki/TUN/TAP
I'm working on an online turn based card game for the PC. It features a lobby which will automatically update the list of active games so I'll be sending many updates to many clients. I will have a game server for this. Lag is not too big a deal for me, if players have to wait an extra 1/4 second sometimes until their card is shown, it dosn't really concern me. What concerns me is reliability and stability. I want to be able to host many 4 player games, I also allow people to watch a specific game too.
I also will need to log them in, and remember their session if they disconnect so they can get back in the game if they get disconnected.
What I am debating is if I should go with Enet which is UDP based with reliability, or plain old TCP/IP.
I will eventually need to be able to send them additional content created such as extra decks which will be in the form of a zip file. But for that I'm sure there is a library that could help me get these from an HTTP source.
If anyone has experience with either or both of these, I'd appreciate your input.
Thnks
I'd stick with classic TCP, not some library that has hacked in correct ordering and packet delivery reliability into a protocol not designed for it.
Enet's claim is to have a few of the benefits of TCP, while delivering speed for "real-time" games. That phrase makes me think of FPS, which isn't your app at all.
I haven't worked with ENet, but I had a quick read of it's documentation, and it seems like it could be useful in certain circumstances.
First question: Do you need a reliable packet delivery mechanism, or is unreliable OK? ENet can do reliable, but at that point it's getting kind of similar to TCP, and unless you need it's multi-stream support, I'm not sure it's worth fooling with.
If unreliable is OK, then the next question is: do you need the packet fragmentation support that ENet has? If your packets are going to be small, then I'd say just use UDP directly.
I'm unclear on how ENet uses UDP sockets and how it manages connections, so I'm unclear whether you're better off with lots of open TCP sockets or lots of open ENet connections.
Lots of questions, I am sorry!
I am doing a voice-chat (VoIP) application and I was thinking of doing a custom implementation of the IP&UDP headers, along with small, extra information mainly seq number. Sounds alot like RTP yes, but I'm mainly just interested in the seq number or timestamp, and trying to implement my own whole RTP sounds like a nightmare with all the complexity involved and data im not likely to use.
Target OS for the application is windows xp and above. I have read http://msdn.microsoft.com/en-us/library/ms740548%28v=vs.85%29.aspx on the topic of Raw sockets in windows, and now I just want some confirmation.
I also have some general networking questions.
Here's the following questions;
1) According to MSDN, you cannot send custom IP packets with a source that is not on the network list. I understand it from a security PoV, but is there any way around this? My idea was to have for example two clients open UDP communication to a non-NAT protected server, and then have the clients spoof the source-header to make it look like packets come from the server instead of each other, thereby eliminating the need for a server as a relay of data to get through NAT, which would improve latency.
I have heard of winpcap but I don't want each client to have to install any 3rd party apps. Considering the number of DoS attacks surely there must be some way around this, like spoofing the network table the OS uses to check if source-header is legit? Will this trigger anti-virus systems?
I feel it would be really fun to actually toy with IP headers and above instead of just using predefined headers.
2) I've been having issues with free RTP libraries like JRTPLIB(which probably is very good anyway it just dosn't want to work for me) to make them work, more than I could almost tolerate, and am thinking of just writing my own interpretation ontop of UDP. Does application-level protcols like RTP simply build their header directly inside the UDP payload with the actual data afterwards? I suspect this considering the encapsulation process but just want to make sure.
If so, one does not need to create a RAW socket to implement application-level protocol, just an ordinary UDP socket and then your own payload interpretation above?
3) RTP does not give any performance boost compared to UDP since it adds more headers, all it does is making sure packets arrive in a sort-of correct manner based on timestamps and sequence numbers, right?
Is it -really- that usefull to use an RTP implementation for your basic VoIP project needs instead of adding basic sequencing yourself? I realise for video conferencing perhaps you reaally don't want frames to play out of order, but in audio conversations, would you really notice it?
4) If my solution in #1 is not applicable and I would have to use a server as a data relay between clients, would multicast be a good solution to reduce server loads? Is multicast supported enough in routing hardware?
5) It is related to question 1). Why do routers/firewalls allow things like UDP hole punching? For example, two clients first conenct to the server, then the server gives a client port / ip on to other clients, so the clients can talk to each other on those ports.
Why would firewalls allow data to be received from another IP than the one used in making the connection on that very port? Sounds like a big security hole that should easly be filtered? I understand that source IP spoofing would trick it, but this?
6) To set up a UDP session between two parties (the client which is behind NAT, server whic his non-NAT) does the client simply have to send a packet to the server and then the session is allowed through the firewall? Meaning the client can receive too from the server.
Based on article at wiki, http://en.wikipedia.org/wiki/UDP_hole_punching
7) Is SIP dependant on RTP? For some reason I got this impression but I cant find data to back it up. I may plan to add softphone functionality to my VoIP client in the future and want to make sure I have a good foundation (RTP if I really must, otherwise my own UDP interpretation)
Thanks in advance!
1, Raw sockets seems unnecessary for this application
2, Yes
3, RTP runs on top of UDP, of course it adds overhead. In many ways RTP (ignoring RTCP) is pretty much the bare minimum already and if you implemented a half-way decent alternative it would save you a few bytes at best and you wouldn't be able to use any of the many RTP test tools.
7, SIP is completely independent of RTP. SIP is used to Initiate Sessions. SDP is the protocol commonly transported by SIP, and it is SDP that negotiates and controls RTP video/voice voice.