I have an application which communicates over the local area network. However, I want to instead make it communicate over the internet. To do this I propose making an intermediate program which will read the network traffic generated from the application on one computer and send it to the application on another computer.
This involves:
Reading the outgoing network traffic of the application
Sending a copy of this traffic over the internet to another computer
Giving this copy to the application on the other computer
Instead of this:
Application on computer A <-LAN-> Application on computer B
I want to achieve this:
Application on A <--> My Program on A <-INTERNET-> My program on B <--> Application on B
I can accomplish (2), but with (1) and (3) my problem is that I have very little experience with networking and I do not know where to start. I can program in python but would be willing to use c++ to accomplish this.
(Hamachi does not work for this application, I do not know why.)
In response to comments
I do not intend to manipulate any data unless it is necessary to make the connection work. I have no control over the application itself and it does not provide me with any methods to configure the connection with the exception of a port number.
TCP and UDP are both used on the port 6112. The IP addresses used are first 255.255.255.255 for a generic broadcast used to discover other applications on the LAN (with UDP), then a TCP connection is established.
The term you are missing in your original question is proxy. You specifically need a transparent forwarding proxy.
Here is a link to some source code in Python that will get you started with writing a proxy.
That said, if you search around you should be able to find a transparent forwarding proxy that you don't have to write yourself.
If you want to do this the most robust way, you can do it in hardware and setup a managed router/switch/firewall to route things to where ever you need without having to write anything.
Why re-engineer the wheel? Why not just use OpenVPN, n2n or vtun etc etc.
Related
As I understand it, it is not simply possible to implement a rest webservice on a device which is behind NAT. So i was searching for some solutions for this.
Is it possible to use long polling in order to implement the webservice? this way, the local device will make a call to the remote client (which is exactly what i want), the client has to keep the connection open (keep alive?) until the client want to call a webservice method. It can do so, because the connection is still open. After the call the client will immediately send another poll to the client ... etc..
Is it possible to implement it this way?
Another solutions on which i came across:
ReverseHTTP - I don't know very much about this, but it sounds like i can implement the webservice with this. right?
There are several other solutions, like TURN or STUN but they seem to be very complicated.
Do you have any suggestions?
I am using c++/linux on my network devices.
EDIT: Port Forwarding is not an option.
You've got a lot of different concepts here in this question. You can certainly implement a RESTful service behind a firewall/NAT... you just need to configure your firewall/NAT to forward connections to your service. There are issues of firewall/NAT devices timing out connections... here again, you can configure your device to not do that, or you can update your communication mechanism with some kind of "keep-alive". "long polling" is somewhat unrelated, and is used as a way of getting an "interactive like response" from a server... basically the server sits on a poll request from a client until it has something to respond with, or the request times out and the client makes another one. STUN and TURN are more voice/video communications-related technologies. I suggest starting with simply having your firewall/NAT device forward web-based requests to your web server.
You don't say what transfer protocol you are using, I'm assuming HTTP.
HTTP uses TCP/IP, so your device NAT needs to redirect the connection request to your server.
There's others ways, like if you have more than one internet IP address, so the requests could be directed to the server too, but thats more complicated than port forward so I think its not what you have.
So basically you need to configure the port forward. Take it like a PABX, calls from the exterior lines needs to know a ramal to reach a phone, thats a distant mean to think of it.
And as said, the suggestions you said, are not intended for that, is mainly for client connection, what for many NAT is not necessary, as the NAT is prepared for doing that.
If a program is running on a Linux machine, is there a way for that program to scan for ports that are allowed through the firewall? for example, if a programmer wants to make a chat system, but the program needs to know what ports aren't being refused access to incoming connections by a user's firewall, is there a way to check for this in your code? A program may not fail to bind a socket to a port even if the firewall is blocking that same port from incoming connections. Is there a way to check for open firewall ports?
Sidenote: This is purely for educational purposes and free of bad intentions, to be clear I am writing a chat system, and during testing, I was unable to connect desktop->laptop until I manually opened a port via allowing it through my firewall. This seemed a bit off to me, and unlike something that a programmer's code should require a user to do. Not to mention I don't want to leave the few people using this code at risk (if leaving a port permanently open does so). So It seems like I would be better suited finding a way to utilize ports that are already open to incoming TCP connections.
sidesidenote: all clients are running fedora
You have stumbled onto the second biggest issue governing the creation of new Internet applications nowadays. The first biggest of course is NAT, which is a strongly related issue (and hopefully going away eventually because of IPv6).
And there is no easy answer. One good answer is UPnP, but that's not an easy answer, and by no means universal. My network doesn't run it.
Another answer is to somehow tunnel everything you do over https (or http if you must). But that's a huge pain for something like a chat application.
I've been reading around on the www but just can't get the most important basics of P2P.
The diagram is like this:
[peer1]<-->[dsl-router1]<-->[central server]<-->[dsl-router2]<-->[peer2]
I'm developing a chat software on the central server. Chat messages being transfered thru' the central server well by now, however, I need to make the p2p file sharing feature because the bandwidth (the cable bandwith, not the transfer limit) of the server supposed for transfering chat messages only.
The problem is that, my software on central server knows the IPs and ports of router1 and router2, but not the peer1 and peer2 as these peers are behind the routers and don't have IP addresses.
How to actually transfer some data from peer1 to peer2 and vice versa without having this data passing thru' central server?
(and the worst case is that there is a wireless router between peer and dsl-router)
There are two basic ways of doing this. The new way is to use IGDP (opening a port via uPnP). This is described quite well here:
http://www.codeproject.com/Articles/13285/Using-UPnP-for-Programmatic-Port-Forwardings-and-N
If neither of the two nodes have a router supporting uPnP then another alternative is TCP hole punching, which is not perfect but works quite well in practice. This is described here:
http://www.brynosaurus.com/pub/net/p2pnat/
During some situations, "routers" supplied by the ISP may run on bridge mode, which directly exposes the peer computer on the internet (the computer gets a public internet address). If at least one side has this configuration (or in a similar situation that the peer client is not behind another device), then things should be rather straight forward: simply assign the central server's job to whoever that have this privilege.
In the other case where both peers only have a local address (e.g. 192.168.0.2) assigned to their computers, it would then be rather difficult to get through the routers; clients behind routers are for the most part unreachable from the outside unless they originated the request. Then, one solution to the problem is port forwarding. By doing port forwarding, either through explicitly written rules or UPnP, some ports on the peer computer is exposed to the public internet, as in the first situation where instead of only some ports the entire computer is exposed.
If you are without either of these, then there is no simple way to avoid sending through the central server. Though you could, potentially, find other peers who have the capability to transfer for others.
Let's say I have a server, and two clients connected to it. (via TCP, but it doesn't matter)
My goal is to allow a direct connection between those two clients. This is to allow direct voice contact between two players, for example, or any other client plugin they may have installed which don't need server interaction (like playing some kind of random game between the two). The server can be there to help setting up the connection.
From duskwuff's answer, I got several leads:
http://en.wikipedia.org/wiki/STUN which describes an algorithm to do that, and
http://en.wikipedia.org/wiki/UDP_hole_punching
From those, I got more leads:
http://www.h-online.com/security/features/How-Skype-Co-get-round-firewalls-747197.html
http://nutss.gforge.cis.cornell.edu/stunt.php -- A possible STUN implementation with TCP
With time, I could surely work out something for my program. For now I'm using C++ and TCP (Qt Sockets or Boost sockets), but if needed I don't mind doing UDP in C and wrapping it.
The bounty is there for any programmer having experience with those in C and C++ that may give tips to make this easier, by linking to example programs, updated libraries, or any other useful information. A documented, flexible & working C++ TCP implementation would be the best but I'll take what I get!
Punching TCP holes in NAT is sometimes/often possible (it depends of the NAT behavior). This is not a simple subject to learn, but read the corresponding chapter about NAT traversal from Practical JXTA II (available online on Scribd) to understand the nature of the issues to solve.
Then, read this. It comes from the guy who wrote that: http://nutss.gforge.cis.cornell.edu/stunt.php (one of the links in your question).
I am not a C/C++ specialist, but the issues to solve are not language specific. As long as you have access to TCP from your code base, that's enough. Keep in mind that implementing UDP traversal is easier than TCP.
Hope these tips help.
P.S.: I am not aware of a C/C++ implementation of the solution. The code mentioned in Cornell's link is NOT operational as confirmed by the author. I tried to resuscitate it myself, but he let me know it was completely tweaked for research purposes and far from production ready.
I'm not aware of any way to reliably punch through firewalls for TCP, but there's a similar method for UDP traffic that's pretty well documented:
http://en.wikipedia.org/wiki/STUN
http://en.wikipedia.org/wiki/UDP_hole_punching
A few links to projects that might be of interest or helpful:
http://sourceforge.net/projects/stun/
http://udt.sourceforge.net/
http://www.telehash.org/
You're looking for rendezvous server for NAT hole punching: the server that is publicly accessible (not behind NAT/firewall or they are properly configured) to help computers behind NAT/firewall to establish peer-to-peer connection.
UDP is more popular in NAT punching because provides much better results than TCP. Clear and informative description of UDP NAT hole punching can be found here.
If you need reliable communication, you can use reliable protocols over UDP:
SCTP (libraries) - standardized one, or
one of many custom protocols, e.g. RakNet (I used this library, it's quite mature and feature-rich and has NAT punching implementation), Enet or many others (Q8)
Ephemeral ports won't magically eliminate the need to relay through the server, because they are only valid during the life of the session opened through a well known service port. Basically ephemeral ports depend on a server session.
You will need to use the server to relay communications between both clients, that is act as a proxy server. One option would be to setup a SSH tunnel through a SSH proxy server, with the added benefit of security.
Still this doesn't guarantee that the firewall won't block the connection. That depends on the firewall type and configuration. Most residential routers that act as firewalls, by default block all incoming connections. This is normally fine because most of the time the computers behind the firewall act only as clients, which initiate the connections to the outside. And this setup varies, because some restrict initiating connections only to well known service ports like HTTP, HTTPS, FTP, SFTP, SSH, etc., and if your proxy server uses a non-well-known-service port then the connection will be blocked.
But firewalls can be setup to block outgoing traffic also, this is most common in corporate networks, which don't even allow direct connections to web servers and route everything through proxy servers, in order to control resource usage.
You can also research on the use of UPnP to open ports dynamically.
Is there an existing Linux/POSIX C/C++ library or example code for how to rebind a socket from one physical interface to another?
For example, I have ping transmitting on a socket that is associated with a physical connection A and I want to rebind that socket to physical connection B and have the ping packets continue being sent and received on connection B (after a short delay during switch-over).
I only need this for session-less protocols.
Thank you
Update:
I am trying to provide failover solution for use with PPP and Ethernet devices.
I have a basic script which can accomplish 90% of the functionality through use of iptables, NAT and routing table.
The problem is when the failover occurs, the pings continue being sent on the secondary connection, however, their source IP is from the old connection.
I've spoken with a couple of people who work on commercial routers and their suggestion is to rebind the socket to the secondary interface.
Update 2:
I apologise for not specifying this earlier. This solution will run on a router. I cannot change the ping program because it will run on the clients computer. I used ping as just an example, any connection that is not session-based should be capable of being switched over. I tested this feature on several commercial routers and it does work. Unfortunately, their software is proprietary, however, from various conversations and testing, I found that they are re-binding the sockets on failover.
As of your updated post, the problem is that changing the routing info is not going to change the source address of your ping, it will just force it out the second interface. This answer contains some relevant info.
You'll need to change the ping program. You can use a socket-per-interface approach and somehow inform the program when to fail over. Or you will have to close the socket and then bind to the second interface.
You can get the interface info required a couple of ways including calling ioctl() with the SIOCGIFCONF option and looping through the returned structures to get the interface address info.
I do't think that's quite a well-defined operation. the physical interfaces have different MAC addresses, so unless you have a routing layer mapping them (NAT or the like) then they're going to have different IP addresses.
Ports are identified by a triple of <IP addr, Port number, protocol> so if your IP address changes the port is going to change.
What are you really trying to do here?
I'm not at all sure what you're trying to accomplish, but I have a guess... Are you trying to do some kind of failover? If so, then there are indeed ways to accomplish that, but why not do it in the OS instead of the application?
On one end you can use CARP, and on the other you can use interface trunking/bonding (terminology varies) in failover mode.