I am working on a TUN-based VPN server whose goal is to analyze packets it receives before forwarding them to their destination. Currently I am receiving the IP packets from a TUN interface, and simply sending them off to their destination unmodified.
I understand that analyzing the content of UDP packets would be as simple as stripping the IP and UDP headers. However, to analyze the contents of TCP traffic, I would need to reconstruct the message from multiple IP packets. Is there an easy way to do this without re-implementing TCP? Are there any easily accessible C/C++ libraries meant for this task? I would prefer Linux system libraries and/or open-source, non-viral/non-copyleft libraries.
One thing I have already considered is making a copy of each IP packet, and changing the destination IP of the copy to localhost, so that a different part of my server may receive these TCP requests and responses fully reconstructed and without headers. However, I would not be able to associate destination IPs with traffic content, which is something that I desire.
It is likely functionality you need will be always tightly coupled with packet dissection. Good protocol dissectors are really needed to extract required information. So my suggestion is to use best open source tool available - wireshark.org
It provides "Follow TCP stream" functionality:
I doesn't look like you can easily extract part of Wireshark dissection logic, but at least there is a good example packet-tcp:
typedef struct _tcp_flow_t {
guint32 base_seq; /* base seq number (used by relative sequence numbers)
* or 0 if not yet known.
*/
tcp_unacked_t *segments;
guint32 fin; /* frame number of the final FIN */
guint32 lastack; /* last seen ack */
nstime_t lastacktime; /* Time of the last ack packet */
guint32 lastnondupack; /* frame number of last seen non dupack */
guint32 dupacknum; /* dupack number */
guint32 nextseq; /* highest seen nextseq */
guint32 maxseqtobeacked;/* highest seen continuous seq number (without hole in the stream) from the fwd party,
* this is the maximum seq number that can be acked by the rev party in normal case.
* If the rev party sends an ACK beyond this seq number it indicates TCP_A_ACK_LOST_PACKET contition */
guint32 nextseqframe; /* frame number for segment with highest
* sequence number
*/
Basically, there is separate conversation extraction logic, please notice find_conversation usage:
/* Attach process info to a flow */
/* XXX - We depend on the TCP dissector finding the conversation first */
void
add_tcp_process_info(guint32 frame_num, address *local_addr, address *remote_addr, guint16 local_port, guint16 remote_port, guint32 uid, guint32 pid, gchar *username, gchar *command) {
conversation_t *conv;
struct tcp_analysis *tcpd;
tcp_flow_t *flow = NULL;
conv = find_conversation(frame_num, local_addr, remote_addr, PT_TCP, local_port, remote_port, 0);
if (!conv) {
return;
}
The actual logic is well documented and available here:
/*
* Given two address/port pairs for a packet, search for a conversation
* containing packets between those address/port pairs. Returns NULL if
* not found.
*
* We try to find the most exact match that we can, and then proceed to
* try wildcard matches on the "addr_b" and/or "port_b" argument if a more
* exact match failed.
* ...
*/
conversation_t *
find_conversation(const guint32 frame_num, const address *addr_a, const address *addr_b, const port_type ptype,
const guint32 port_a, const guint32 port_b, const guint options)
{
conversation_t *conversation;
/*
* First try an exact match, if we have two addresses and ports.
*/
if (!(options & (NO_ADDR_B|NO_PORT_B))) {
So what I'm actually suggesting is to use EPAN library. It is possible to extract this library and use it independently. Please be careful with the license.
Maybe you might be interested in libipq - iptables userspace packet queuing library.
#include <linux/netfilter.h>
#include <libipq.h>
Netfilter provides a mechanism for passing packets out of the stack
for queueing to userspace, then receiving these packets back into the
kernel with a verdict specifying what to do with the packets (such as
ACCEPT or DROP). These packets may also be modified in userspace prior
to reinjection back into the kernel. For each supported protocol, a
kernel module called a queue handler may register with Netfilter to
perform the mechanics of passing packets to and from userspace.
The standard queue handler for IPv4 is ip_queue. It is provided as an
experimental module with 2.4 kernels, and uses a Netlink socket for
kernel/userspace communication.
Once ip_queue is loaded, IP packets may be selected with iptables and
queued for userspace processing via the QUEUE target
here is brief example how to decompose tcp/ip packet:
ipq_packet_msg_t *m = ipq_get_packet(buf);
struct iphdr *ip = (struct iphdr*) m->payload;
struct tcphdr *tcp = (struct tcphdr*) (m->payload + (4 * ip->ihl));
int port = htons(tcp->dest);
status = ipq_set_verdict(h, m->packet_id,
NF_ACCEPT, 0, NULL);
if (status < 0)
die(h);
quick intro
If this is not what you are looking for you might try to use wireshark EPAN library.
Related
I want to write an algorithm that calculates the MTU. I'm writing all of this with functions from the WinSock2 library.
From what I understood from other posts people recommend setting the don't fragment bit to true and then sending messages with various lengths until the maximal packet size is found, but what I don't understand is:
How to check for a dropped packet?
E.g.:
/// socket has the "don't fragment" bit set.
int res = sendto(socket, dataToSend, dataToSendLen, flags /* = NULL*/, remoteAddr, remoteAddrLen);
/// How to check whether this packet has been dropped due to fragmentation?
I've got a c++ program using libnetfilter_queue library, designed to work on a Linux system.
I'd need to filter only application layer packets (hence, packets including payload for the application layer of the host).
I know that it's not possible with iptables, without rebuilding the kernel.
Since I can't do that on the final host device, I'm working from my c++ program.
My aim is to directly accept non-application layer packets and to process layer-7 packets.
I tried using the nfq_get_payload function, returning -1 if an error (hence, I suppose, no payload) if found.
ret = nfq_get_payload(tb, &data);
if (ret < 0) { /* accept packet */ }
else { /* process packet */ }
I know that the nfq_get_payload function depends on the "adopted mode" (see nfq_set_mode function), but it is not working to me.
How can I discriminate between application layer packets and "lower-layers" ones?
Knowing that the ip_src byte location is in data + 12 (see also here), since the TCP layer size is 64 bytes, the payload should be found, if available, in position data + 12 + 64.
unsigned char* pkt_payload = (buffer + 12 + 64));
Nevertheless, if I try to print the pkt_payload variable, it is not compliant to the expected results.
How can I solve it?
I have a rather strange question. Lately, I have been tasked with developing software to simulate a large (hundreds of nodes and up) network. To make a long story short, we have a head-end server that communicates with each host through a predictable IP addressing scheme via Linux sockets using a mixture of broadcast and unicast. The head-end will issue a request to a given client and will (sometimes) receive data pertaining to the command executed. All data / commands are sent via UDP on a well-defined port.
Now, for testing purposes, we would like to use the original server binary in a virtual environment an still receive reasonable data. For example, we would like to issue a reset command to a particular node and receive a fake notification back. The broadcast bit is easy, as I simply have to listen in on the proper broadcast address and act accordingly. The unicast is what has me stuck.
The Question
Is it possible to receive UDP requests for a large number of discrete hosts via a single (or a reduced) number of Linux sockets? All hosts are on the same subnet and all IP addresses / hosts / network topology are known ahead of time.
Desired Output
Ultimately, we would like to have an app that runs on a host on the network and responds as if it were each of these discrete 'virtualized' hosts based on input datagrams.
Do note that I am not asking for someone to write me a program. I am just simply looking for some direction as to the 'vehicle' by which this can be accomplished.
Possible Solutions
RAW Sockets: This has promise as I can trap all inbound data via a
single socket and punt it off to a worker thread for processing and
response. Unfortunately, I only receive packets that are
destined for my host IP and none of the 'fake' IPs.
Abuse IP aliases on Linux, one for each host: This seems to be the most direct approach but it feels like duck hunting with a bazooka. It has the added benefit of appearing to 'be' the host for any other forms of communication, I just worry that creating 400+ aliases might be a bit much for our bastard-child of a Linux environment. As an added complication, the hosts do change based on configuration and can be in any manner of states (up, down, command processing, etc.).
The source code of the server is to be treated as immutable for the purpose of our testing. I fully expect this will be impossible with the constraints given, but someone may have an idea of how to accomplish this as, quite frankly, I have never done anything of this sort before.
Thank you in advance for any assistance.
Personally, I would use your second option - add all the IP addresses to the host, then bind to INADDR_ANY address. This would mean you could use just one socket.
An alternative is to set the IP_TRANSPARENT socket option on your socket, which will then allow your application to bind to non-local addresses (you would route the networks containing those addresses through the machine that your application is running on). This method does require one socket per address, though.
So, using a combination of both of caf's solutions, I was able to have my cake and eat it too. I was also heavily influenced by
Python/iptables: Capturing all UDP packets and their original destination
which is a Python example, but does show how I can 'cheat' the packets back to a single interface, negating the need for maintenance of many sockets. That question is well worth the read and contains a lot of good information. For compactness, though, I will restate part of it below.
Hopefully it can help someone else down the road.
Part 1 - Host Configuration
As stated in the above question, we can use a combination of iptables and ip routes to redirect the packets to loopback for processing. This was not stated in my original question, but it is acceptable for the 'simulator' to run on the head-end host itself and not be a discrete node on the network. To do this, we mark each packet via iptables and then route it to lo based on said mark.
iptables -A OUTPUT -t mangle -p udp --dport 27333 -j MARK --set-mark 1
ip rule add fwmark 1 lookup 100
ip route add local 0.0.0.0/0 dev lo table 100
In my case, I only need traffic to a certain port so my iptables rule has been adjusted accordingly from the original.
Part 2 - Software
As caf stated in his post, the real trick is to use IP_TRANSPARENT and a raw socket. Raw sockets are necessary in order to get the original source / destination IP addresses. One gotchya that took me a while was the use of IPPROTO_UDP in the call to socket(). Even though this is a raw socket, it will strip out the Ethernet header. A lot of code online shows the calculation of the IP header offset using something similar to the following:
struct iphdr* ipHeader = (struct iphdr *)(buf + sizeof(ethhdr));
Offsetting by ethhdr (which is stripped) will give you some rather entertaining garbage data. With that particular header removed, the necessary IP header is simply the first structure in the buffer.
The Test Code
Below you will find a proof-of-concept example. It is no way fully functional or complete. In particular, no checking in done on the incoming packets for malicious data (ex. format string exploits in the payload, pointer math problems, malformed / malicious packets, etc).
Note that the code binds to lo specifically. This does not mean that we will only get packets destined for one of our 'fake' hosts (other services use loobpack, too). Additional checking / filtering is required to get only the packets we want.
#include <arpa/inet.h>
#include <netinet/if_ether.h>
#include <netinet/in.h>
#include <netinet/ip.h>
#include <netinet/udp.h>
#include <sys/socket.h>
#include <stdio.h>
#include <string>
int main(int argc, char *argv[]) {
//Set up listening socket
struct sockaddr_in serverAddr;
struct iphdr* ipHeader;
struct udphdr* udpHeader;
int listenSock = 0;
char data[65536];
static int is_transparent = 1;
std::string device = "lo";
//Initialize listening socket
if ((listenSock = socket(AF_INET, SOCK_RAW, IPPROTO_UDP)) < 0) {
printf("Error creating socket\n");
return 1;
}
setsockopt(listenSock, SOL_IP, IP_TRANSPARENT, &is_transparent, sizeof(is_transparent));
setsockopt(listensock, SOL_SOCKET, SO_BINDTO_DEVICE, device.c_str(), device.size());
memset(&serverAddr, 0x00, sizeof(serverAddr));
memset(&data, 0x00, sizeof(data));
//Setup server address
serverAddr.sin_family = AF_INET;
serverAddr.sin_addr.s_addr = htonl(INADDR_ANY);
serverAddr.sin_port = htons(27333);
//Bind and listen
if (bind(listenSock, (struct sockaddr *) &serverAddr, sizeof(serverAddr)) < 0) {
printf("Error binding socket\n");
return 1;
}
while (1) {
//Accept connection
recv(listenSock, data, 65536, 0);
//Get IP header
ipHeader = (struct iphdr*)(data);
//Only grab UDP packets (17 is the magic number for UDP protocol)
if ((unsigned int)ipHeader->protocol == 17) {
//Get UDP header information
udpHeader = (struct udphdr*)(data + (ipHeader->ihl * 4));
//DEBUG
struct sockaddr_in tempDest;
struct sockaddr_in tempSource;
char* payload = (char*)(data + ipHeader->ihl * 4) + sizeof(struct udphdr));
memset(&tempSource, 0x00, sizeof(tempSource));
memset(&tempDest, 0x00, sizeof(tempDest));
tempSource.sin_addr.s_addr = ipHeader->saddr;
tempDest.sin_addr.s_addr = ipHeader->daddr;
printf("Datagram received\n");
printf("Source IP: %s\n", inet_ntoa(tempSource.sin_addr));
printf("Dest IP : %s\n", inet_ntoa(tempDest.sin_addr));
printf("Data : %s\n", payload);
printf("Port : %d\n\n", ntohs(udpHeader->dest));
}
}
}
Further Reading
Some very helpful links are below.
http://www.binarytides.com/packet-sniffer-code-in-c-using-linux-sockets-bsd-part-2/
http://bert-hubert.blogspot.com/2012/10/on-binding-datagram-udp-sockets-to-any.html
I'm writing code to send raw Ethernet frames between two Linux boxes. To test this I just want to get a simple client-send and server-receive.
I have the client correctly making packets (I can see them using a packet sniffer).
On the server side I initialize the socket like so:
fd = socket(PF_PACKET, SOCK_RAW, htons(MY_ETH_PROTOCOL));
where MY_ETH_PROTOCOL is a 2 byte constant I use as an ethertype so I don't hear extraneous network traffic.
when I bind this socket to my interface I must pass it a protocol again in the socket_addr struct:
socket_address.sll_protocol = htons(MY_ETH_PROTOCOL);
If I compile and run the code like this then it fails. My server does not see the packet. However if I change the code like so:
socket_address.sll_protocol = htons(ETH_P_ALL);
The server then can see the packet sent from the client (as well as many other packets) so I have to do some checking of the packet to see that it matches MY_ETH_PROTOCOL.
But I don't want my server to hear traffic that isn't being sent on the specified protocol so this isn't a solution. How do I do this?
I have resolved the issue.
According to http://linuxreviews.org/dictionary/Ethernet/ referring to the 2 byte field following the MAC addresses:
"values of that field between 64 and 1522 indicated the use of the new 802.3 Ethernet format with a length field, while values of 1536 decimal (0600 hexadecimal) and greater indicated the use of the original DIX or Ethernet II frame format with an EtherType sub-protocol identifier."
so I have to make sure my ethertype is >= 0x0600.
According to http://standards.ieee.org/regauth/ethertype/eth.txt use of 0x88b5 and 0x88b6 is "available for public use for prototype and vendor-specific protocol development." So this is what I am going to use as an ethertype. I shouldn't need any further filtering as the kernel should make sure to only pick up ethernet frames with the right destination MAC address and using that protocol.
I've worked around this problem in the past by using a packet filter.
Hand Waving (untested pseudocode)
struct bpf_insn my_filter[] = {
...
}
s = socket(PF_PACKET, SOCK_DGRAM, htons(protocol));
struct sock_fprog pf;
pf.filter = my_filter;
pf.len = my_filter_len;
setsockopt(s, SOL_SOCKET, SO_ATTACH_FILTER, &pf, sizeof(pf));
sll.sll_family = PF_PACKET;
sll.sll_protocol = htons(protocol);
sll.sll_ifindex = if_nametoindex("eth0");
bind(s, &sll, sizeof(sll));
Error checking and getting the packet filter right is left as an exercise for the reader...
Depending on your application, an alternative that may be easier to get working is libpcap.
How do I receive layer 2 packets in POSIXy C++? The packets only have src and dst MAC address, type/length, and custom formatted data. They're not TCP or UDP or IP or IGMP or ARP or whatever - they're a home-brewed format given unto me by the Hardware guys.
My socket(AF_PACKET, SOCK_RAW, IPPROTO_RAW) never returns from its recvfrom().
I can send fine, I just can't receive no matter what options I fling at the network stack.
(Platform is VxWorks, but I can translate POSIX or Linux or whatever...)
receive code (current incarnation):
int s;
if ((s = socket(AF_PACKET, SOCK_RAW, IPPROTO_RAW)) < 0) {
printf("socket create error.");
return -1;
}
struct ifreq _ifr;
strncpy(_ifr.ifr_name, "lltemac0", strlen("lltemac0"));
ioctl(s, IP_SIOCGIFINDEX, &_ifr);
struct sockaddr_ll _sockAttrib;
memset(&_sockAttrib, 0, sizeof(_sockAttrib));
_sockAttrib.sll_len = sizeof(_sockAttrib);
_sockAttrib.sll_family = AF_PACKET;
_sockAttrib.sll_protocol = IFT_ETHER;
_sockAttrib.sll_ifindex = _ifr.ifr_ifindex;
_sockAttrib.sll_hatype = 0xFFFF;
_sockAttrib.sll_pkttype = PACKET_HOST;
_sockAttrib.sll_halen = 6;
_sockAttrib.sll_addr[0] = 0x00;
_sockAttrib.sll_addr[1] = 0x02;
_sockAttrib.sll_addr[2] = 0x03;
_sockAttrib.sll_addr[3] = 0x12;
_sockAttrib.sll_addr[4] = 0x34;
_sockAttrib.sll_addr[5] = 0x56;
int _sockAttribLen = sizeof(_sockAttrib);
char packet[64];
memset(packet, 0, sizeof(packet));
if (recvfrom(s, (char *)packet, sizeof(packet), 0,
(struct sockaddr *)&_sockAttrib, &_sockAttribLen) < 0)
{
printf("packet receive error.");
}
// code never reaches here
I think the way to do this is to write your own Network Service that binds to the MUX layer in the VxWorks network stack. This is reasonably well documented in the VxWorks Network Programmer's Guide and something I have done a number of times.
A custom Network Service can be configured to see all layer 2 packets received on a network interface using the MUX_PROTO_SNARF service type, which is how Wind River's own WDB protocol works, or packets with a specific protocol type.
It is also possible to add a socket interface to your custom Network Service by writing a custom socket back-end that sits between the Network Service and the socket API. This is not required if you are happy to do the application processing in the Network Service.
You haven't said which version of VxWorks you are using but I think the above holds for VxWorks 5.5.x and 6.x
Have you tried setting the socket protocol to htons(ETH_P_ALL) as prescribed in packet(7)? What you're doing doesn't have much to do with IP (although IPPROTO_RAW may be some wildcard value, dunno)
I think this is going to be a bit tougher problem to solve than you expect. Given that it's not IP at all (or apparently any other protocol anything will recognize), I don't think you'll be able to solve your problem(s) entirely with user-level code. On Linux, I think you'd need to write your own device agnostic interface driver (probably using NAPI). Getting it to work under VxWorks will almost certainly be non-trivial (more like a complete rewrite from the ground-up than what most people would think of as a port).
Have you tried confirming via Wireshark that a packet has actually been sent from the other end?
Also, for debugging, ask your hardware guys if they have a debug pin (you can attach to a logic analyzer) that they can assert when it receives a packet. Just to make sure that the hardware is getting the packets fine.
First you need to specify the protocol as ETH_P_ALL so that your interface gets all the packet. Set your socket to be on promiscuous mode. Then bind your RAW socket to an interface before you perform a receive.