matching against a list of subnets - c++

Question

There is a list of subnets in the form of net-addr/mask, such as
12.34.45.0/24 192.168.0.0/16 45.0.0.0/10 ...
Wonder what is the best way to tell if a given IP address is in any of the subnets.
Here is a little background on the matching:
For an IP address x, we convert it to an integer. For example, 11.12.13.14 is converted to 0x0b0c0d0e. For a mask m, we convert it to integer whose leading (32-m) bits are 1, the rest are 0.
To check if IP x is in subnet A/m,
we just need to check (x&m) == (A&m)
Curious what's the data structure or functions that makes matching against a range of subnets fast. Of course, we can go through the subnets in a loop but that's not efficient.

Answer 1

Make a tree where each level represents n bits of the IP address. Store subnets on each level so that the number of masks bits is between n * level and n * (level +1). So for example with n = 4, you have 16 children per node. So if you are testing against 11.12.13.14 (== 0x0b0c0d0e), you could walk the tree like this:
0 -> b -> 0 -> c -> 0 -> d -> 0 -> e
And on node you keep track of the subnets with the corresponding size. I mean: level 0 should have subnets /1 to /4 (inclusive), level 1 should have subnets /5 to /8, and so on up to /29 to /32. Note that /0 matches everything, so that would be useless to have in the data structure.
To search in the tree, group the IP in groups of n bits (in my example 4). Descend to the first level matching the first n bits and test all subnets on that level. If not found descend to the next level matching the next n bits.
This way you would have to test 32/n levels of each 2^n subnets maximum. For n=4, you would have to test 8 levels, each with at max 16 subnets. This is done in no time.
Clarification: A node is a subnet, for example (in hex, one digit is a nibble, which is 4 bits): 0a.5a.00.00/16. The parent of this node would be a subnet containing this subnet: for example: 0a.50.00.00/12. The edge towards a child node could be interpreted as: "contains", like in: "this (the parent) subnet contains the subnet represented by child node". For this tree to contain all the subnets you want, you will likely have to insert nodes, which represent a subnet that is not in your list. So mark these nodes as auxiliary nodes so you know that when searching this tree, you know that there are more specific subnets after under it, but the node itself is not part of the list of subnets you want to check against. You only should add these nodes that are directly in the list, and all parent nodes to make the nodes reachable in the tree structure.
Here is a struct on how I see it:
struct subnet_tree_node
{
uint_32 ip; // 32 bit IP address
subnet_tree_node *children;
uint_8 number_of_children;
uint_8 mask; // number of bits for this subnet
uint_8 valid; // wether this node is valid or auxiliary
}

Answer 2

So you've established performance is a problem.
Consider each netmask/addr pair as a pair of IP addresses: First valid, last-valid.
Let us assume last-valid is always odd (Not sure if that's true with a /32 network - but that's really, really strange).
Construct a sorted vector of these IP addresses. (Complain if the networks overlap or anything stupid.)
Search the vector for your target IP address with some sort of binary chop.
If the IP address is in the vector, it is a) wierd; b) in one of the subnets.
If the IP address is not in the vector and the value below is even - it is in a sub-net. If the value below is odd, it is not in a sub-net.

Answer 3

Do you have any evidence that performance is an issue? There are only 2^24 subnets (well, you can have /28 subnets, but they are usually internal to an organization, so even if the organization has a class A network, there's still only 2^24 of them).
Doing 16 million ands and comparisons is going to take no time.
Keep it simple (until you really have to do better).

Answer 4

Thanks for the discussions here, they got me inspired with this solution.
First, with loss of generality, we assume none of the subnet covers the other subnet (or we just remove the smaller one).
Each subnet is considered as an interval [subnet_min, subnet_max].
We just need to organize all the subnets into a binary tree, each node being a pair (subnet_min, subnet_max). When searching for an IP, it traverses the tree just like a regular binary search basing only on subnet_min, with the purpose of finding the node with subnet_min is largest among all the subnet_min's that are <= the given IP. Once we find this node, we check whether the node's subnet_max is greater >= the given IP. If so, the given IP is covered by the subnet, otherwise, we can say this IP is not covered by the subnet in this node, it's also not covered by any of the subnets neither.
The last point is guaranteed by the assumption that none of the two subnets contain each other.