The IP identification field of the IP header (IP-ID) was introduced with RFC 791 in the early ’80s, to assist network-layer fragmentation and reassembly processes.
In this very first document, valid IP-ID choices were not fully specified: it only stated that each IP packet must have a unique IP-ID for the trio of source, destination and protocol within the maximum datagram lifetime.
Over the years, IP-ID has been the object of numerous changes (for example, to enforce more secure policies), and about a decade ago the different ways in which the IP-ID could be set by different operating systems were listed and detailed in RFC 4413 and RFC 5225. As a result, the IP-ID field can, nowadays, be set in different manners: for instance, the IP-ID can be set as a global counter (incremented by one at every new packet), or as a local counter (in which separate counters are kept for different destinations), or as the output of a pseudo-random number generator or finally as a (typically null-valued) constant.
In our recent work, we find evidence that local and constant implementations of the IP-ID are prevalent — this is in contrast with common knowledge [a, b, c, d, e, f], from which the global counter was expected, even in recent times, to be the most popular IP-ID implementation.
In our study [PDF 786 KB], we first propose a framework to robustly classify the different IP-ID behaviours with only a handful of IP packets. Our methodology consists of three main blocks: data collection, model construction and validation.
The data collection relies on an experimental testbed comprising one sender and two receivers, which collect the IP packets and, specifically, the information related to the IP-ID field. The sender sends a burst of packets, minimizing the impact of external traffic and purposely exploiting spoofing to precisely alternate addresses in the sequence.
In the model construction block, we exploit a decision tree classifier, which is trained and validated over datasets gathered from real measurements and additionally tested in the presence of controlled losses to assess its robustness. Training of the model required manual validation of thousands of sequences: during this phase, we also discovered some odd behaviour, not documented in any of the previous RFCs, and which may be attributed to different reasons such as bogus endings, load balancing, and an increment that is not standard.
In instances where odd behaviour was previously reported, our classifier is the first to automatically and correctly label such instances, making it easier to perform large-scale analysis over the Internet. Moreover, classification only requires a handful of packets, making the methodology extremely lightweight.
Given that our classifier is lightweight and robust to losses, we finally performed a census (in 2017) of the IPv4 address space, selecting responsive representatives for each /24 block. See the results of the census that we depict as a Hilbert curve in Figure 1 below.
Figure 1 — IP-ID census results, shown as a 12th order Hilbert curve, a fractal space-filling curve that allows the mapping of the one-dimensional IPv4 address space into a bi-dimensional image. The use of Hilbert curves to compactly represent Internet-wide characteristics was first popularized by the Xkcd comic and then used ever since.
Experimental results show that the majority of hosts adopt local IP-IDs (39%) or a constant counter (34%) of which:
- A fraction of global counters (18%) is significantly lower than expected
- A non-marginal number of hosts have an odd behaviour (7%)
- Random IP-IDs are only slightly more than an exception (2%)
This outcome provides a picture of Internet-wide adoption of the different IP-ID implementations.
Whereas especially over long timescales, changes in the popularity of protocol implementations and settings in the Internet are to be expected, the direction of such changes are, in part, surprising. Indeed, we gather that the 18% breakdown of the global implementation in 2017 is three times lower with respect to the 57% reported in 2013. While the quantitative reduction is in line with the statistics reported by recent work that leverages global IP-ID behaviour to detect censorship in the Internet, one could have expected the decrease in global implementation to be compensated by an increase of random IP-IDs, which is not the case.
Indeed, the decrease in global IP-ID makes it harder to apply inference techniques such as the numerous ones [g, h, i, j] developed over the years, for example, to count hosts behind NATs, detect load balancing, and infer censorship. At the same time, if local and constant IP-ID implementations leak less information with respect to a global IP-ID, they are still, nevertheless, predictable, whereas RFC 7739 observes that a (pseudo)-random IP-ID implementation “would be desirable from a security standpoint”.
All our datasets, including the testing with manual ground truth, as well as the results of our census, are publicly available: we hope that the former can assist scientists to build and test new techniques for IP-ID classification, whereas the latter provides practitioners with readily usable lists of the approximate half million hosts with global IP-ID implementations for their inference.
Finally, we invite interested readers to consult our paper [PDF 786 KB], which we recently presented at the Passive and Active Measurement Conference (PAM 2018), for more details on our method, dataset and results and leave any questions and comments you have on our work below.
Flavia Salutari is a PhD student at Télécom ParisTech working on Cisco’s Chair NewNet@Paris.
The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.