• I have a program that uses libpcap, is there a way to print out debug statements at runtime?
• I get packets from libpcap after a long delay, is this a bug?
• Why is the pcap-int.h header file missing from libpcap?
• Why my VLAN/MPLS packet filter does not work?
• How to decode a protocol on a non-standard port?
• How come I can not compile on Solaris?
• Why is tcpdump's address space steadily growing?
• Why don't I see all the traffic I expect?
• Why don't I see TCP traffic other than traffic to and from my machine?
• Why do I only see ARP packets when I try to capture traffic?
• How do I put an interface into promiscuous mode?
• When is a packet time-stamped? How accurate are the time stamps?
• Why do my time stamps on Linux have only 100ms resolution?
• How to update the man pages?
• What should and what should not be in the change log?

A: In addition to the usual methods (using printf() or fprintf(stderr, ...) in your code or in libpcap source files), you can enable function instrumentation to help debugging or learning a program linked with libpcap.

The goal is to generate instrumentation calls for entry and exit to functions. Just after function entry and just before function exit, the profiling functions are called and print the function names with indentation and call level. If entering in a function, print also the calling function name with file name and line number. There may be a small shift in the line number.

Step-by-step usage example for Debian Linux and derived distributions:

1. Get the GNU binary utilities (BFD development files):

   $ sudo apt install binutils-dev
   

2. git clone (or git pull) the libpcap repository and checkout the required branch/tag. Configure and build with:

   $ [ -x autogen.sh ] && ./autogen.sh
   $ ./configure --quiet --enable-instrument-functions
   $ make -s clean
   $ make -s all
   

3. Create a project directory in the same parent directory as libpcap.

   |-- libpcap
   |-- my_project
   

4. cd into the project directory. Copy an existing program to test or use your program.

   $ cp -vai ../libpcap/testprogs/findalldevstest.c my_program.c
   

5. Build using the following Makefile:

   PROG = my_program
   
   all: $(PROG)
   
   $(PROG): ../libpcap/libpcap.a
   	gcc -O0 -ggdb -finstrument-functions \
   	-I../libpcap \
   	-o $(PROG) $(PROG).c ../libpcap/instrument-functions.c \
   	-lbfd -no-pie \
   	../libpcap/libpcap.a \
   	$$(../libpcap/pcap-config --additional-libs --static-pcap-only)
   
   clean:
   	@rm $(PROG)
   

6. Run the program. If the environment variable INSTRUMENT is

   • unset or set to an empty string, print nothing, like with no instrumentation;
   • set to all or a, print all the functions names;
   • set to global or g, print only the global functions names.

   This allows to run with tcpdump, tcpslice or any program linked with libpcap:

   $ INSTRUMENT=a ./my_program ...
   $ INSTRUMENT=g ./my_program ...
   $ INSTRUMENT= ./my_program ...
   

   or

   $ export INSTRUMENT=global
   $ ./my_program ...
   

   This also allows to run the statically compiled binary on another host after copying it. You should get something like:

   [>> main (0)
    [>> pcap_findalldevs (1) from main my_program.c:144]
     [>> pcap_platform_finddevs (2) from pcap_findalldevs pcap.c:730]
      [>> pcap_findalldevs_interfaces (3) from pcap_platform_finddevs pcap-linux.c:1769]
       [>> can_be_bound (4) from pcap_findalldevs_interfaces fad-getad.c:207]
       [<< can_be_bound (4)
       [>> get_sa_len (4) from pcap_findalldevs_interfaces fad-getad.c:223]
       [<< get_sa_len (4)
       [>> pcap_add_addr_to_if (4) from pcap_findalldevs_interfaces fad-getad.c:270]
        [>> pcap_find_or_add_if (5) from pcap_add_addr_to_if pcap.c:1094]
         [>> get_if_description (6) from pcap_find_or_add_if pcap.c:1059]
         [<< get_if_description (6)
         [>> pcap_find_or_add_dev (6) from pcap_find_or_add_if pcap.c:1058]
   [...]
   

Q: I have a program that uses libpcap on Linux, it works as expected with old versions of Linux kernel and/or old versions of libpcap; with modern Linux kernels and modern libpcap versions the program waits on libpcap for a much longer time (or even indefinitely). Is this a libpcap bug?

A: This is a consequence of certain developments in Linux kernel and libpcap; from a libpcap user's point of view the key concept involved is "immediate mode", even though the latter is not Linux-specific.

In immediate mode libpcap receives captured packets from the OS kernel one at a time, which results in a higher CPU overhead and a more likely packet loss; in buffered mode it receives multiple packets at once, but only after the buffer is full or its timeout has expired, which might take much longer than expected—see "packet buffer timeout" in pcap(3PCAP) for a more detailed explanation. For many use cases one particular mode works much better than the other, also in buffered mode different values of the timeout produce different results.

Linux kernel for a long time supported immediate mode only; since the introduction of TPACKET_V3 in version 3.2 it supports both immediate and buffered modes. libpcap in version 1.5.0 added support for the buffered mode on Linux and made it the default mode on Linux (subject to compile- and runtime conditions, as discussed here). This did not change libpcap API behaviour, as far as the man pages documented it, but in a number of cases it did change the default user experience on Linux, as was discussed in great detail in libpcap (335, 350, 354, 382, 472, 550, 572) and other bug reports.

The best approach would be not to rely on the default behaviour, but to study the use case and the documentation, and to have the program explicitly request the most appropriate mode using pcap_set_immediate_mode(3PCAP) and the most appropriate timeout value (beware that zero is a platform-dependent edge case) using pcap_set_timeout(3PCAP) or pcap_open_live(3PCAP). For example, tcpdump does that conditionally at runtime to achieve lower latency decoding live packets on one hand, and higher throughput writing packets to a file on the other.

Q: I am trying to compile a program that uses libpcap, this works fine when I use a complete source tree of libpcap. When I use the libpcap development package provided with the operating system, on some systems the compilation fails because of a missing header file (pcap-int.h or another header that it includes). What is the problem and how to solve it?

A: In libpcap the pcap-int.h file has always been an internal header, which means its contents may change at any time in any way. Programs should never include libpcap internal headers directly and libpcap packages should never install internal header files. However, some libpcap packages do install pcap-int.h and some programs do include it and, consequently, fail to compile with libpcap packages that [correctly] do not install the internal headers.

The right solution is to fix the program not to include pcap-int.h, as discussed in greater detail in libpcap (457, 560) and tcpslice (6) bug reports.

Q: My packet filter expression includes the vlan or mpls keyword(s); what it actually matches and what I expect it to match are somewhat (or completely) different things, is this a bug?

A: Most often this is not a bug, but a result of misunderstanding of how these keywords work. The matter is, to test the packet for one or another matching condition, filter primitives need to read data at specific offsets from the beginning of the packet. As pcap-filter(7) mentions it, some keywords automatically increment that offset for all subsequent keywords in the filter expression—this stands for every vlan, mpls or geneve keyword and for the first pppoes or lane keyword.

On one hand, this simplifies expression syntax required to skip the outer header(s) once and to test multiple conditions in the inner header(s), e.g.:

vlan and src net (192.0.2.0/24 or 2001:db8::/32) and port 53

On the other hand, and rather counterintuitively, the filter expression

vlan 100 or vlan 200

means "a packet with at least one 802.1Q tag and the outermost VLAN ID 100 or a packet with at least two 802.1Q tags and the second outermost VLAN ID 200". Unfortunately, the current syntax of those primitives does not support expressions such as vlan (100 or 200), which means that each VLAN/MPLS/Geneve layer can be tested at most once.

In some cases it may be possible to achieve the desired effect by rearranging the same primitives in the filter expression to place vlan or mpls before or after other primitives, see tcpdump (113, 115, 606, 900) and libpcap (408, 798, 916, 1070) bug reports for specific examples. Alternatively, it may be possible (although not trivial) to rewrite the expression using different primitives, as explained in this post.

A: This may be possible under certain conditions. First of all, the network protocol of interest must be one of the protocols supported by the -T flag of tcpdump. You can find the complete list in tcpdump(1) (most of these protocols are UDP-based).

If the protocol is supported, keep in mind that using -T type will usually override the protocol detection for many unrelated packets as well. For the best results you would need to apply a packet filter to leave only the correct packets in scope. For example, to decode RADIUS authentication packets on non-standard port 21812, you would need to run tcpdump in a way similar to the following:

tcpdump -T radius -nv udp port 21812

If this approach does not solve the problem, you can consider contributing code to extend tcpdump (see this discussion) or using a different packet analyzer. For example, this mailing list post explains how to use tshark to examine the SSL protocol exchange of an FTPS connection that uses a non-standard TCP port. In the specific case of SSL/TLS, it is also possible to use ssldump.

A: You need to install flex. AT&T lex has some parameter set too low. See the thread summary. See also libpcap generic and Solaris-related installation notes.

Q: A tcpdump command that's been running for a long time seems to be leaking memory; its address space size seems to be increasing steadily over time. Why is this happening, and how can I keep it from leaking memory?

A: Run it with -S. Otherwise, tcpdump keeps track of all the connections it has seen so it can generate relative sequence numbers rather than absolute sequence numbers. This looks like a leak, but is in fact just state accumulation.

Q: When I use tcpdump to capture packets, why do I see only packets to or from my machine, or why do I not see all the traffic I'm expecting to see from or to the machine I'm trying to monitor?

A: This might be because the interface on that you're capturing is plugged into a switch; on a switched network, unicast traffic between two ports will not necessarily appear on other ports—only broadcast and multicast traffic will be sent to all ports.

Note that even if your machine is plugged into a hub, the "hub" may be a switched hub, in which case you're still on a switched network.

Note also that on the Linksys Web site, they say that their auto-sensing hubs "broadcast the 10Mb packets to the port that operate at 10Mb only and broadcast the 100Mb packets to the ports that operate at 100Mb only", which would indicate that if you sniff on a 10Mb port, you will not see traffic coming sent to a 100Mb port, and vice versa. This problem has also been reported for Netgear dual-speed hubs, and may exist for other "auto-sensing" or "dual-speed" hubs.

Some switches have the ability to replicate all traffic on all ports to a single port so that you can plug your analyzer into that single port to sniff all traffic. You would have to check the documentation for the switch to see if this is possible and, if so, to see how to do this. See the switch reference page on the Wireshark Wiki for information on some switches. (Note that it's a Wiki, so you can update or fix that information, or add additional information on those switches or information on new switches, yourself.)

Note also that many firewall/NAT boxes have a switch built into them; this includes many of the "cable/DSL router" boxes. If you have a box of that sort, that has a switch with some number of Ethernet ports into which you plug machines on your network, and another Ethernet port used to connect to a cable or DSL modem, you can, at least, sniff traffic between the machines on your network and the Internet by plugging the Ethernet port on the router going to the modem, the Ethernet port on the modem, and the machine on which you're running tcpdump into a hub (make sure it's not a switching hub, and that, if it's a dual-speed hub, all three of those ports are running at the same speed.

If your machine is not plugged into a switched network or a dual-speed hub, or it is plugged into a switched network but the port is set up to have all traffic replicated to it, the problem might be that the network interface on that you're capturing doesn't support "promiscuous" mode, or because your OS can't put the interface into promiscuous mode. Normally, network interfaces supply to the host only:

• packets sent to one of that host's link-layer addresses;
• broadcast packets;
• multicast packets sent to a multicast address that the host has configured the interface to accept.

Most network interfaces can also be put in "promiscuous" mode, in which they supply to the host all network packets they see. Tcpdump will try to put the interface on that it's capturing into promiscuous mode unless the -p option was specified. However, some network interfaces don't support promiscuous mode, and some OSes might not allow interfaces to be put into promiscuous mode.

If the interface is not running in promiscuous mode, it won't see any traffic that isn't intended to be seen by your machine. It will see broadcast packets, and multicast packets sent to a multicast MAC address the interface is set up to receive.

You should ask the vendor of your network interface whether it supports promiscuous mode. If it does, you should ask whoever supplied the driver for the interface (the vendor, or the supplier of the OS you're running on your machine) whether it supports promiscuous mode with that network interface.

In the case of Token Ring interfaces, the drivers for some of them, on Windows, may require you to enable promiscuous mode in order to capture in promiscuous mode. Ask the vendor of the card how to do this, or see, for example, this information on promiscuous mode on some Madge Token Ring adapters (note that those cards can have promiscuous mode disabled permanently, in which case you can't enable it).

In the case of wireless LAN interfaces, it appears that, when those interfaces are promiscuously sniffing, they're running in a significantly different mode from the mode that they run in when they're just acting as network interfaces (to the extent that it would be a significant effor for those drivers to support for promiscuously sniffing and acting as regular network interfaces at the same time), so it may be that Windows drivers for those interfaces don't support promiscuous mode.

Q: Why can't I see any TCP packets other than packets to or from my machine, even though another sniffer on the network sees those packets?

A: You're probably not seeing any packets other than unicast packets to or from your machine, and broadcast and multicast packets; a switch will normally send to a port only unicast traffic sent to the MAC address for the interface on that port, and broadcast and multicast traffic—it won't send to that port unicast traffic sent to a MAC address for some other interface—and a network interface not in promiscuous mode will receive only unicast traffic sent to the MAC address for that interface, broadcast traffic, and multicast traffic sent to a multicast MAC address the interface is set up to receive.

TCP doesn't use broadcast or multicast, so you will only see your own TCP traffic, but UDP services may use broadcast or multicast so you'll see some UDP traffic—however, this is not a problem with TCP traffic, it's a problem with unicast traffic, as you also won't see all UDP traffic between other machines.

I.e., this is probably the same question as this earlier one; see the response to that question.

Q: I'm trying to capture traffic on my network; why am I only seeing ARP packets?

A: You're probably on a switched network, and running tcpdump on a machine that's not sending traffic to the switch and not being sent any traffic from other machines on the switch. ARP packets are often broadcast packets, which are sent to all switch ports.

I.e., this is probably the same question as this earlier one; see the response to that question.

Q: I'm trying to capture all the traffic on my network; how do I put my interface into promiscuous mode?

A: By not disabling promiscuous mode when running tcpdump. Note, however, that:

• the form of promiscuous mode that libpcap (the library that programs such as tcpdump use to do packet capture) turns on will not necessarily be shown if you run ifconfig on the interface on a UNIX system;
• some network interfaces might not support promiscuous mode, and some drivers might not allow promiscuous mode to be turned on—see this earlier question for more information on that;
• the fact that you're not seeing any traffic, or are only seeing broadcast traffic, or aren't seeing any non-broadcast traffic other than traffic to or from the machine running tcpdump, does not mean that promiscuous mode isn't on—see this earlier question for more information on that.

You're probably on a switched network, and running tcpdump on a machine that's not sending traffic to the switch and not being sent any traffic from other machines on the switch. ARP packets are often broadcast packets, which are sent to all switch ports.

I.e., this is probably the same question as this earlier one; see the response to that question.

A: Tcpdump gets time stamps from libpcap, and libpcap gets them from the OS kernel, so tcpdump—and any other program using libpcap, such as Wireshark or snoop—is at the mercy of the time stamping code in the OS for time stamps.

In most OSes on that tcpdump and libpcap run, the packet is time stamped as part of the process of the network interface's device driver, or the networking stack, handling it. This means that the packet is not time stamped at the instant that it arrives at the network interface; after the packet arrives at the network interface, there will be a delay until an interrupt is delivered or the network interface is polled (i.e., the network interface might not interrupt the host immediately—the driver may be set up to poll the interface if network traffic is heavy, to reduce the number of interrupts and process more packets per interrupt), and there will be a further delay between the point at which the interrupt starts being processed and the time stamp is generated.

On some OSes, such as GNU Hurd, Haiku and HP-UX, the OS kernel does not time stamp the packet at all; instead, it's time stamped by libpcap at the time it reads the packet from the OS kernel, which means that there will be an even greater delay between the time the packet arrives and the time that it's time-stamped.

Thus, the packet time stamp is not necessarily a very accurate indication of the time it arrived at the machine that captured the packet.

Q: I'm running tcpdump on Linux; why do my time stamps have only 100ms resolution, rather than 1µs resolution?

A: At least on x86-based machines, Linux can get high-resolution time stamps on newer processors with the Time Stamp Counter (TSC) register; for example, Intel x86 processors, starting with the Pentium Pro, and including all x86 processors since then, have had a TSC, and other vendors probably added the TSC at some point to their families of x86 processors.

The Linux kernel must be configured with the CONFIG_X86_TSC option enabled in order to use the TSC. Make sure this option is enabled in your kernel.

In addition, some Linux distributions may have bugs in their versions of the kernel that cause packets not to be given high-resolution time stamps even if the TSC is enabled. See, for example, bug 61111 for Red Hat Linux 7.2. If your distribution has a bug such as this, you may have to run a standard kernel from kernel.org in order to get high-resolution time stamps.

A: All man pages for libpcap, tcpdump and tcpslice come from the same git repository as the software they document. Most man pages are stored in their original format (e.g. pcap-config.1, pcap_strerror.3pcap, tcpslice.1). Some man pages depend on particulars handled by a build system. In such case there is a corresponding source .in file in the repository, which the build system converts to an output file in the working copy (with GNU Autoconf this can be done again later by running ./config.status), e.g:

• tcpdump.1.in → tcpdump.1
• pcap-filter.manmisc.in → pcap-filter.manmisc
• pcap_dump_open.3pcap.in → pcap_dump_open.3pcap

A few words about the man page dates. Less the comments, every man page usually starts with a command similar to the following:

.TH PCAP_INIT 3PCAP "11 April 2020"

Both the text and the HTML versions of the man page display this date, which informs the readers about substantial changes to the content (to make it easy, for example, to tell if a specific function's man page is different between two particular versions of libpcap). With this in mind, when updating a man page, please update the date if and only if the changes include any of the following:

• a C function's arguments or return value
• a program's command-line arguments, i/o data or exit status
• other documented behaviour or usage semantics
• backward compatibility

The man pages use a mixture of formatting languages. When making changes, please try to keep the style consistent. Below are a few examples of common use cases.

.\" Add a comment.

.\" Start a new paragraph.
.PP

.\" Start a new section.
.SH BACKWARD COMPATIBILITY

.\" Use a bold font for the rest of the line, including any
.\" punctuation marks.
.B \-\-list\-interfaces
.B \-\-help
.B 21h36m 26d1h54m.

.\" Idem, italics.
.I tcpdump
.I libpcap
.I snaplen

.\" Use a bold font for the first word and regular font for
.\" the second word, including any punctuation marks.
.BR date (1).
.BR "ifconfig \-a" );

.\" Refer to a special constant, e.g. input or output of a
.\" C function.
.B PCAP_ERROR_IFACE_NOT_UP
.BR NULL ,
.B \-1
.B DLT_EN10MB

.\" Refer to another man page for the first time in this page.
.BR pcap_dump (3PCAP)
.BR select (2)
.BR pcap-filter (@MAN_MISC_INFO@)

.\" The 2nd and subsequent times.
.BR pcap_dump ()
.BR select ()
.BR pcap-filter ()

.\" Prevent a long word (a man page name, an e-mail address
.\" or an URL), especially one that contains dashes, from
.\" hyphenation. This keeps the plain text output readable.
.BR \%pcap-tstamp (7);
.BR \%pcap-savefile (@MAN_FILE_FORMATS@)
.BR \%pcap_open_offline_with_tstamp_precision (3PCAP)
Example web site is \%https://example.org/
Example e-mail is \%nobody@example.org.

A: The change log purpose is to help the end users understand what a particular software release means for their particular use case. With this in mind, when preparing files for a commit, please consider whether to include a change log entry and what to say in the entry to document the change well. The following guidelines can be used as a starting point.

Any of the following likely has to have a change log entry:

• bug fixes
• new features
• API changes
• difference in observed or documented behaviour
• difference in the intended usage
• difference in backward compatibility
• substantial changes to the man page(s)
• difference in build dependencies (Autoconf, CMake, compilers, libraries)
• difference in the result of make install
• difference in runtime dependencies (OS kernel or configuration, libraries)
• difference in integration level of specific OSes

Any of the following likely does not require a change log entry:

• cosmetic improvements (comments, typos, formatting)
• changes to integration of external development tools (CI, code analysis)
• changes to build matrix scripts
• changes to repository-specific files such as .mailmap, .gitignore and .gitattributes
• changes that have been replaced by later changes within the same release
• changes that fix minor issues with earlier changes that have already been described for the same release
• changes that do not result in meaningful difference in the release tarball
• changes to the CREDITS file
• changes to the CHANGES file itself