12.2 Packet Capture and Protocol Analysis: Choose Your Observation Point, Then Interpret the Data

Packet capture can answer the question, "What packets did this observation point actually see?" But it cannot automatically reveal what happened across the entire end-to-end flow. Choosing the wrong interface, capturing packets at different positions before and after NAT, or overlooking network card offloading can all yield seemingly contradictory results, each of which may be factually accurate in its own context.

Learning Objectives

• Select packet capture points that align with fault assumptions;
• Distinguish between capture filters and Wireshark display filters;
• Extract evidence from TCP connection establishment, retransmissions, termination, and TLS handshake phases;
• Identify observation boundaries introduced by offload, namespaces, proxies, and encryption;
• Securely store and share pcap files.

First, Draw the Observation Points

A single request might traverse the following path:

client process
  -> client namespace / host
  -> NAT or proxy
  -> load balancer
  -> server host / container namespace
  -> server process

Capturing a SYN packet emitted by the client only confirms that the packet reached the client-side packet capture point; if no packet is seen on the server side, the issue could lie in the middle network layers, or it might be a misidentified interface or namespace. The strongest form of validation comes from simultaneous packet captures at both ends within the same time window, correlated using five-tuples, TCP sequence numbers, and timestamps. Match NAT address mappings first. A terminating proxy creates independent TCP connections with separate sequence spaces; correlate those using proxy logs, request IDs, and traces.

Before beginning, record the following:

• UTC time and clock synchronization status;
• Client and server host addresses and ports;
• Container, Pod, and network namespace boundaries, as well as proxy boundaries;
• Address transformations before and after NAT or load balancing;
• The trace ID or unique request header used to reproduce the request.

Using tcpdump for bounded packet capture

Only capture packets from systems and traffic you own or have explicit authorization to diagnose. These examples use Linux and GNU timeout; replace the interface and documentation address with your target. Bound the interface, host, port, duration, and file size:

sudo timeout --signal=INT 30s tcpdump -i eth0 -nn -s 0 \
  'host 203.0.113.10 and tcp port 443' \
  -c 500 -w incident-443.pcap

Parameter meanings:

• -i eth0: Explicitly specify the interface to monitor; on Linux, any is convenient for initial screening, but link-layer information and behavior may differ from the actual interface;
• -nn: Skip hostname and service name resolution to reduce additional traffic and ambiguity;
• -s 0: Use the default maximum snaplen, commonly 262144 bytes in current versions, not an unlimited packet length. Choose a smaller value when sufficient for diagnosis;
• -c 500: Stop after 500 packets. If fewer arrive, the outer timeout sends SIGINT after 30 seconds, allowing capture to close normally, and returns 124;
• -w: Save raw capture data rather than parsed text output.

For long-duration captures, rotate files and limit the total number of files to prevent disk exhaustion:

sudo timeout --signal=INT 300s tcpdump -i eth0 -nn -s 256 \
  'host 203.0.113.10 and tcp port 443' \
  -C 10 -W 5 -w incident.pcap

-C 10 -W 5 cycles through five files of about 10 million bytes each, overwriting older files. The size check happens before writing a packet, so a file can exceed the limit by one packet, plus header overhead. The outer command stops after 300 seconds. In contrast, -G 60 -W 5 rotates every minute and exits after five files; it is not a circular buffer and does not bound bytes per minute. Rehearse the selected mode on the target version without mixing the two rotation semantics.

Two Filters Are Not the Same Syntax

tcpdump's capture filter typically uses BPF syntax to decide which packets are captured and written to a file during acquisition:

host 203.0.113.10 and tcp port 443
tcp[tcpflags] & (tcp-syn|tcp-ack) != 0

Wireshark's display filter operates after packets have been captured, filtering what is displayed in the interface, and uses a different syntax:

ip.addr == 203.0.113.10 && tcp.port == 443
tcp.flags.syn == 1
tcp.analysis.retransmission
tls.handshake.type == 1

With current pcap-filter semantics, the tcp[tcpflags] expression above matches IPv4 TCP only. Do not treat it as a complete IPv6 filter. IPv6 extension headers and fragmentation can also affect transport-header matching; verify with actual samples.

Pasting a display filter into tcpdump or a BPF filter into Wireshark's display filter box will result in failure or produce unexpected behavior.

Connection Establishment: Read the Packets

Typical TCP connection establishment:

client -> server  SYN
server -> client  SYN, ACK
client -> server  ACK

Several evidence patterns:

Client Packet Capture	Server Packet Capture	More Accurate Interpretation
SYN retransmitted with no response	No SYN observed	Request never reached the server's observation point, or the server captured the packet at the wrong location
SYN retransmitted with no response	SYN observed, no SYN-ACK received	Server path, policy, resource, or kernel handling requires further investigation
RST received	Possible RST sent	Host or intermediate device actively rejected the connection
SYN-ACK received, but client continues to send SYN	Server repeatedly sends SYN-ACK	Client did not accept that response, a separate connection attempt is mixed in, or capture is incomplete
Final ACK sent, followed by another SYN-ACK received	Server still retransmits SYN-ACK	ACK was lost or not accepted by the server. An established client normally replies with another ACK; losing the final ACK alone does not restart its SYN exchange

Do not conclude with certainty that a packet loss occurred at a specific device based solely on a single client-side capture. Dual-end evidence narrows the scope to between the two observation points, but still does not identify which intermediate device is at fault.

Retransmission and Out-of-Order: Analyzer Insights Are Inferences

Wireshark's tcp.analysis.retransmission, fast_retransmission, and out_of_order are generated based on the current capture and analysis state; they are not kernel logs from the sender. Capture drops, snaplen settings, splitting traffic across capture streams, timestamp precision, and observation points can all influence the analyzer's conclusions.

When determining retransmissions, at least verify the following:

• The five-tuple and direction of the flow;
• TCP sequence number ranges and ACK values;
• SACK blocks;
• Whether the same payload reappears;
• Whether the capture mechanism dropped packets before saving them;
• Whether both endpoints observe the same segment.

When capturing from the sender side, TSO/GSO may cause the capture to show packets larger than the actual line MTU. On the receiver side, GRO/LRO may merge multiple segments into one. Additionally, checksums might be filled in by the network interface card after the packet has been captured, leading to a local capture showing "checksum incorrect" while the actual transmitted packet is valid. To accurately assess the frame structure in production, capture on the peer endpoint or at an intermediate TAP, or temporarily inspect offload settings in a controlled experiment. Avoid modifying NIC offload features in production environments without thorough evaluation.

TLS: What You Can See, What You Can't

Without a session key, application data in TLS 1.2 or 1.3 remains encrypted. Packet captures still reveal:

• IP addresses, ports, packet sizes, direction, and timing;
• Partial TCP or QUIC transmission behavior;
• In TLS ClientHello, unencrypted fields such as SNI, ALPN, and supported versions, unless protected by ECH;
• Server certificate visibility depends on TLS version and handshake phase; in TLS 1.3, most handshake messages are encrypted after the ServerHello.

Thus, "Follow TCP Stream" on HTTPS by default only yields TLS records and cannot reconstruct the underlying HTTP traffic.

In controlled testing environments, supported clients can export session secrets via SSLKEYLOGFILE, enabling Wireshark to decrypt the traffic. Key logs are equivalent to sensitive decryption material for the session and must be stored strictly under access control and promptly destroyed. Not all runtime environments or clients support this environment variable.

For QUIC/HTTP/3, standard TCP analysis is not applicable. Instead, analysis should prioritize combining client key logs, implementation-provided qlog files, connection IDs, server-side logs, and application-level traces.

NAT, Proxies, and Load Balancing

It's normal for IP addresses to change along the network path:

• Client-side SNAT (source network address translation) causes the server to see a public IP address as the source;
• DNAT (destination network address translation) or load balancers map a virtual address to a backend server's actual IP;
• Reverse proxies terminate the client connection and establish a new, independent upstream connection;
• Protocols like PROXY protocol or trusted forwarding headers can pass through the original client IP, but only if both ends are properly configured and validated.

Therefore, the statement "the server sees a source IP that isn't the user's IP" cannot be used directly to conclude that NAT configuration is wrong. Instead, draw out the endpoint of each connection segment and clearly identify which segment you're observing.

A Reusable Packet Capture Workflow

1. Document the hypothesis to be validated, such as "The client's SYN packet reached the load balancer but not the backend."
2. Select observation points that can distinguish between the two possible outcomes, and synchronize them to a common timestamp.
3. Reproduce the scenario using the minimal possible filter and a short capture window.
4. Preserve the original pcap file, the command used, tool version, and host location.
5. First verify that the packet exists, then examine timing, sequence numbers, and protocol fields.
6. Align the packet capture with connection tables, firewall counters, load balancer logs, and application traces.
7. Draw a narrowly scoped conclusion and clearly identify the intervals that remain unobserved.

Data Security

PCAP files may contain sensitive information such as cookies, authorization tokens, query parameters, internal IP addresses, DNS names, and unencrypted business data. Before sharing, ensure that:

• The capture scope and snaplen are restricted at the source;
• Data is stored under controlled conditions with minimal required permissions;
• Text-based replacement of binary PCAP data is not used directly for "de-identification";
• When necessary, use dedicated tools to generate sanitized copies and verify their integrity;
• Retention periods are clearly defined and temporary key logs are securely deleted.

Lesson Summary

Packet capture gives a sensor's view from one point in the network, not an omniscient view. First, choose the observation point, then interpret the protocol; first verify the raw packet, then consult the analyzer's hints. In the next lesson, these tools will be organized into three safe, repeatable local experiments.

Standards and Documentation Entry Points

• tcpdump and pcap-filter manuals;
• Wireshark User's Guide: capture filters, display filters, TCP analysis;
• RFC 9293 (TCP), RFC 8446 (TLS 1.3), RFC 9000 (QUIC).