2.2 Message Boundaries, Deadlines, and Backpressure

Call send twice on one connection and the peer application still receives a continuous byte stream. TCP reliably delivers ordered bytes, but it does not preserve application-message boundaries. Treating one recv as one message may survive a friendly test and then fail the moment scheduling or buffering changes.

This lesson implements a four-byte length prefix. A local socket example sends a stream of frames, and a controlled reader forces short reads. We then examine how timeouts, deadlines, pipelining, and backpressure affect connection state.

1. Four Common Types of Framing

Fixed Length

Each message is exactly N bytes. Easy to parse, but short messages waste space, and variable-length fields require additional rules. Suitable for hardware records or known-size blocks.

Delimiter

End with a newline or special sequence:

PING\r\n
SET key value\r\n

The implementation must handle separators spanning two recv, payload escaping, maximum line length, and incomplete lines. It cannot indefinitely wait and grow the buffer.

Length Prefix

[4-byte length][payload]

Binary and empty payloads can be transmitted. After reading the header, the upper limit must be verified before allocating and reading the body; otherwise, a 0xffffffff could trigger memory exhaustion.

Syntax and Protocol Boundaries

A JSON object has a syntactic end, but escaping, nesting, and top-level numbers need explicit parsing rules. HTTP defines its own message-length rules. The Protobuf wire format does not delimit a sequence of messages; it needs outer framing such as a length prefix. Structured data still requires a way to identify a complete message.

2. Correctly Reading a Fixed Number of Bytes

recv_exact must distinguish:

• Reading all N bytes;
• EOF before a new header, which cleanly ends the message stream;
• EOF inside a header or a declared body, which truncates the frame.

import socket
import struct
import threading

HEADER = struct.Struct("!I")
MAX_FRAME = 1024 * 1024

def recv_exact(connection, length, allow_clean_eof=False):
    if length < 0:
        raise ValueError("length must be nonnegative")
    data = bytearray()
    while len(data) < length:
        chunk = connection.recv(length - len(data))
        if chunk == b"":
            if allow_clean_eof and len(data) == 0:
                return None
            raise EOFError(
                f"stream ended after {len(data)} of {length} bytes"
            )
        data.extend(chunk)
    return bytes(data)

def encode_frame(payload):
    if len(payload) > MAX_FRAME:
        raise ValueError("frame too large")
    return HEADER.pack(len(payload)) + payload

def recv_frame(connection):
    header = recv_exact(
        connection,
        HEADER.size,
        allow_clean_eof=True,
    )
    if header is None:
        return None

    (length,) = HEADER.unpack(header)
    if length > MAX_FRAME:
        raise ValueError(f"declared frame too large: {length}")
    return recv_exact(connection, length)

def fragmented_writer(connection, frames):
    with connection:
        encoded = b"".join(encode_frame(frame) for frame in frames)
        for byte in encoded:
            connection.sendall(bytes([byte]))
        connection.shutdown(socket.SHUT_WR)

left, right = socket.socketpair()
left.settimeout(2)
right.settimeout(2)
expected = [b"alpha", b"", b"omega"]
writer = threading.Thread(
    target=fragmented_writer,
    args=(left, expected),
)
writer.start()

with right:
    actual = []
    while True:
        frame = recv_frame(right)
        if frame is None:
            break
        actual.append(frame)

writer.join(timeout=3)
assert not writer.is_alive()
assert actual == expected
print(actual)

This Python 3.8+ example uses socketpair for a local connection. On typical Unix systems its default is an AF_UNIX byte stream, so it does not exercise TCP. Implementations on other platforms can differ. One-byte sendall calls can still be combined by the receiver; they do not prove coverage of every short-read path. !I is a network-byte-order unsigned 32-bit integer counting bytes, not characters.

Run this adapter after the preceding block to cap each read deterministically. It reuses encode_frame and recv_frame, making short reads reproducible:

class ChunkReader:
    def __init__(self, data, max_chunk):
        if max_chunk <= 0:
            raise ValueError("max_chunk must be positive")
        self.data = data
        self.max_chunk = max_chunk

    def recv(self, length):
        count = min(length, self.max_chunk)
        part, self.data = self.data[:count], self.data[count:]
        return part

wire = b"".join(encode_frame(p) for p in [b"alpha", b"", b"omega"])
for chunk_size in [1, 2, 3, 4, len(wire)]:
    reader = ChunkReader(wire, chunk_size)
    frames = []
    while (frame := recv_frame(reader)) is not None:
        frames.append(frame)
    assert frames == [b"alpha", b"", b"omega"]
print("all chunk sizes passed")

None means EOF before a new header; b"" is a valid empty payload. Using if not frame would confuse an empty message with the end of the stream. This adapter tests parsing, while real sockets are still needed to test timeouts and shutdown.

3. The Header Might Also Be Split

Many incorrect implementations write:

length = struct.unpack("!I", connection.recv(4))[0]

recv(4) may return only 1 to 3 bytes; it does not wait for a complete length field on the application’s behalf. Both header and body need exact-read loops. Consistently full reads in a local test do not change the API contract.

Another mistake is treating EOF as "no data temporarily":

while True:
    if connection.recv(4096) == b"":
        continue

This will busy-wait on a closed connection. A blocking socket blocks or times out when no data is available temporarily; returning empty bytes indicates a graceful EOF.

4. Treat the Length as Untrusted Input

Check after reading length, before allocating:

• Does the declared length exceed the protocol limit?
• Are zero-length frames allowed?
• Could length arithmetic overflow the implementation’s integer type?
• Is decompressed output bounded separately?
• How many frames may be outstanding on one connection?
• Is the total buffer subject to global budget control?

“Up to 1 MiB” is just an example. The actual limit should be determined by business semantics, memory budget, and proxy chain constraints.

Compression protocols must also guard against high-compression-ratio payloads. Short wire length does not mean small decompression memory; compression input, decompression output, and CPU workload must all be limited.

5. Per-operation Timeouts and Total Deadlines

Setting a 5-second timeout for each recv does not mean the entire request is limited to 5 seconds. The remote end could send one byte every 4 seconds, making a 1 MiB message last for weeks.

Reuse HEADER and MAX_FRAME from section 2 and calculate the remaining budget with a monotonic clock. Pass the same absolute deadline through header and body; starting the body must not grant another 5 seconds:

import time

def recv_exact_before(connection, length, deadline, allow_clean_eof=False):
    if length < 0:
        raise ValueError("length must be nonnegative")
    data = bytearray()
    previous_timeout = connection.gettimeout()
    try:
        while len(data) < length:
            remaining = deadline - time.monotonic()
            if remaining <= 0:
                raise TimeoutError("message deadline exceeded")
            connection.settimeout(remaining)
            chunk = connection.recv(length - len(data))
            if chunk == b"":
                if allow_clean_eof and not data:
                    return None
                raise EOFError("truncated message")
            data.extend(chunk)
        return bytes(data)
    finally:
        connection.settimeout(previous_timeout)

def recv_frame_before(connection, deadline):
    header = recv_exact_before(connection, HEADER.size, deadline, True)
    if header is None:
        return None
    (length,) = HEADER.unpack(header)
    if length > MAX_FRAME:
        raise ValueError("frame too large")
    return recv_exact_before(connection, length, deadline)

# Use one budget for both header and body:
# frame = recv_frame_before(connection, time.monotonic() + 5)

The function restores the socket’s previous timeout and assumes this operation owns its I/O and timeout settings. Concurrent tasks must not overwrite each other’s timeouts. This deadline bounds these reads; parsing, queueing, and downstream calls need the same budget propagated through their own operations.

The protocol still needs decisions about failure:

• How do you handle a half-frame in a connection after a timeout?
• Should we close the connection, or does the protocol support secure resynchronization?
• Does the request deadline include waiting time and downstream calls?
• Can you retry when a response is partially sent?
• How do you signal cancellation to workers and I/O?

A deadline belongs to the connection state machine; setting a socket timeout alone does not implement it.

6. Backpressure: What Happens When Buffers Fill

A length prefix identifies message boundaries. A slow reader can still fill buffers along this simplified send path:

1. The local socket send buffer accepts bytes;
2. TCP sends data based on the receiving window and congestion window;
3. The peer kernel stores them in its receive buffer;
4. The peer application eventually calls recv.

If the remote end reads slowly, the buffer will fill up along the path. Blocking sendall will eventually block, nonblocking send will return would-block, and the pending send queue of the async writer will grow.

The wrong approach is for each producer to unconditionally append responses to the user-space list, turning network backpressure into a process OOM.

Define these limits and policies:

• Maximum queued bytes per connection;
• Global maximum queued bytes;
• When limits are exceeded, decide whether to pause upstream reads, reject new requests, or close slow connections;
• Which messages can be discarded or merged;
• A write deadline;
• Fair scheduling to prevent a large response from starving smaller ones.

Python asyncio’s StreamWriter.write attempts a write and buffers bytes it cannot send immediately. After the buffer reaches its high watermark, await drain() waits for it to fall to the low watermark. This Python 3.11+ fragment limits each submission and lets downstream capacity pace an asynchronous source:

import asyncio

async def send_chunks(writer, chunks):
    async with asyncio.timeout(5):
        async for chunk in chunks:
            if len(chunk) > 64 * 1024:
                raise ValueError("producer chunk exceeds 64 KiB")
            writer.write(chunk)
            await writer.drain()

This assumes one writer task and a chunks async iterator producing bytes on demand. drain does not acknowledge peer reads, and a high watermark is not a hard memory cap: the last write can overshoot it by a chunk. Source prefetch, other connections, and kernel buffers need separate budgets. On timeout or cancellation, the connection owner must close or otherwise recover according to protocol rules; it cannot blindly resend.

7. Request/Response Order and Request ID

The simplest protocol allows only one outstanding request per connection:

send request A
receive response A
send request B
receive response B

Pipelining allows A and B to be in transit simultaneously. If responses must be returned in request order, a slow A will block already completed B; if out-of-order responses are allowed, each frame must carry a request ID.

[length][request-id][type][payload]

The protocol must also define:

• Is an ID reusable? If so, when?
• How responses, errors, and cancellation refer to requests;
• Which ID space server push uses;
• How duplicate IDs are handled;
• Whether old IDs remain meaningful after reconnection.

HTTP/2, gRPC, and QUIC provide established stream machinery. Reusing an appropriate protocol avoids maintaining another multiplexing state machine.

8. Blocking, Nonblocking, and Readiness

A nonblocking socket returns would-block when it temporarily cannot complete the operation. epoll/kqueue, and similar readiness APIs, inform the event loop that a particular fd "might be ready to read/write" now, without guaranteeing that the next operation will complete all requested work.

The event handler still has to:

• Loop read until a would-block condition occurs or the fair share budget is reached;
• Save parser state for half a header/body;
• Only care about writable when there is data pending to be sent;
• Handle hangup/error and remaining readable data;
• Prevent a constantly active connection from monopolizing the loop;
• Cancel the timer and business tasks before closing.

Edge-triggered mode typically requires draining to would-block, otherwise no new edge may arrive. Level-triggered will continue to notify as long as the condition holds, offering more intuitive behavior but possibly causing repeated wakeups.

If an edge-triggered handler stops early for fairness, it must retain the connection in an application ready queue or use an appropriate rearming strategy. Simply waiting for a fresh edge can stall unread data.

Async/await hides the state machine inside the compiler/runtime, but it doesn't eliminate these protocol rules.

9. TLS Adds Record Framing

For TLS over TCP, TLS wraps application bytes in records. One application write can span several records and TCP segments; one TCP receive can contain a partial record. QUIC uses the TLS handshake and derived keys without carrying data in this TLS record layer.

Applications should use the TLS library's read/write APIs, letting the library manage record, authentication, and reassembly states, and should not manually slice TCP bytes at the underlying encrypted socket level. TLS clean shutdown is distinct from TCP EOF, and security-sensitive protocols must verify receipt of a proper close notification.

10. UDP Framing Boundaries

UDP preserves datagram boundaries, so a length prefix is typically not needed to reassemble the same datagram across multiple recvfrom. However, applications may still include multiple records within a single datagram or split large messages into multiple datagrams.

Custom UDP fragmentation requires message ID, fragment number, total count, timeout, deduplication, memory limit, and congestion control. Losing one fragment can render the entire message unusable. If your needs are reliable, secure, and multi-stream transmission, evaluate QUIC first rather than building it from scratch.

11. Protocol Test Matrix

Don't just test "normally sending a single message." At least cover:

Input	Expected
header reaches only 1 byte each time	correct reassembly
Split the body into arbitrary chunks	Correctly reassemble
Merge two frames into one recv	Parse two
Zero-length frame	Accept or reject according to protocol
Mid-header EOF	Truncated error
EOF in the middle of body	Truncated error
length exceeds limit	reject before allocation
Slow, byte-by-byte input	Total deadline takes effect
Remote end doesn't read response	queued bytes are bounded
cancel and close occur simultaneously	release only once, state remains consistent

A property-based test can split the same encoded stream in many ways and check that every tested partition produces the same messages. Random samples increase coverage; they do not prove all partitions were tested.

12. Summary

An application protocol over TCP must define the behavior missing from a byte stream:

• Framing defines message boundaries;
• recv_exact handles arbitrary splits of header/body;
• The length and decompression result must be bounded before allocation;
• An operation timeout cannot replace an end-to-end deadline;
• Backpressure lets the slow downstream limit the upstream, rather than infinitely accumulating memory;
• Pipelining needs cancellation and response-order rules; out-of-order responses also need IDs or another correlation mechanism;
• Readiness/async still needs state for partial frames, errors, and closure;
• Randomize chunk boundaries in testing and cover truncation and slow clients.

At this point, the protocol defines message boundaries over a continuous byte stream and propagates pressure upstream when buffers reach their limits. The next lesson examines how TCP connections are established and terminated, then tracks sequence numbers, ACKs, flow control, and retransmissions.

References

• Python 3.12: socket
• Python 3.12: asyncio Streams
• Protocol Buffers: Streaming Multiple Messages
• RFC 9001: Using TLS to Secure QUIC
• Linux man-pages: epoll(7)