Architecting Conflict Resolution Strategies for Concurrent Multi-Device Message Streams

Architectural Decision Memo: Deterministic Conflict Resolution in Multi-Device Messaging

Context and Problem Statement

In modern distributed messaging architectures, users frequently interact with the same conversation thread across multiple endpoints—typically a mobile device and a desktop client. A common failure mode occurs when a user initiates messages from both devices simultaneously under conditions of variable network latency.

If the backend relies on server-side ingestion timestamps to order these messages, the resulting conversation history often appears fragmented. For instance, a message sent from a mobile device at 10:00:01 AM might arrive at the server at 10:00:05 AM due to a cellular dead zone, while a desktop message sent at 10:00:03 AM arrives at 10:00:04 AM. A server-side timestamping approach would incorrectly sequence the desktop message before the mobile message, breaking the logical flow of the conversation and causing inconsistent read-state indicators across the user's devices.

This memo outlines the architectural shift from server-side ingestion ordering to client-side causal ordering to ensure a consistent, deterministic conversation history.

The Architectural Choice: Causal Ordering via Vector Clocks

To resolve these conflicts, we must decouple the message sequence from the server's arrival time. The proposed solution is to implement a causal ordering mechanism using logical clocks, specifically a simplified version of Vector Clocks or Hybrid Logical Clocks (HLC).

Each client maintains a local counter for the conversation thread. When a message is generated, the client attaches a tuple consisting of (device_id, sequence_number, physical_timestamp). The backend treats the sequence_number as the primary sort key for the thread, while the physical_timestamp serves as a tie-breaker for messages originating from different devices.

Alternatives Considered and Rejected

1. Server-Side Ingestion Timestamping (The Status Quo)

• Mechanism: The server assigns a received_at timestamp upon message arrival.

• Rejected because: It is non-deterministic. It relies on the network path and server processing load rather than the user's intent. It fails to account for "offline-first" scenarios where messages are queued on the client and flushed in batches.

2. Global Sequence Numbering (Centralized Authority)

• Mechanism: Every message must request a monotonically increasing ID from a centralized service (e.g., a distributed counter or a database sequence) before being broadcast.

• Rejected because: This introduces a synchronous dependency on a central authority. In a multi-device scenario, if the mobile device loses connectivity while waiting for a sequence ID, the entire thread stalls. It creates a single point of failure and increases latency for every message sent.

Trade-offs and Operational Risks

Trade-offs:

• Complexity: Implementing causal ordering requires the client to track state. If a client loses its local sequence counter (e.g., due to a cache clear or app reinstallation), it must perform a "state reconciliation" with the server to determine the next valid sequence number.

• Storage Overhead: Each message metadata payload increases slightly to accommodate the device_id and sequence_number.

Operational Risks:

• Clock Skew: While we prioritize sequence numbers, physical timestamps are still used for tie-breaking. If a device's system clock is significantly drifted, the tie-breaking logic may behave unexpectedly. We mitigate this by enforcing a "drift threshold" where the server rejects messages with timestamps outside a reasonable window (e.g., +/- 5 minutes) and forces a clock synchronization.

• Reconciliation Latency: When a device reconnects after a long period of offline activity, it may attempt to push a large batch of messages. If these messages are processed sequentially, the UI may experience "jitter" as the conversation history updates rapidly.

A Surprising Observation: The "Ghost Read" Phenomenon

During testing, we observed a specific edge case: the "Ghost Read." When a user reads a message on their desktop, the read-receipt is broadcast to the server. If the mobile device is offline, it does not receive the update. When the mobile device comes back online, it sends a message that it generated before it received the read-receipt.

If the server blindly accepts the mobile device's state, it may overwrite the "read" status with an "unread" status because the mobile device's local state was stale. To prevent this, our architecture now mandates that read-receipts are treated as immutable events with higher precedence than message content updates. The server must maintain a "high-water mark" for read-receipts, ensuring that a device cannot "roll back" the read-state of a conversation.

Counterexample: The Out-of-Order Batch

Consider a scenario where a user sends three messages (A, B, C) from a mobile device. Due to a network hiccup, the server receives them in the order (A, C, B).

Under a naive implementation, the server would persist them as (A, C, B). With our causal ordering strategy, the server inspects the sequence_number attached to each message. It detects that message B is missing (or arrived late) and holds message C in a temporary buffer (a "reordering buffer") for a short duration (e.g., 500ms). If B arrives, the server re-sequences them correctly before committing to the database and broadcasting to other devices. If B does not arrive, the server proceeds with the available sequence, marking the gap for the client to resolve.

Evidence for Invalidation

This architectural decision would be considered invalid if:

1. Latency Spikes: The reordering buffer on the server becomes a bottleneck, causing message delivery latency to exceed 200ms for more than 0.1% of traffic.

2. Client-Side Divergence: We observe a statistically significant increase in "state mismatch" reports where the mobile and desktop clients fail to converge on the same conversation history after a network partition.

3. Resource Exhaustion: The overhead of maintaining sequence state for millions of concurrent threads exceeds the memory capacity of our ingestion layer, necessitating a move back to a stateless, server-timestamped model.

Conclusion

By shifting the responsibility of ordering to the client via causal metadata, we ensure that the conversation history reflects the user's intent rather than the vagaries of network routing. While this increases the complexity of the client-side state machine, it is a necessary trade-off for maintaining a consistent, reliable experience in a multi-device, real-time environment. Architects should focus on robust reconciliation protocols to handle the inevitable edge cases where client-side state becomes desynchronized from the server's source of truth.