How A Message Gets From Your Phone To Your Friend's In Milliseconds

You send a text, and it shows up on your friend’s phone almost instantly — whether they’re on the other side of the world or across the room, whether their app is open or their phone is asleep in a pocket. That instant delivery is doing a lot of hidden work: your friend’s device isn’t listening on a fixed address the way a website is, it’s behind NAT, switching networks, going to sleep, and reconnecting constantly — and somewhere in a fleet of thousands of stateless backend servers, exactly one of them has to know precisely which one is currently holding the live connection to that specific phone.

The Problem, Properly Framed

Most backend services are built to be stateless and horizontally scalable — any request can land on any server, because no server holds anything unique. A persistent connection breaks that assumption immediately: once your friend’s phone opens a socket to gateway server number 47, that connection lives on server 47 specifically, not anywhere else in the fleet. Your message can arrive at any of the thousand servers handling incoming traffic, but it only does any good if it eventually reaches the one specific server holding your friend’s live connection — and that mapping changes constantly as devices disconnect, reconnect, and get load-balanced to different servers.

The Core Mechanics

The Connection Registry: turning “somewhere in the fleet” into “exactly here”

When a device establishes a persistent connection, the gateway server that accepts it writes an entry to a shared, fast lookup store — user ID mapped to “which gateway server, which connection ID.” This registry is the single source of truth that turns the routing problem into a two-step lookup: find which server holds the connection, then hand the message to that server directly.

# simplified connection registry write, on connect
def on_connect(user_id, gateway_id, connection_id):
    registry.set(f"conn:{user_id}", {"gateway": gateway_id, "conn": connection_id}, ttl=60)
    # short TTL — the gateway must heartbeat-refresh this or the entry expires,
    # which is what makes stale entries self-cleaning after a crash

Routing: pub-sub, not a direct call

The sending server doesn’t call the recipient’s gateway directly — it publishes the message to a pub-sub topic scoped to that recipient, and the gateway holding the live connection is the one subscriber that’s actually listening. This decouples the sender from needing to know anything about connection topology at all; it just publishes and trusts the routing layer.

The Offline Fallback: two completely different delivery paths, one message

If the registry lookup comes back empty — no live connection — the message can’t be pushed in real time at all, so the system falls back to a platform push notification service (APNs or FCM) to wake the device, and separately writes the message to a durable per-user queue so it’s there waiting the next time the device reconnects. These are two different systems solving two different problems: the push notification gets attention now, the queue guarantees the message isn’t lost.

The Trade-offs / What Goes Wrong

At-least-once delivery means duplicates are a client-side problem to solve, not a server-side one to prevent. Acknowledgments can get lost on a flaky mobile network just as easily as the original message, and re-sending on an unacknowledged message is the only sane default — which means the client has to deduplicate by message ID, because “guaranteed exactly once” delivery over an unreliable network is not actually achievable without that cooperation.

A crashed gateway server silently strands every connection it was holding. The registry entry doesn’t know the server died — it just expires on its own TTL if the heartbeat stops, which means there’s a real window (the TTL duration) where messages route toward a server that’s already gone before the registry catches up and a device reconnects elsewhere.

Push notifications are not a delivery guarantee, they’re a best-effort wake-up call. APNs and FCM can be delayed, coalesced, or dropped entirely under load or on certain device power states, and mobile OSes throttle background wake-ups aggressively to protect battery life. This is exactly why the durable queue exists as a separate mechanism — the push notification’s job is speed when it works, the queue’s job is correctness when it doesn’t.

The registry itself becomes a single point of contention at scale. Every connect, disconnect, and message send touches it, which means it has to be fast, highly available, and horizontally scalable in its own right — and if it becomes the bottleneck, the exact feature that made the rest of the system stateless just reintroduced a hot, stateful dependency in a new place.

Takeaways

• A connection registry exists to solve one specific problem — turning a stateless fleet’s “somewhere in there” into a precise, fast lookup for exactly which server holds a given connection.
• Pub-sub routing decouples sender from recipient topology entirely — the sender never needs to know which server, or even whether the recipient is currently connected.
• Push notifications and durable queues solve different problems and both are necessary — one is a best-effort wake-up, the other is the actual correctness guarantee.
• At-least-once delivery pushes deduplication to the client by necessity, not by a lazy design choice — true exactly-once delivery isn’t achievable over an unreliable network.
• The registry that makes the system stateless is itself a stateful bottleneck waiting to happen if it isn’t built to scale as aggressively as everything it’s coordinating.

If you found this useful or want to discuss it further, connect with me on GitHub or LinkedIn.