Global Real-Time Chat & Messaging (Discord & WhatsApp)

Real-time chat platforms power instant communication across 1-on-1 direct messages (DMs), private groups, and large community channels. This worked example assumes 50 million concurrent connections. It targets low delivery latency, a deterministic order within each conversation, synchronized delivery/read receipts, live presence, and durable append-only message persistence. These are design targets for the exercise, not claims about one named product.

The communication engine must satisfy uncompromising real-time SLAs while supporting diverse conversation topologies.

Capacity planning sizes the socket memory footprint, network bandwidth, and five-year storage tier for 50M concurrent connections.

The architecture decouples stateful connection management from business logic, message routing, and persistent storage.

1. 01Layer 4 TCP Load Balancers (AWS NLB / Maglev): Distributes incoming TCP/TLS handshakes across the gateway fleet using 5-tuple hashing with Direct Server Return (DSR).
2. 02Stateful WebSocket Gateways (Erlang/Elixir BEAM or Go): Maintains open TCP sockets, serializes JSON/Protobuf frames, validates user tokens, and runs local connection heartbeats.
3. 03Session Registry Cluster (Redis Cluster): Maintains a lightning-fast distributed mapping: user_id -> gateway_instance_id. Keys are kept alive via 60-second TTLs refreshed by gateway heartbeats.
4. 04Message Routing Engine (NATS / RabbitMQ / Redis Pub/Sub): Routes messages from the author's gateway to the recipient's gateway node.
5. 05Message Persistence Workers (Kafka Consumer): Consume durably accepted messages and batch writes to ScyllaDB. Whether delivery waits for database persistence is an explicit latency-versus-durability decision.
6. 06ScyllaDB / Cassandra Cluster: Durable wide-column storage for channel message histories, partitioned by channel ID and time buckets.
7. 07Push Notification Worker Fleet: Dispatches Apple APNs and Google FCM alerts when the session registry indicates that a recipient is offline.

The persistence layer uses ScyllaDB (or Apache Cassandra) CQL schemas designed around immutable, append-only wide-column rows.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

-- ScyllaDB / Apache Cassandra Message Storage Schema
CREATE KEYSPACE chat_system WITH replication = {
    'class': 'NetworkTopologyStrategy',
    'us-east': 3,
    'eu-west': 3
};

-- Time-bucketed message schema prevents unbounded partition growth
CREATE TABLE chat_system.messages (
    channel_id UUID,
    bucket_id INT,                              -- Time bucket: YYYYMM (e.g., 202609)
    message_id BIGINT,                          -- Monotonic Snowflake ID
    sender_id UUID,
    content TEXT,
    attachments JSON,                           -- Image/video metadata URLs
    is_pinned BOOLEAN,
    created_at TIMESTAMP,
    PRIMARY KEY ((channel_id, bucket_id), message_id)
) WITH CLUSTERING ORDER BY (message_id DESC);

-- Channel Read Receipts and Watermarks
CREATE TABLE chat_system.channel_read_markers (
    channel_id UUID,
    user_id UUID,
    last_read_message_id BIGINT,
    last_read_at TIMESTAMP,
    PRIMARY KEY (channel_id, user_id)
);

-- Redis Session Registry Key Schema:
-- Key: session:{user_id} -> Value: "gateway-node-88" (TTL: 60s)
-- Key: presence:{user_id} -> Value: "ONLINE" (TTL: 35s)

The end-to-end message delivery flow coordinates socket gateways, session registries, and message brokers with sub-100ms latency.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49

// Production Gateway Message Dispatcher & Routing Logic
export interface ChatMessageFrame {
  channelId: string;
  messageId: bigint;
  senderId: string;
  recipientId?: string; // Present in 1-on-1 DM
  content: string;
  timestamp: number;
}

export class GatewayMessageRouter {
  constructor(
    private sessionRegistry: RedisClient,
    private messageBus: PubSubBroker,
    private localSocketManager: SocketManager
  ) {}

  async handleInboundMessage(frame: ChatMessageFrame): Promise<void> {
    // 1. Assign monotonic Snowflake message ID if not present
    frame.messageId = SnowflakeGenerator.nextId();
    frame.timestamp = Date.now();

    // 2. Publish to Kafka for asynchronous durable persistence (WAL)
    await kafkaProducer.send('chat-messages-persistence', frame);

    // 3. 1-on-1 Direct Message Route
    if (frame.recipientId) {
      const targetGateway = await this.sessionRegistry.get(`session:${frame.recipientId}`);

      if (!targetGateway) {
        // Recipient is OFFLINE: Route to Push Notification Queue
        await kafkaProducer.send('offline-push-notifications', {
          userId: frame.recipientId,
          title: `New message from ${frame.senderId}`,
          body: frame.content,
        });
        return;
      }

      if (this.localSocketManager.hasLocalSocket(frame.recipientId)) {
        // Recipient is connected to THIS SAME gateway instance! Direct write:
        this.localSocketManager.sendFrame(frame.recipientId, frame);
      } else {
        // Recipient is on a different gateway: Dispatch via Pub/Sub topic for that node
        await this.messageBus.publish(`gateway-inbox:${targetGateway}`, frame);
      }
    }
  }
}

1. 01Step 1: Socket Ingress: Alice sends a chat frame over her WebSocket to Gateway Node 12.
2. 02Step 2: Conversation Sequencing: A partition or sequencer keyed by channel_id assigns a monotonically increasing channel sequence. A Snowflake-style ID may remain the globally unique identifier, but it is not used as a perfect global clock.
3. 03Step 3: Persistence Queue: Message is emitted to Kafka topic chat-messages. A consumer worker persists the row into ScyllaDB with quorum consistency (LOCAL_QUORUM).
4. 04Step 4: Recipient Discovery: Router queries Redis Session Registry for Bob's active gateway: GET session:bob_uuid.
5. 05Step 5a (Bob is Online): Registry returns "gateway-node-88". Router publishes the frame to NATS topic gateway.node88. Gateway Node 88 consumes the frame and pushes it down Bob's open WebSocket socket in <15ms.
6. 06Step 5b (Bob is Offline): Registry returns null. Router pushes payload to Apple APNs / Google FCM queue to wake up Bob's phone.
7. 07Step 6: Delivery Acknowledgment: Bob's client receives frame and automatically emits an ack frame DELIVERED. Gateway updates Alice's UI with a double-check marker.