levendlee

Stay humble. Stay hungry. Stay foolish.

Design WhatsApp / Messenger

Written in

by

levendlee

,
  1. Scenario
    1. Features
      1. Register
      2. Contacts
      3. Message
        1. Between two users
        2. Between multiple users
      4. Online Status
      5. Chat History
      6. Multiple Devices
    2. Request
      1. Reality: 1B active users. 750M daily active users.
      2. Assumption:
        1. 100M daily active users. 20 messages per day.
        2. Average QPS 100M * 20 / 86400 = 20K. Peak QPS = 20K * 5 = 100K. Each message 30 bytes. 60G data storage.
  2. Service
    1. Message Service
    2. Real-time Service
  3. Storage
    1. Message Table
      1. Basic: id; from_user; to_user; timestamp; content
      2. Problem: SQL query is really slow; Not extendable for group chat
      3. Solution: Add thread table to accelerate chat query
      4. Improved: id; thread_id; from_user; timestamp; content;
        1. Primary key: id
        2. Sharding key: thread_id
      5. NoSQL: Massive data; No modification.
    2. Thread Table
      1. Basic: id; participaint_id; created_at; updated_at
      2. Problem: Cannot query which threads are the user involved; Cannot have private information
      3. Solution: Add owner id and duplicate the thread table for all participants
      4. Improved: owner_id; thread_id; participaints_id; is_muted; nickname; created_at; updated_at;
        1. Primary key: owner_id & thread_id
        2. Sharding key: owner_id
      5. SQL: (NoSQL doesn’t support secondary index well)
        1. Index by:
        2. owner_id + thread_id (primary key)
        3. owner_id + updated_time (sorted according to time)
  4. Solution
    1. Basic Solution:
      1. The client sends message thread to the server;
      2. The server creates a thread for all receivers.
        1. thread_id can be unique.
        2. multiple users: creater_id + timestamp;
        3. two users: two user_id sorted;
      3. The server creates a message.
      4. The receiver pulls messages from the server.
    2. Realtime Service:
      1. Use sockets to push messages when the user is actively using the application. (Linux supports up to 1M socket connections, but is performance bound)
        1. Http: TCP short connection. The client can only pull data from the server.
        2. Socket: TCP long connection. The server can push data to the client.
      2. Use pull at app launch / Android GCM / IOS APNS service when the user is not actively using the application.
    3. Work Flow:
      1. A launches app, asks the webserver for a push server connection.
      2. A connects push server through socket.
      3. B sends a message to the webserver.
      4. The web server stores the message and sends it to the push server.
      5. The push server pushes the message to A.
  5. Scale:
    1. Group chat
      1. Lots of participants. Lots of push requests.
        Solution: Add channel service (in memory). Message service sends a message to the channel service and channel service sends a message to the subscriptions (online users).
    2. Online Status
      1. The webserver needs to know who is online
        Solution: Heartbeat
      2. The user needs to know which friends are online
        Solution: Webserver summarize

Tags

Leave a comment