Design a collaborative editor (Google Docs)

A collaborative editor lets many people type into one document at the same time and see each other's changes and cursors appear within a fraction of a second. The difficulty is not the text box. It is that everyone is editing the same shared state at once, so two people who type in the same spot at the same moment must both keep their change and end up looking at an identical document, no matter which edit reached the server first. Get that wrong and you produce the worst bug in the category, which is a keystroke that silently disappears.

Scope and requirements

Functionally, a user creates a document, many users edit it at the same time, everyone sees the others' edits live, and everyone sees the others' cursors and whether they are present. Rich text, permissions, and version history are real features that can be parked once they are named, so this stays a plain-text editor to keep the focus on multi-writer consistency.

Non-functionally, four constraints shape the design. Every editor must converge on the same document, and this only needs to be eventual, since two people do not have to see edits in the same physical order as long as they end up with identical text. Updates must land in under 100 milliseconds so typing feels shared rather than laggy. The system must hold millions of concurrent users across billions of documents. And there is one deliberately load-bearing limit, at most 100 concurrent editors per document, which is what makes it possible to keep all of a document's connections and operations in the memory of one server. Real Google Docs does the same, dropping extra people to read-only past a threshold. Durability rounds it out, since edits have to survive a server restart.

Sizing the problem

The 100-editor cap is the number the whole design leans on. It means a document is never a throughput problem, so one Document Service server can own a document outright, hold its operations in memory, and order edits without coordinating with anyone. The pressure lives elsewhere. Shipping the full document on every keystroke would move the whole document on every key press, which is the first reason that approach dies. Storage is the other one, since billions of documents at roughly 50 KB each is about 50 TB, and a busy document accumulates operations without end, so something has to collapse that log. A single unscaled server handles only a few thousand users total, which is what forces the connection-sharding work later.

Why sending the whole document loses edits

The first design has each editor send the entire document to the server on every change, with the server keeping the latest copy. It fails immediately under concurrency. If User A and User B both edit at the same moment, each sends a full snapshot built from what they saw a moment ago, and whichever snapshot arrives last overwrites the other completely. A's paragraph vanishes because B's snapshot never contained it. This is last-write-wins on the whole document, and it silently destroys concurrent edits, on top of the bandwidth problem. The fix is to stop sending documents and start sending operations, small deltas like insert this text at this position or delete this many characters here.

Core entities and the data model

Four nouns carry the design. An Editor is a user in a document. A Document is the text being edited. An Edit, or operation, is a single change such as insert(pos, text) or delete(pos, count). A Cursor is an editor's position and, by extension, their presence.

The stores split by how they are used. Document metadata is small and stable, so it lives in Postgres, holding the document id, title, and a documentVersionId that points at the current version. The operations are large and append-heavy, so they live in Cassandra, partitioned by document id and ordered by a server-assigned timestamp. The important idea is that the document is not stored as a blob. It is stored as the sequence of operations that built it, and the text is reconstructed by replaying them. Opening a document reads its version pointer from Postgres, reads that version's operations from Cassandra, and replays them.

metadata (Postgres)          operations (Cassandra, partition = docId, order = ts)
  docId                        docId | version | ts        | op
  title                        d1    | 41      | ...       | insert(0, "H")
  documentVersionId  ---------> d1    | 42      | ...       | insert(1, "i")
                               d1    | 43      | ...       | delete(0, 1)

The connection model

Editing is a two-way, high-frequency stream. The client is constantly pushing keystrokes and cursor moves up, and the server is constantly pushing other people's edits down, so the connection is a WebSocket, one per open document. Server-sent events are one-directional and cannot carry the client's own edits back up, which rules them out here even though they fit a read-only feed. Polling cannot meet the 100-millisecond target without wasting a request on every quiet interval. A user with three documents open holds three sockets, which is fine because those are usually separate tabs anyway.

The high-level design

Creating a document is ordinary web-service work. The client calls an API gateway, a CRUD service writes the id and title to Postgres, and returns the id. Editing is where the design earns its keep. A client sends each change as an operation to the Document Service, the stateful server that owns the document in memory. That server transforms the operation against the operations it has already ordered, appends it to Cassandra with a server-set timestamp that fixes its place in the canonical order, and only acknowledges the client after the durable write so a lost operation is simply retried. It then broadcasts the operation to every other editor over their sockets. On connect, the server replays all of the document's prior operations to the newcomer so everyone starts from the same baseline.

Every editor of a document holds a WebSocket to the one server that owns it. That server orders operations, appends them to the Cassandra log, tracks the version pointer in Postgres, and fans edits back out. Databases are drawn as cylinders.

Operational Transformation, the heart of it

An operation carries an index, and that index is only meaningful against the exact document state it was created on. A delete(6) means delete the character at position 6 as of the moment the user pressed backspace. If someone else's insert lands first and shifts everything to the right, position 6 now points at a different character, and applying the delete as written removes the wrong one. Operational Transformation fixes this by rewriting an incoming operation against the operations already ordered ahead of it, so it still does what the user meant.

The document is "Hello!", and the exclamation mark is its sixth character. User A sends insert(5, ", world"), which drops seven characters in just after Hello, and User B sends delete(6) to remove the mark. The server orders A first, so those seven characters now sit ahead of the mark, which has slid from the sixth character to the thirteenth. B's delete is transformed from delete(6) into delete(13), so the exclamation mark is still what gets removed, and both editors end up with "Hello, world".

The delete is re-expressed against the insert that was ordered ahead of it, so the intended character is still the one removed. Green is the converged result both editors see.

The central server is the single source of the canonical order, so there is one ground truth to transform against. And each client transforms locally too. A client applies its own edits optimistically the instant they are typed, so the editor feels immediate, and holds them unacknowledged. When a remote operation arrives, the client transforms it against its own unacknowledged operations before applying it. The physical order of edits therefore differs per screen, yet everyone converges on the same text, which is the guarantee OT gives.

# an operation is a small delta, never the whole document
#   ("insert", pos, text)   e.g. ("insert", 5, ", world")
#   ("delete", pos, count)  e.g. ("delete", 6, 1)

def transform_against_insert(op, ins):
    """Re-express op so it still means what the user intended, given that
    `ins` (an insert) was ordered before it."""
    _, ins_pos, ins_text = ins
    kind, pos, arg = op
    if pos >= ins_pos:                  # the insert landed at or before us
        pos += len(ins_text)            # ... so shift right by its length
    return (kind, pos, arg)

# "Hello!", A = insert(5, ", world") sequenced first, B = delete(6)
transform_against_insert(("delete", 6, 1), ("insert", 5, ", world"))
#   -> ("delete", 13, 1)   the "!" is still the character removed

# client -> server: each edit is an operation tagged with the version it was made against
ws.send({"type": "insert", "pos": 5, "text": ", world", "baseVersion": 41})

# the Document Service that owns this doc
def on_op(op, base_version):
    for prior in ops_since(base_version):      # transform against newer ordered ops
        op = transform(op, prior)
    version = append_to_cassandra(doc_id, op)  # server timestamp fixes the order
    ack(op, version)                           # only ACK after the durable write
    broadcast_to_others(doc_id, op, version)   # local fan-out, all editors are here

OT versus CRDT, and what will not work

The other way to make concurrent edits converge is a conflict-free replicated data type. A CRDT gives every character a unique, immutable position identifier that never changes once assigned. The identifiers are densely orderable, so a character inserted between positions 4 and 5 gets something like 4.3, built from a random value plus a per-site tiebreaker so that two people inserting in the same spot still get distinct, orderable ids. A new character slots between two existing ones without touching either, and because merges commute and need no referee, CRDTs can run peer to peer and offline. The cost is heavier metadata on every character and tombstones for deletes. Libraries like Yjs and Automerge implement CRDTs for exactly this kind of text.

The trade is clean. OT is light on memory and fits text well, but it needs a central server to order operations. CRDT needs no central authority but carries more metadata on every character, and OT is in turn the one that is notoriously hard to prove correct once the central server is removed, which is much of why CRDTs exist. Google Docs uses OT, because it already runs a central stateful server for durability and broadcast, and because the 100-editor cap and plain-text focus erase the advantages a CRDT would bring. You would reach for a CRDT when you have far more collaborators, a peer-to-peer target over WebRTC, or serious offline editing.

Keeping storage bounded with compaction

Storing the operation log and replaying it is clean until the log gets long. Billions of documents push storage toward tens of terabytes, a long-lived document accumulates millions of operations that are slow to transfer and replay on join, and active documents sit in a server's memory. Compaction fixes all three by collapsing many operations into one, so a whole document becomes a single insert(0, contents). Compaction writes a new version and then flips the documentVersionId pointer in Postgres with a compare-and-swap.

The timing is the subtle part. Compaction rewrites the operations that follow, so it must not corrupt a document that is being edited. Cassandra offers only row-level transactions, so the design leans on that version pointer as the atomic swap, and if new edits arrive mid-compaction the compaction is aborted. The easy case, and the one to lead with, is offline compaction that runs when the last editor disconnects, because at that instant the Document Service holds every operation in memory, owns the document exclusively, and has no one to coordinate with. A common and reasonable extension writes a full snapshot to object storage and keeps only the tail of recent operations in Cassandra, which is the same shape as a database checkpoint plus its write-ahead log.

Sharding connections by document

One server cannot hold millions of sockets, and it is a single point of failure. The constraint that decides how to scale is that OT needs all of a document's operations and all of its editors on one server, so the shard key is the document. The Document Service runs as a ring of servers placed by consistent hashing on the document id, with ZooKeeper holding the ring. A client opens a plain HTTP connection to any server with the document id, that server checks the ring, and if it is not the owner it redirects the client to the one that is. The owner upgrades the connection to a WebSocket and loads the document's operations from Cassandra if they are not already in memory. Every editor of that document is now on one server, so a broadcast is a local fan-out.

Servers sit on a consistent hash ring keyed by document id, so adding or removing one reshuffles only a small slice of documents. A client is redirected to the owner of its document, which then holds the WebSocket and the in-memory operations.

Consistent hashing keeps a membership change cheap, since only a fraction of documents move, but the cost is that a ring change has to move state, displacing editors so they reconnect to the new owner and migrating the document's operations. This is also why chat-style Redis pub/sub fan-out, which works when a channel's users can sit on different stateless servers, does not fit here. OT needs every operation for a document on one stateful server, so a server publishing only to itself buys nothing, and because pub/sub is fire-and-forget, a client that drops for a moment simply misses the operations it needed, which the one owning server would otherwise have replayed.

Questions and answers

The core ideas as questions with the answers given outright. Each wrong multiple-choice option is marked with why it is wrong, and the ordering ones show the correct sequence.

References

1. Ellis and Gibbs, Concurrency Control in Groupware Systems (1989), the original Operational Transformation paper.
2. Shapiro et al., Conflict-free Replicated Data Types (2011).
3. Google Drive Engineering, What's different about the new Google Docs (2010), on managing edits server-side.
4. Kleppmann, Designing Data-Intensive Applications (2017), on logs, snapshots, and consistency.