Learn

You added read replicas, and now people cannot see the change they just saved

21 October 2026 · System design: one trade-off

You have drawn a primary database and three read replicas, because reads outnumber writes and replicas are cheap. The interviewer nods and asks one thing: a user edits their profile bio, presses save, and the page reloads. Which bio do they see?

The requirement

A person who saves a change must see that change on their next read, from any page. Other people may see it a few seconds later. That is read-your-own-writes, and it is narrower than "everything is always consistent". Saying that difference out loud is half the answer, because it tells the interviewer you will not pay for a promise nobody asked for.

Replicas apply the primary's log with a delay. Most of the time the delay is small. The design question is what happens in the moments when it is not.

Two designs and what each one costs

Option A: pin the writer to the primary for a while. After a user writes, send their reads to the primary for the next few seconds, then go back to the replicas.

• What you get: one rule, no new state beyond a timestamp per session.
• What you pay: the primary carries every read from every recent writer. And the rule is only right while the replica delay stays shorter than the window.

Option B: carry the position of the last write. Each write returns its position in the primary's log. The session keeps the highest position it has written. A read goes to any replica that has applied at least that far, and to the primary only if none has.

• What you get: replicas serve the writer as soon as they have caught up, and the primary is used only when it must be.
• What you pay: every write path has to return a position, every read path has to check one, and the position has to live somewhere the next request can find it.

def save_bio(session, primary, user_id, text):
    pos = primary.write("bio", user_id, text)  # log position of this write
    session["seen_pos"] = max(session.get("seen_pos", 0), pos)

def read_bio(session, primary, replicas, user_id):
    need = session.get("seen_pos", 0)
    for r in replicas:
        if r.applied_pos() >= need:
            return r.read("bio", user_id)
    return primary.read("bio", user_id)

Run it on one case. The save lands at position 120. Replica one has applied 118, replica two has applied 121. The loop skips the first and reads from the second, which already has the new bio. A visitor who never wrote has a position of 0, so any replica serves them.

What goes wrong, and the input that shows it

Option A breaks on a slow afternoon. Say the window is five seconds and a large batch job has pushed replica delay to eight. The user saves at second zero and reloads at second six. The window has closed, the read goes to a replica, and the old bio comes back. Most people then save again, which doubles the writes during the exact period the replicas are behind.

Option B breaks on a second device. The user saves on a laptop, then opens the phone. The phone's session has no position, so it reads from any replica and shows the old bio. If the requirement is per person rather than per browser, the position has to be kept per user on the server, which costs one small lookup on every read.

What to say out loud

Start with the deciding question: "Does only the writer need to see the change at once, and does that include their other devices? And how far behind can a replica fall at its worst, not on average?" Then commit: "If replica delay is reliably short and writes are rare, I pin writers to the primary for a window longer than the worst delay I have measured. If delay can spike or writes are frequent, I carry the last write position, kept per user so a second device gets it too."

The exercise

1. Pick a feature where a person changes something and looks at it straight away: a renamed file, a posted comment, an updated address.
2. Write option A and option B for it in one line each, with what each costs the primary and what each needs stored.
3. Write the two inputs that break them: a delay longer than the window, and a second device with no position.
4. Say the deciding question aloud, then your choice for two different answers to it. Aim for under a minute.

Where this fits in Gadfly

System design is one of Gadfly's four rooms, next to algorithms, behavioral and code review. The design runs on a shared board, guided or as a mock, one trade-off at a time. In guided mode the tutor asks the next question, such as which bio a second device sees, and does not draw the answer. In a mock it behaves like an interviewer, with no coaching during the round and a debrief at the end that quotes what was said, not a grade.