Scaling Server-Sent Events with Redis Pub/Sub: SRVJ's Notifications

SRVJ is a collaborative diagram tool I've been building — think Miro, but as a playground for backend architecture. The collaborative canvas itself runs over WebSockets (that story gets its own post), but notifications — board invitations, chat messages, mentions — needed a delivery path of their own.

This post is about that path: why it's Server-Sent Events rather than another WebSocket, and the Node.js pipeline behind it — BullMQ, PostgreSQL, and Redis Pub/Sub, arranged so notifications survive crashes, reach every open tab, and keep working when the app scales past one instance.

What is SSE?#

Server-Sent Events is the boring half of real-time: a plain HTTP response the server never finishes. The client opens a request, the server holds the connection open and writes events into it whenever something happens. Communication is strictly one-way — server to client.

The client side is almost embarrassingly simple, because browsers ship it natively as the EventSource API: automatic reconnection, named events, last-event-ID tracking — no library required.

Why SSE?#

Because notifications don't need a second direction. Collaborative editing is genuinely bidirectional — clients push document updates continuously — so it earns its WebSocket. A notification is different: the server has something to say, and the client just listens.

Paying for a bidirectional protocol — the upgrade handshake, a separate connection lifecycle, load-balancer configuration — to send messages one way is buying capability you'll never use. SSE is plain HTTP: it flows through the same middleware, proxies, and auth as every other request.

Notification Architecture#

The pipeline is persist-then-fan-out, and every stage after the user action is asynchronous:

• A domain event occurs (board invitation, chat message, etc.).
• A BullMQ job is created.
• A worker processes the job.
• The notification is persisted in PostgreSQL.
• The worker publishes the event to Redis Pub/Sub.
• The application instance that owns the user's SSE connection delivers the notification instantly.

Generation and delivery are fully decoupled: the API returns as soon as the job is queued, the worker guarantees the notification lands in PostgreSQL, and Redis answers "which instance holds this user's connection" without anyone ever having to ask.

Opening the SSE Stream#

Every authenticated user establishes a long-lived HTTP connection to /stream.

res.writeHead(200, {
  "Content-Type": "text/event-stream",
  "Cache-Control": "no-cache",
  Connection: "keep-alive",
  "X-Accel-Buffering": "no",
});
res.flushHeaders?.();

Every one of those headers is load-bearing:

• Content-Type: text/event-stream — Tells the browser that this endpoint will continuously stream events rather than returning a traditional HTTP response.
• Cache-Control: no-cache — Prevents intermediaries and browsers from caching streamed events.
• Connection: keep-alive — Keeps the HTTP connection open for future events.
• X-Accel-Buffering: no — Disables buffering in Nginx. Without this header, notifications may be delayed because Nginx could buffer responses before sending them to clients.

Managing Active Connections#

The same user is routinely connected from three browser tabs and a phone at once, and the registry has to model that. Each new connection is registered like this:

const client: SSEClient = { userId, res };
let connections = clients.get(userId);
if (!connections) {
  connections = new Set<SSEClient>();
  clients.set(userId, connections);
}
connections.add(client);

Internally, the structure looks like:

Map<userId, Set<SSEClient>>

The Map gives constant-time lookup of everything a user has open; the Set inside it gives cheap add/remove and de-duplication as tabs come and go. When a notification arrives for a user, delivery is one lookup and a loop — every tab, every device, one write each.

Immediately Opening the Stream#

After the connection is registered, the server immediately writes an empty event:

res.write(`: connected\n\n`);

That line is an SSE comment — clients ignore its content — but writing it flushes the response and makes the browser fire onopen immediately, instead of leaving the connection in limbo until the first real notification happens to arrive.

Cleaning Up Disconnected Clients#

Because SSE connections are long-lived, proper cleanup is essential.

req.on("close", () => {
  connections!.delete(client);
  if (connections!.size === 0) {
    clients.delete(userId);
  }
});

Skip this and three things go wrong at once: the registry grows without bound, dead sockets accumulate, and the delivery loop starts writing into closed responses. With long-lived connections, cleanup is a correctness requirement, not hygiene.

Redis Pub/Sub as the Distribution Layer#

Everything so far lives in one process's memory — which breaks the moment SRVJ runs more than one instance:

• Instance A → User 1 connected
• Instance B → User 2 connected
• Instance C → Worker running

The worker on instance C has no idea which instance holds user 1's connection — and it shouldn't have to. Redis Pub/Sub solves the routing problem by never asking it: the worker publishes once, and whichever instance owns the connection delivers. Two Redis clients are needed:

export const redis = createClient({ url });
export const subscriber = redis.duplicate();

The duplicate isn't optional: a Redis connection in subscriber mode can't issue normal commands anymore, so Pub/Sub gets its own dedicated connection while the original client keeps serving the rest of the application.

Publishing Notifications#

After the worker persists the notification in PostgreSQL, it publishes an event.

const payload = {
  id: uuidv4(),
  sender: data.sender,
  userId: data.userId,
  type: data.type,
  title: data.title,
  message: data.message,
  createdAt: new Date().toISOString(),
};

await prisma.notification.create({
  data: {
    fromUserId: Number(payload.sender),
    toUserId: Number(payload.userId),
    title: payload.title,
    message: payload.message,
  }
});

await redis.publish(
  "notifications",
  JSON.stringify(payload)
);

The ordering is the whole design: persist first, publish second. PostgreSQL is the source of truth — an offline user finds the notification waiting when they fetch via the REST API, and a crash between the two steps loses only a realtime push, never the notification itself. Flip the order and the failure mode inverts: a user could see a notification that was never stored.

Delivering Notifications to Connected Users#

Every application instance subscribes to Redis.

await subscriber.subscribe(
  "notifications",
  (message) => {
    const payload = JSON.parse(message);
    const connections = clients.get(
      String(payload.userId)
    );
    if (!connections || connections.size === 0)
      return;

    const frame =
      `event: notification\n` +
      `data: ${JSON.stringify(payload)}\n\n`;

    for (const client of connections) {
      client.res.write(frame);
    }
  }
);

The elegance is in what each part doesn't need to know:

• Each server instance only knows about its local SSE connections.
• Redis broadcasts the event to every instance.
• Only the instance holding the user's connection actually sends the event.

This architecture allows horizontal scaling without introducing sticky sessions or centralized connection management.

Background Processing with BullMQ#

The front of the pipeline matters as much as the delivery end: the API never creates notifications inline. It drops a job on BullMQ and returns.

User Action
      ↓
BullMQ Job
      ↓
Worker
      ↓
Database
      ↓
Redis Pub/Sub
      ↓
SSE

The queue buys the usual things, and every one of them matters here:

• Prevents request blocking.
• Improves API response times.
• Supports retries.
• Handles transient failures.
• Decouples business logic from delivery logic.

A failed notification can be retried without affecting the user's original request.

Hardening for Production#

The pipeline above is the version that ships first, and it's deliberately simple. As SRVJ grows past a single instance and starts retrying jobs under load, three refinements matter. None of them change the core idea — they make it correct at scale.

Scaling the Fan-Out: Per-User Channels#

The version above publishes every notification to a single global notifications channel, and every instance subscribes to it. That's the simplest thing that works, and at a small number of instances it's completely fine.

But notice what happens as you scale out: every instance receives every notification and then discards the ones it doesn't own. With N instances, roughly (N-1)/N of that fan-out is wasted CPU and network that grows with both notification volume and instance count.

The refinement is per-user channels — notif:user:{id}. Each instance subscribes only to the users currently connected to it, and unsubscribes when the last tab for that user disconnects:

// on connect (first tab for this user on this instance)
await subscriber.subscribe(`notif:user:${userId}`, handleMessage);

// on disconnect (last tab gone)
await subscriber.unsubscribe(`notif:user:${userId}`);

The publish side targets the user directly instead of broadcasting:

await redis.publish(`notif:user:${payload.userId}`, JSON.stringify(payload));

Now each instance receives only the messages for users it actually holds.

Tradeoffs. You trade a fixed broadcast cost for subscribe/unsubscribe churn on every connect and disconnect, plus many short-lived channels in Redis. That's a good trade once instance count and notification volume grow; the single global channel is fine while you're small. Pick the per-user model the moment you horizontally scale the app tier in earnest.

Idempotent Worker Writes#

BullMQ delivers at-least-once. A worker that crashes after writing to PostgreSQL but before acking the job will see that job again on restart — and a blind create produces a duplicate notification.

The fix is a stable dedup key (the domain eventId, or a deterministic hash of type + sender + recipient + target) plus a unique constraint, so the second delivery becomes a no-op instead of a duplicate:

await prisma.notification.upsert({
  where: { eventId: payload.eventId },
  update: {},
  create: {
    eventId: payload.eventId,
    fromUserId: Number(payload.sender),
    toUserId: Number(payload.userId),
    title: payload.title,
    message: payload.message,
  },
});

Tradeoff. You need a deterministic key and a unique column — a little schema discipline. And worth being honest: exactly-once across queue → DB → Redis doesn't really exist. Idempotent writes are how you approximate it, and they're non-negotiable the moment the consumer has side effects.

Surviving Reconnections#

SSE auto-reconnects, but Redis Pub/Sub has no buffer: anything published while a client was disconnected is simply gone. Two ways to close that gap:

• Refetch on reconnect — When EventSource fires onopen, the client calls the REST list endpoint (GET /notifications) to reconcile against PostgreSQL. This needs nothing extra and is the pragmatic default for SRVJ.
• Last-Event-ID replay — On reconnect the browser sends the Last-Event-ID header automatically, and the server replays what was missed. This requires a durable per-user log to replay from — which pushes you toward Redis Streams.

For SRVJ, the refetch path wins: the durable store already exists, so the realtime layer is free to be lossy.

Pub/Sub vs Redis Streams#

Redis Pub/Sub is fire-and-forget — no subscriber connected at publish time means the message is dropped, with no replay and no acknowledgement. That's acceptable here precisely because the REST list reconciles anything lost.

If you ever need "no notification missed in realtime, even across reconnects, without a refetch," move the channel to Redis Streams (XADD + consumer groups + XACK). You get at-least-once delivery and replay via XRANGE, at the cost of a trimming policy (MAXLEN), consumer-group bookkeeping, and more memory. Reach for it only when the refetch model stops being good enough — not before.

Why I Chose SSE#

For server-generated notifications, SSE provided:

• Native browser support.
• Automatic reconnection.
• Simple architecture.
• Lower operational complexity.
• Lightweight server-to-client communication.
• Seamless integration with existing HTTP infrastructure.

SSE is not a replacement for WebSockets, but for notification delivery in SRVJ, it turned out to be the right tool for the job.

Next in the series: How CRDTs and Yjs power collaborative editing in SRVJ.