L-03Project / Deep DiveEL +0.00 m
Projects
SRVJ logo

Deep Dive · Real-time

SRVJ

A real-time collaborative diagram editor: CRDT synchronization, presence, and persistence built from protocol level up.

Role
Solo — design and implementation, backend and frontend
Timeline
Stack
TypeScript Express.js Yjs (CRDTs) PostgreSQL Prisma MongoDB Redis BullMQ Docker Vue.js ...

01 — Overview

SRVJ is a collaborative diagram editor where multiple people edit the same board at the same time — live cursors, presence, and conflict-free merging of concurrent edits. The interesting work is invisible: keeping every client convergent without a central lock, and persisting a document that is technically a binary CRDT state.

I built it solo, end to end: the synchronization server, authentication and access control, the persistence layer, notifications, background processing, and the Vue frontend. The goal was to understand real-time collaboration infrastructure by building it from the protocol level up, not by wrapping a hosted service.

02 — What I Built

CRDT synchronization server

Problem
Concurrent edits from multiple clients must converge to the same document without locking or a "last write wins" data loss.
Approach
Yjs CRDTs carry the conflict resolution; the server’s job is transport and fan-out, not merging.
Implementation
A custom WebSocket server implementing the Yjs sync and awareness protocols. Each board gets a dedicated room, so collaboration sessions are isolated and a busy board cannot leak updates or presence into another.
Outcome
Live cursors, presence, and seamless multiplayer editing, with convergence guaranteed by the CRDT rather than by server-side coordination.

Authenticated, authorized WebSocket connections

Problem
WebSockets bypass the usual per-request middleware chain, so authentication and permissions have to be enforced at the connection boundary.
Implementation
Connections authenticate with PASETO v4 tokens, and project-level role-based access control is enforced before a client joins a board room.
Outcome
A client that should not see a board never receives its state or its awareness traffic.

Dual-snapshot persistence

Problem
The authoritative document is a Yjs binary state — lossless, but opaque to queries. An API that needs to list, search, or render previews cannot work against a binary blob.
Implementation
Every snapshot is stored twice: the authoritative Yjs binary state for lossless recovery, and a denormalized JSON representation for efficient querying and API reads.
Outcome
Recovery replays exact CRDT state; read paths never touch or decode the binary format.

Split storage: PostgreSQL and MongoDB

Implementation
PostgreSQL with Prisma holds the relational data — users, projects, memberships, sharing — where integrity and joins matter. MongoDB with Mongoose absorbs the high-frequency diagram mutation writes.

Scalable notifications and background jobs

Problem
Server-sent notification streams are pinned to one process; with multiple instances behind a load balancer, an event raised on one instance must reach clients connected to another.
Implementation
Notifications go out over SSE with Redis Pub/Sub fanning events out across instances. BullMQ workers handle asynchronous email and notification processing off the request path.
Outcome
Notification delivery scales horizontally, and slow work never blocks an API response.

Reliability and operations baseline

Implementation
Zod validation on inputs, Helmet and CORS, rate limiting, centralized error handling, structured logging with Pino, and Prometheus metrics. Collaboration behavior is verified with integration tests, and the platform runs containerized via Docker Compose behind Nginx.

03 — Engineering Decisions

Two snapshot formats instead of one

The Yjs binary state is the only lossless representation of the document, but it cannot be queried. Rather than force one format to do both jobs badly, each read path gets the format built for it: binary for recovery, denormalized JSON for the API.

Tradeoff

Every snapshot is written and stored twice, and the two representations must be kept consistent.

PostgreSQL and MongoDB side by side

Identity, membership, and sharing are relational problems — foreign keys and constraints catch real bugs there. Diagram mutations are high-frequency, schema-light writes that fit a document store.

Tradeoff

Two databases mean two operational surfaces: separate backups, migrations, and failure modes.

SSE for notifications, WebSockets for collaboration

Board collaboration is genuinely bidirectional, so it runs over WebSockets. Notifications are one-way server-to-client, and SSE with Redis Pub/Sub fan-out delivers them across horizontally scaled instances without holding a second full-duplex socket per user.

04 — Security

PASETO v4 tokens

WebSocket connections are secured with PASETO v4 authentication before any board state is exchanged.

Project-level RBAC

Role-based access control is enforced per project, and collaboration sessions are isolated into dedicated board rooms.

Input and transport hardening

Zod validates inputs at the boundary; Helmet, CORS, and rate limiting cover the transport layer; errors flow through one centralized handler so failures never leak internals.

Further Reading