Today, we're launching a demo of Gitlip Editors - our new collaborative coding environment. If you ever wanted to collaborate on code or Markdown with the convenience of Google Docs, this demo is for you.

Gitlip Editors are powered by a multiplayer filesystem built with Durable Objects, WebSockets, and Yjs. We've recorded a video covering the most important features and in the rest of this blog post we'll cover the interesting technical details.

To try out the demo yourself, visit demo.gitlip.com. You don't need to sign in to try the demo, but if you want to collaborate with someone, you might need to add their IP address in the editor "Settings".

Architecture

We've built Gitlip Editors with the intention of integrating them with our infinitely scalable Git infrastructure and therefore enabling collaborative coding with unparalleled Git support. The multiplayer filesystem that powers Gitlip Editors builds on the same principles and reuses some of the components described in "Infinite Git repos on Cloudflare Workers". This means that we can very easily support arbitrarily many collaborative coding environments from day one.

The filesystem supports collaboration via either 1) the HTTP or 2) the WebSockets interface. Both interfaces interact with the same coherent state.

Persistence

Our multiplayer filesystem persists its data to Durable Objects with the help of DOFS - a mostly POSIX compatible filesystem layer with Emscripten's filesystem interface. We made a few upgrades to this layer to ensure optimal storage management for the use case in which there are many small updates to files and directories.

The filesystem stores files in two different representations:

1. block based representation suitable for being operated on by POSIX system calls,
2. update based representation suitable for synchronization and reconciliation using Yjs.

These two representations are managed by a carefully constructed state machine which ensures that each representation is synchronized with the other. Covering the exact behavior of this state machine would take too much space, so we'll describe just the most interesting details.

To achieve great performance we needed to pay attention to the characteristics of the underlying storage. We're still using the key/value interface of Durable Objects and not the (very exciting) recently introduced SQLite interface. For our use case, the most important Durable Object limits to keep in mind are:

1. the 128 KB limit for the maximum size of the stored value, and
2. 4 KB write unit size.

When files are being updated with Yjs updates are being sent over the wire for every keystroke or on any other minor change that the user makes. Before persisting the received updates in the Durable Object, we merge them with the previous ones up to the size of a single write unit (4 KB) and slice them into 128 KB sized blocks which are then batch written into the storage.

We don't flush the updates to the storage on each update, but we debounce and repack them in memory and write them to storage only when that's absolutely necessary. Our custom StorageEngine described in the previous post is very useful for this purpose.

Importing files

It might come as a surprise, but we use the same git-server WebAssembly binary that powers our Git servers in our multiplayer filesystems. The reason for this is that we currently reuse some of git-server functionality (importing data from Git providers, detecting content types), and we plan to reuse even more in the future (computing diffs between trees, applying patches, creating Git commits).

The ability to robustly detect the content type of a file in an environment like this is incredibly useful. We detect content type of each file on either import or upload to decide whether we should enable collaboration for it via the WebSockets/Yjs interface. Currently we allow collaboration for:

• directories
• text files (we treat symlinks as special text files)
• tldraw drawings

All other files are treated as binary files. Some of them we can display over HTTP (images, videos, PDF documents), and others we just allow to be downloaded.

Markdown as a notebook

We love open file formats and we strongly believe they're underutilized, especially Markdown. We think that with a few tweaks Markdown has the potential to be used for much more than it is currently.

Given that at any moment we know the file-type and the content-type for each entry in our multiplayer filesystem, we've been able to repurpose Markdown's syntax for embedding images ![alt text](/path/to/image.png "Title") to enable embedding of any other file type, including: code files, images and tldraw drawings. Each of these files live updates in Markdown as they're modified in the filesystem. Additionally, JavaScript files can be executed directly from the Markdown and we plan to make this possible for code files for other languages in the future.

All of this makes Markdown a kind of notebook suitable for collaborative scientific and technical thinking:

1. Markdown markup - makes it possible to structure the document into sections and paragraphs.
2. Embedded code files - make intertwining code with its broader context possible. The ability to execute that code within the document itself, makes the document seem alive and reminiscent of Jupyter Notebooks.
3. Embedded math - enables succinct expression of mathematical ideas (using KaTeX).
4. Embedded tldraw drawings - enable free-hand collaborative whiteboarding within the Markdown document itself.
5. Embedded SVG images - make precise technical drawing within the document possible.

Please watch the video recording to see this in action. We plan to build further in this direction, by utilizing open file formats and making Markdown as powerful of a notebook as it can be.

Reflections on Yjs

Yjs is an amazing CRDT library and Kevin Jahns is an incredible engineer from which we've learned a lot. The biggest selling points of Yjs is that it's very lean, very performant and very easy get started with.

During our experimentations with Yjs, we wrote several Yjs providers for synchronization and storage. Finally, for our multiplayer filesystem we wrote a custom WebSockets Yjs provider that supports multi-document synchronization and is fine tuned for our use case.

In Durable Objects themselves, to make an effective use of the Hibernation API, we needed modify Yjs' presence mechanisms that eagerly report presence every 30 seconds in every client. This way we avoid unnecessarily waking up the Durable Object when WebSockets are not actively used.

Yjs made it easy for us to orchestrate loading and unloading of the Yjs documents from persistent storage into memory, depending on which exact documents are currently being viewed or edited by the clients. This way we make sure we always stay well below 128 MB of memory limit in Durable Objects.

Our issues with Yjs

Before mentioning the issues we encountered, we have to emphasize once more that we're huge fans of Yjs and we're grateful to work with it. We're aware that CRDTs in general are an active area of research and library development is in constant flux. What follows is subjective and based on our experiences.

Issue #1 - The API is simpler than it should be. Unfortunately the API seems to be optimized for the simplest use case which invariably involves synchronizing only one document. For example, the concept of a "client" is bundled with the concept of a "document" and this has caused us some issues in our implementation.

Issue #2: Subdocuments API is inadequate for multi-document sync. Primarily because it necessitates a strict ordering between syncing the parent document and the child document. It also makes the API seemingly unnecessarily complex by introducing a two-tier system for documents. We currently rely on this mechanism, but we're planning to replace it by introducing a concept of a "document store" which would be the central place to manage relationships between documents.

Issue #3: Overuse of synchronous callbacks. Yjs seems to ideologically prefer synchronous callbacks and almost every class in Yjs extends ObservableV2 from lib0. Mixing synchronous callbacks with asynchronous code based on promises makes it really difficult to design and implement a complex state machine in which order of execution is really important. This is the exact point in our system where we want to ensure maximum correctness possible and synchronous callbacks hamper these efforts.

Overall, Yjs is an incredible library for introducing collaborative features into your app, but you could stretch it if you push hard enough.

Future

Our long term vision encompasses these goals:

1. introducing IDE features into our editor,
2. making it possible for human and AI collaborators to collaborate seamlessly,
3. making mobile coding incredible,
4. making our Editor local-first,
5. enabling time-travel in our filesystem,
6. making it possible to broadcast a livestream coding session to an audience.

Our immediate goal is to launch Gitlip over the coming weeks - where collaborative coding, Git repos and 1-click deployments will be integrated into one app.

Conclusion

We're launching very soon! Stay up to date with our journey by following @nataliemarleny or subscribing to our mailing list.

We are considering offering Gitlip Editor as a service for other platform builders, if you would like to integrate this please reach out.

We're grateful for amazing open-source and commercial software we used to build this, most notably: Yjs, tldraw, KaTeX, Next.js and the Cloudflare Workers platform.