| |
| #  |
| |
| > A CRDT framework with a powerful abstraction of shared data |
| |
| Yjs is a [CRDT implementation](#yjs-crdt-algorithm) that exposes its internal |
| data structure as *shared types*. Shared types are common data types like `Map` |
| or `Array` with superpowers: changes are automatically distributed to other |
| peers and merged without merge conflicts. |
| |
| Yjs is **network agnostic** (p2p!), supports many existing **rich text |
| editors**, **offline editing**, **version snapshots**, **undo/redo** and |
| **shared cursors**. It scales well with an unlimited number of users and is well |
| suited for even large documents. |
| |
| * Demos: [https://github.com/yjs/yjs-demos](https://github.com/yjs/yjs-demos) |
| * Discuss: [https://discuss.yjs.dev](https://discuss.yjs.dev) |
| * Chat: [Gitter](https://gitter.im/Yjs/community) | [Discord](https://discord.gg/T3nqMT6qbM) |
| * Benchmark Yjs vs. Automerge: |
| [https://github.com/dmonad/crdt-benchmarks](https://github.com/dmonad/crdt-benchmarks) |
| * Podcast [**"Yjs Deep Dive into real time collaborative editing solutions":**](https://www.tag1consulting.com/blog/deep-dive-real-time-collaborative-editing-solutions-tagteamtalk-001-0) |
| * Podcast [**"Google Docs-style editing in Gutenberg with the YJS framework":**](https://publishpress.com/blog/yjs/) |
| |
| :construction_worker_woman: If you are looking for professional support, please |
| consider supporting this project via a "support contract" on |
| [GitHub Sponsors](https://github.com/sponsors/dmonad). I will attend your issues |
| quicker and we can discuss questions and problems in regular video conferences. |
| Otherwise you can find help on our community [discussion board](https://discuss.yjs.dev). |
| |
| ## Sponsorship |
| |
| Please contribute to the project financially - especially if your company relies |
| on Yjs. [](https://github.com/sponsors/dmonad) |
| |
| ## Professional Support |
| |
| * [Support Contract with the Maintainer](https://github.com/sponsors/dmonad) - |
| By contributing financially to the open-source Yjs project, you can receive |
| professional support directly from the author. This includes the opportunity for |
| weekly video calls to discuss your specific challenges. |
| * [Synergy Codes](https://synergycodes.com/yjs-services/) - Specializing in |
| consulting and developing real-time collaborative editing solutions for visual |
| apps, Synergy Codes focuses on interactive diagrams, complex graphs, charts, and |
| various data visualization types. Their expertise empowers developers to build |
| engaging and interactive visual experiences leveraging the power of Yjs. See |
| their work in action at [Visual Collaboration |
| Showcase](https://yjs-diagram.synergy.codes/). |
| |
| ## Who is using Yjs |
| |
| * [AFFiNE](https://affine.pro/) A local-first, privacy-first, open source |
| knowledge base. :star2: |
| * [Huly](https://huly.io/) - Open Source All-in-One Project Management Platform :star2: |
| * [Cargo](https://cargo.site/) Site builder for designers and artists :star2: |
| * [Gitbook](https://gitbook.com) Knowledge management for technical teams :star2: |
| * [Evernote](https://evernote.com) Note-taking app :star2: |
| * [Lessonspace](https://thelessonspace.com) Enterprise platform for virtual |
| classrooms and online training :star2: |
| * [Ellipsus](ellipsus.com) - Collaborative writing app for storytelling etc. |
| Supports versioning, change attribution, and "blame". A solution for the whole |
| publishing process (also selling) :star: |
| * [Dynaboard](https://dynaboard.com/) Build web apps collaboratively. :star: |
| * [Relm](https://www.relm.us/) A collaborative gameworld for teamwork and |
| community. :star: |
| * [Room.sh](https://room.sh/) A meeting application with integrated |
| collaborative drawing, editing, and coding tools. :star: |
| * [Nimbus Note](https://nimbusweb.me/note.php) A note-taking app designed by |
| Nimbus Web. :star: |
| * [Pluxbox RadioManager](https://getradiomanager.com/) A web-based app to |
| collaboratively organize radio broadcasts. :star: |
| * [modyfi](https://www.modyfi.com) - Modyfi is the design platform built for |
| multidisciplinary designers. Design, generate, animate, and more — without |
| switching between apps. :star: |
| * [Sana](https://sanalabs.com/) A learning platform with collaborative text |
| editing powered by Yjs. |
| * [Serenity Notes](https://www.serenity.re/en/notes) End-to-end encrypted |
| collaborative notes app. |
| * [PRSM](https://prsm.uk/) Collaborative mind-mapping and system visualisation. |
| *[(source)](https://github.com/micrology/prsm)* |
| * [Alldone](https://alldone.app/) A next-gen project management and |
| collaboration platform. |
| * [Living Spec](https://livingspec.com/) A modern way for product teams to collaborate. |
| * [Slidebeamer](https://slidebeamer.com/) Presentation app. |
| * [BlockSurvey](https://blocksurvey.io) End-to-end encryption for your forms/surveys. |
| * [Skiff](https://skiff.org/) Private, decentralized workspace. |
| * [JupyterLab](https://jupyter.org/) Collaborative computational Notebooks |
| * [JupyterCad](https://jupytercad.readthedocs.io/en/latest/) Extension to |
| JupyterLab that enables collaborative editing of 3d FreeCAD Models. |
| * [JupyterGIS](https://github.com/geojupyter/jupytergis) Collaborative GIS |
| (Geographic Information System) editor in Jupyter |
| * [Hyperquery](https://hyperquery.ai/) A collaborative data workspace for |
| sharing analyses, documentation, spreadsheets, and dashboards. |
| * [Nosgestesclimat](https://nosgestesclimat.fr/groupe) The french carbon |
| footprint calculator has a group P2P mode based on yjs |
| * [oorja.io](https://oorja.io) Online meeting spaces extensible with |
| collaborative apps, end-to-end encrypted. |
| * [LegendKeeper](https://legendkeeper.com) Collaborative campaign planner and |
| worldbuilding app for tabletop RPGs. |
| * [IllumiDesk](https://illumidesk.com/) Build courses and content with A.I. |
| * [btw](https://www.btw.so) Open-source Medium alternative |
| * [AWS SageMaker](https://aws.amazon.com/sagemaker/) Tools for building Machine |
| Learning Models |
| * [linear](https://linear.app) Streamline issues, projects, and product roadmaps. |
| * [Arkiter](https://www.arkiter.com/) - Live interview software |
| * [Appflowy](https://www.appflowy.io/) - They use Yrs |
| * [Multi.app](https://multi.app) - Multiplayer app sharing: Point, draw and edit |
| in shared apps as if they're on your computer. They are using Yrs. |
| * [AppMaster](https://appmaster.io) A No-Code platform for creating |
| production-ready applications with source code generation. |
| * [Synthesia](https://www.synthesia.io) - Collaborative Video Editor |
| * [thinkdeli](https://thinkdeli.com) - A fast and simple notes app powered by AI |
| * [ourboard](https://github.com/raimohanska/ourboard) - A collaborative whiteboard |
| application |
| * [Ellie.ai](https://ellie.ai) - Data Product Design and Collaboration |
| * [GoPeer](https://gopeer.org/) - Collaborative tutoring |
| * [screen.garden](https://screen.garden) - Collaborative backend for PKM apps. |
| * [NextCloud](https://nextcloud.com/) - Content Collaboration Platform |
| * [keystatic](https://github.com/Thinkmill/keystatic) - git-based CMS |
| * [QDAcity](https://qdacity.com) - Collaborative qualitative data analysis platform |
| * [Kanbert](https://kanbert.com) - Project management software |
| * [Eclipse Theia](https://github.com/eclipse-theia/theia) - A cloud & desktop |
| IDE that runs in the browser. |
| * [ScienHub](https://scienhub.com) - Collaborative LaTeX editor in the browser. |
| * [Open Collaboration Tools](https://www.open-collab.tools/) - Collaborative |
| editing for your IDE or custom editor |
| * [Typst](https://typst.app/) - Compose, edit, and automate technical documents |
| * [Kedyou](https://kedyou.com/) - Digital workspaces for tutoring |
| * [Lightpage](https://lightpage.com/) - Personal living notebook |
| * [reearth-flow](https://github.com/reearth/reearth-flow) - |
| Collaboratively calculate and convert various data |
| |
| ## Table of Contents |
| |
| * [Overview](#overview) |
| * [Bindings](#bindings) |
| * [Providers](#providers) |
| * [Tooling](#tooling) |
| * [Ports](#ports) |
| * [Getting Started](#getting-started) |
| * [API](#api) |
| * [Shared Types](#shared-types) |
| * [Y.Doc](#ydoc) |
| * [Document Updates](#document-updates) |
| * [Relative Positions](#relative-positions) |
| * [Y.UndoManager](#yundomanager) |
| * [Yjs CRDT Algorithm](#yjs-crdt-algorithm) |
| * [License and Author](#license-and-author) |
| |
| ## Overview |
| |
| This repository contains a collection of shared types that can be observed for |
| changes and manipulated concurrently. Network functionality and two-way-bindings |
| are implemented in separate modules. |
| |
| ### Bindings |
| |
| | Name | Cursors | Binding | Demo | |
| |---|:-:|---|---| |
| | [ProseMirror](https://prosemirror.net/) | ✔ | [y-prosemirror](https://github.com/yjs/y-prosemirror) | [demo](https://demos.yjs.dev/prosemirror/prosemirror.html) | |
| | [Quill](https://quilljs.com/) | ✔ | [y-quill](https://github.com/yjs/y-quill) | [demo](https://demos.yjs.dev/quill/quill.html) | |
| | [CodeMirror](https://codemirror.net/) | ✔ | [y-codemirror](https://github.com/yjs/y-codemirror) | [demo](https://demos.yjs.dev/codemirror/codemirror.html) | |
| | [Monaco](https://microsoft.github.io/monaco-editor/) | ✔ | [y-monaco](https://github.com/yjs/y-monaco) | [demo](https://demos.yjs.dev/monaco/monaco.html) | |
| | [Ace](https://ace.c9.io/) | ✔ | [y-ace](https://github.com/bajrangCoder/y-ace) | | |
| | [Slate](https://github.com/ianstormtaylor/slate) | ✔ | [slate-yjs](https://github.com/bitphinix/slate-yjs) | [demo](https://bitphinix.github.io/slate-yjs-example) | |
| | [BlockSuite](https://github.com/toeverything/blocksuite) | ✔ | (native) | [demo](https://blocksuite-toeverything.vercel.app/?init) | |
| | [Lexical](https://lexical.dev/) | ✔ | (native) | [demo](https://lexical.dev/docs/collaboration/react#see-it-in-action) | |
| | [BlockNote](https://www.blocknotejs.org/docs/collaboration/real-time-collaboration) | ✔ | [y-prosemirror](https://github.com/yjs/y-prosemirror) | [demo](https://www.blocknotejs.org/docs/collaboration/real-time-collaboration) | |
| | [Tiptap](https://tiptap.dev/) | ✔ | [y-prosemirror](https://github.com/yjs/y-prosemirror) | [demo](https://template.tiptap.dev/preview/templates/simple) | |
| | [Milkdown](https://github.com/Milkdown/milkdown) | ✔ | [y-prosemirror](https://github.com/yjs/y-prosemirror) | [demo](https://milkdown.dev/playground) | |
| | [Superdoc](https://superdoc.dev/) | ✔ | (native) | [demo](https://superdoc.dev/) | |
| | [valtio](https://github.com/pmndrs/valtio) | | [valtio-yjs](https://github.com/dai-shi/valtio-yjs) | [demo](https://codesandbox.io/s/valtio-yjs-demo-ox3iy) | |
| | [immer](https://github.com/immerjs/immer) | | [immer-yjs](https://github.com/sep2/immer-yjs) | [demo](https://codesandbox.io/s/immer-yjs-demo-6e0znb) | |
| | React | | [react-yjs](https://github.com/nikgraf/react-yjs) | [demo](https://react-yjs-example.vercel.app/) | |
| | React / Vue / Svelte / MobX | | [SyncedStore](https://syncedstore.org) | [demo](https://syncedstore.org/docs/react) | |
| | [mobx-keystone](https://mobx-keystone.js.org/) | | [mobx-keystone-yjs](https://github.com/xaviergonz/mobx-keystone/tree/master/packages/mobx-keystone-yjs) | [demo](https://mobx-keystone.js.org/examples/yjs-binding) | |
| | [PSPDFKit](https://www.nutrient.io/) | | [yjs-pspdfkit](https://github.com/hoangqwe159/yjs-pspdfkit) | [demo](https://github.com/hoangqwe159/yjs-pspdfkit) | |
| | [Rows n'Columns](https://www.rowsncolumns.app/) | ✔ | [@rowsncolumns/y-spreadsheet](https://docs.rowsncolumns.app/collaboration/yjs-collaboration) | | |
| |
| ### Utilities |
| |
| Tools that extend the core functionality of Yjs. |
| |
| <dl> |
| <dt><a href="https://github.com/yjs/y-utility">y-utility</a></dt> |
| <dd> |
| Library with <code>YMultiDocUndoManager</code> (undo/redo across Yjs docs) and |
| <code>YKeyValue</code> (optimized key-value store). |
| </dd> |
| <dt> |
| <a href="https://github.com/Tulip-Writer/yjs-orderedtree"> yjs-orderedtree </a> 🌳 |
| </dt> |
| <dd> |
| Class for ordered trees via Y.Map. Handles <code>insert</code>, |
| <code>delete</code>, and <code>move</code> operations for folder-like |
| hierarchies. |
| </dd> |
| |
| </dl> |
| |
| ### Providers |
| |
| Setting up the communication between clients, managing awareness information, |
| and storing shared data for offline usage is quite a hassle. **Providers** |
| manage all that for you and are the perfect starting point for your |
| collaborative app. |
| |
| > This list of providers is incomplete. Please open PRs to add your providers to |
| > this list! |
| |
| #### Connection Providers |
| |
| <dl> |
| <dt><a href="https://github.com/yjs/y-websocket">y-websocket</a></dt> |
| <dd> |
| A module that contains a simple websocket backend and a websocket client that |
| connects to that backend. <a href="https://github.com/yjs/y-redis/"><b>y-redis</b></a>, |
| <b>y-sweet</b>, <b>ypy-websocket</b>, <b>yrs-warp</b> and <a href="https://tiptap.dev/docs/hocuspocus/introduction"> |
| <b>Hocuspocus</b></a> (see below) are alternative |
| backends to y-websocket. |
| </dd> |
| <dt><a href="https://github.com/yjs/y-webrtc">y-webrtc</a></dt> |
| <dd> |
| Propagates document updates peer-to-peer using WebRTC. The peers exchange |
| signaling data over signaling servers. Publicly available signaling servers |
| are available. Communication over the signaling servers can be encrypted by |
| providing a shared secret, keeping the connection information and the shared |
| document private. |
| </dd> |
| <dt><a href="https://github.com/liveblocks/liveblocks">@liveblocks/yjs </a> 🌟</dt> |
| <dd> |
| <a href="https://liveblocks.io/document/yjs">Liveblocks Yjs</a> provides a fully |
| hosted WebSocket infrastructure and persisted data store for Yjs |
| documents. No configuration or maintenance is required. It also features |
| Yjs webhook events, REST API to read and update Yjs documents, and a |
| browser DevTools extension. |
| </dd> |
| <dt><a href="https://github.com/drifting-in-space/y-sweet">y-sweet</a> ⭐</dt> |
| <dd> |
| A standalone yjs server with persistence to S3 or filesystem. They offer a |
| <a href="https://y-sweet.cloud">cloud service</a> as well. |
| </dd> |
| <dt><a href="https://github.com/ueberdosis/hocuspocus">Hocuspocus</a> ⭐</dt> |
| <dd> |
| A standalone extensible yjs server with sqlite persistence, webhooks, auth and more. |
| </dd> |
| <dt><a href="https://docs.superviz.com/collaboration/integrations/YJS/overview">@superviz/yjs</a></dt> |
| <dd> |
| SuperViz Yjs Provider comes with a secure, scalable real-time infrastructure |
| for Yjs documents, fully compatible with a set of real-time |
| collaboration components offered by SuperViz. This solution ensures |
| synchronization, offline editing, and real-time updates, enabling |
| multiple users to collaborate effectively within shared workspaces. |
| </dd> |
| <dt><a href="https://docs.partykit.io/reference/y-partykit-api/">PartyKit</a></dt> |
| <dd> |
| Cloud service for building multiplayer apps. |
| </dd> |
| </dd> |
| <dt><a href="https://github.com/pluv-io/pluv">@pluv/crdt-yjs</a></dt> |
| <dd> |
| Use <a href="https://pluv.io/docs/storage/using-yjs">pluv.io</a> as a |
| full-featured backend for Yjs. pluv.io can either be be used on its |
| fully-managed WebSocket infrastructure, or self-hosted on Cloudflare Workers |
| and Node.js runtimes. Offers a typesafe API with authentication, webhooks, |
| rooms, and more. |
| <dt><a href="https://github.com/marcopolo/y-libp2p">y-libp2p</a></dt> |
| <dd> |
| Uses <a href="https://libp2p.io/">libp2p</a> to propagate updates via |
| <a href="https://github.com/libp2p/specs/tree/master/pubsub/gossipsub">GossipSub</a>. |
| Also includes a peer-sync mechanism to catch up on missed updates. |
| </dd> |
| <dt><a href="https://github.com/yjs/y-dat">y-dat</a></dt> |
| <dd> |
| [WIP] Write document updates efficiently to the dat network using |
| <a href="https://github.com/kappa-db/multifeed">multifeed</a>. Each client has |
| an append-only log of CRDT local updates (hypercore). Multifeed manages and sync |
| hypercores and y-dat listens to changes and applies them to the Yjs document. |
| </dd> |
| <dt><a href="https://github.com/yousefED/matrix-crdt">Matrix-CRDT</a></dt> |
| <dd> |
| Use <a href="https://www.matrix.org">Matrix</a> as an off-the-shelf backend for |
| Yjs by using the <a href="https://github.com/yousefED/matrix-crdt">MatrixProvider</a>. |
| Use Matrix as transport and storage of Yjs updates, so you can focus building |
| your client app and Matrix can provide powerful features like Authentication, |
| Authorization, Federation, hosting (self-hosting or SaaS) and even End-to-End |
| Encryption (E2EE). |
| </dd> |
| <dt><a href="https://github.com/y-crdt/yrb-actioncable">yrb-actioncable</a></dt> |
| <dd> |
| An ActionCable companion for Yjs clients. There is a fitting |
| <a href="https://github.com/y-crdt/yrb-redis">redis extension</a> as well. |
| </dd> |
| <dt><a href="https://github.com/y-crdt/ypy-websocket">ypy-websocket</a></dt> |
| <dd> |
| Websocket backend, written in Python. |
| </dd> |
| <dt><a href="https://tinybase.org/">Tinybase</a></dt> |
| <dd> |
| The reactive data store for local-first apps. They support multiple CRDTs and |
| different network technologies. |
| </dd> |
| <dt><a href="https://codeberg.org/webxdc/y-webxdc">y-webxdc</a></dt> |
| <dd> |
| Provider for sharing data in <a href="https://webxdc.org">webxdc chat apps</a>. |
| </dd> |
| <dt><a href="https://www.secsync.com/">secsync</a></dt> |
| <dd> |
| An architecture to relay end-to-end encrypted CRDTs over a central service. |
| </dd> |
| <dt><a href="https://www.npmjs.com/package/@electric-sql/y-electric">y-electric</a></dt> |
| <dd> |
| Sync Yjs over <a href="https://electric-sql.com/">ElectricSQL</a>. |
| </dd> |
| <dt><a href="https://github.com/TimoWilhelm/yjs-cf-ws-provider">yjs-cf-ws-provider</a></dt> |
| <dd> |
| Cloudflare provider for Yjs based on durable objects. |
| </dd> |
| <dt><a href="https://github.com/yousefED/nostr-crdt">nostr-crdt</a></dt> |
| <dd> |
| Sync Yjs over <a href="https://github.com/nostr-protocol/">nostr</a>. |
| </dd> |
| </dl> |
| |
| #### Persistence Providers |
| |
| <dl> |
| <dt><a href="https://github.com/yjs/y-indexeddb">y-indexeddb</a></dt> |
| <dd> |
| Efficiently persists document updates to the browsers indexeddb database. |
| The document is immediately available and only diffs need to be synced through the |
| network provider. |
| </dd> |
| <dt><a href="https://github.com/MaxNoetzold/y-mongodb-provider">y-mongodb-provider</a></dt> |
| <dd> |
| Adds persistent storage to a server with MongoDB. Can be used with the |
| y-websocket provider. |
| </dd> |
| <dt><a href="https://github.com/podraven/y-fire">y-fire</a></dt> |
| <dd> |
| A database and connection provider for Yjs based on Firestore. |
| </dd> |
| <dt><a href="https://github.com/malte-j/y-op-sqlite">y-op-sqlite</a></dt> |
| <dd> |
| Persist YJS updates in your React Native app using |
| <a href="https://github.com/OP-Engineering/op-sqlite">op-sqlite</a> |
| , the fastest SQLite library for React Native. |
| </dd> |
| <dt><a href="https://github.com/MaxNoetzold/y-postgresql">y-postgresql</a></dt> |
| <dd> |
| Provides persistent storage for a web server using PostgreSQL and |
| is easily compatible with y-websocket. |
| </dd> |
| <dt><a href="https://github.com/kapv89/k_yrs_go">k_yrs_go</a></dt> |
| <dd> |
| Golang database server for YJS CRDT using Postgres + Redis |
| </dd> |
| <dt><a href="https://github.com/malte-j/y-op-sqlite">y-op-sqlite</a></dt> |
| <dd> |
| Yjs persistence provider for op-sqlite |
| </dd> |
| |
| </dl> |
| |
| ### Tooling |
| |
| * [y-sweet debugger](https://y-sweet.cloud/advanced/debugger) |
| * [liveblocks devtools](https://liveblocks.io/devtools) |
| * [Yjs inspector](https://inspector.yjs.dev) |
| |
| ### Ports |
| |
| There are several Yjs-compatible ports to other programming languages. |
| |
| * [y-octo](https://github.com/toeverything/y-octo) - Rust implementation by |
| [AFFiNE](https://affine.pro) |
| * [y-crdt](https://github.com/y-crdt/y-crdt) - Rust implementation with multiple |
| language bindings to other languages |
| * [yrs](https://github.com/y-crdt/y-crdt/tree/main/yrs) - Rust interface |
| * [ypy](https://github.com/y-crdt/ypy) - Python binding |
| * [yrb](https://github.com/y-crdt/yrb) - Ruby binding |
| * [yswift](https://github.com/y-crdt/yswift) - Swift binding |
| * [yffi](https://github.com/y-crdt/y-crdt/tree/main/yffi) - C-FFI |
| * [ywasm](https://github.com/y-crdt/y-crdt/tree/main/ywasm) - WASM binding |
| * [y_ex](https://github.com/satoren/y_ex) - Elixir bindings |
| * [ycs](https://github.com/yjs/ycs) - .Net compatible C# implementation. |
| |
| ## Getting Started |
| |
| Install Yjs and a provider with your favorite package manager: |
| |
| ```sh |
| npm i yjs y-websocket |
| ``` |
| |
| Start the y-websocket server: |
| |
| ```sh |
| PORT=1234 node ./node_modules/y-websocket/bin/server.cjs |
| ``` |
| |
| ### Example: Observe types |
| |
| ```js |
| import * as Y from 'yjs'; |
| |
| const doc = new Y.Doc(); |
| const yarray = doc.getArray('my-array') |
| yarray.observe(event => { |
| console.log('yarray was modified') |
| }) |
| // every time a local or remote client modifies yarray, the observer is called |
| yarray.insert(0, ['val']) // => "yarray was modified" |
| ``` |
| |
| ### Example: Nest types |
| |
| Remember, shared types are just plain old data types. The only limitation is |
| that a shared type must exist only once in the shared document. |
| |
| ```js |
| const ymap = doc.getMap('map') |
| const foodArray = new Y.Array() |
| foodArray.insert(0, ['apple', 'banana']) |
| ymap.set('food', foodArray) |
| ymap.get('food') === foodArray // => true |
| ymap.set('fruit', foodArray) // => Error! foodArray is already defined |
| ``` |
| |
| Now you understand how types are defined on a shared document. Next you can jump |
| to the [demo repository](https://github.com/yjs/yjs-demos) or continue reading |
| the API docs. |
| |
| ### Example: Using and combining providers |
| |
| Any of the Yjs providers can be combined with each other. So you can sync data |
| over different network technologies. |
| |
| In most cases you want to use a network provider (like y-websocket or y-webrtc) |
| in combination with a persistence provider (y-indexeddb in the browser). |
| Persistence allows you to load the document faster and to persist data that is |
| created while offline. |
| |
| For the sake of this demo we combine two different network providers with a |
| persistence provider. |
| |
| ```js |
| import * as Y from 'yjs' |
| import { WebrtcProvider } from 'y-webrtc' |
| import { WebsocketProvider } from 'y-websocket' |
| import { IndexeddbPersistence } from 'y-indexeddb' |
| |
| const ydoc = new Y.Doc() |
| |
| // this allows you to instantly get the (cached) documents data |
| const indexeddbProvider = new IndexeddbPersistence('count-demo', ydoc) |
| indexeddbProvider.whenSynced.then(() => { |
| console.log('loaded data from indexed db') |
| }) |
| |
| // Sync clients with the y-webrtc provider. |
| const webrtcProvider = new WebrtcProvider('count-demo', ydoc) |
| |
| // Sync clients with the y-websocket provider |
| const websocketProvider = new WebsocketProvider( |
| 'wss://demos.yjs.dev', 'count-demo', ydoc |
| ) |
| |
| // array of numbers which produce a sum |
| const yarray = ydoc.getArray('count') |
| |
| // observe changes of the sum |
| yarray.observe(event => { |
| // print updates when the data changes |
| console.log('new sum: ' + yarray.toArray().reduce((a,b) => a + b)) |
| }) |
| |
| // add 1 to the sum |
| yarray.push([1]) // => "new sum: 1" |
| ``` |
| |
| ## API |
| |
| ```js |
| import * as Y from 'yjs' |
| ``` |
| |
| ### Shared Types |
| |
| <details> |
| <summary><b>Y.Array</b></summary> |
| <br> |
| <p> |
| A shareable Array-like type that supports efficient insert/delete of elements |
| at any position. Internally it uses a linked list of Arrays that is split when |
| necessary. |
| </p> |
| <pre>const yarray = new Y.Array()</pre> |
| <dl> |
| <b><code> |
| Y.Array.from(Array<object|boolean|Array|string|number|null|Uint8Array|Y.Type>): |
| Y.Array |
| </code></b> |
| <dd>An alternative factory function to create a Y.Array based on existing content.</dd> |
| <b><code>parent:Y.AbstractType|null</code></b> |
| <dd></dd> |
| <b><code>insert(index:number, content:Array<object|boolean|Array|string|number|null|Uint8Array|Y.Type>)</code></b> |
| <dd> |
| Insert content at <var>index</var>. Note that content is an array of elements. |
| I.e. <code>array.insert(0, [1])</code> splices the list and inserts 1 at |
| position 0. |
| </dd> |
| <b><code>push(Array<Object|boolean|Array|string|number|null|Uint8Array|Y.Type>)</code></b> |
| <dd></dd> |
| <b><code>unshift(Array<Object|boolean|Array|string|number|null|Uint8Array|Y.Type>)</code></b> |
| <dd></dd> |
| <b><code>delete(index:number, length:number)</code></b> |
| <dd></dd> |
| <b><code>get(index:number)</code></b> |
| <dd></dd> |
| <b><code>slice(start:number, end:number):Array<Object|boolean|Array|string|number|null|Uint8Array|Y.Type></code></b> |
| <dd>Retrieve a range of content</dd> |
| <b><code>length:number</code></b> |
| <dd></dd> |
| <b> |
| <code> |
| forEach(function(value:object|boolean|Array|string|number|null|Uint8Array|Y.Type, |
| index:number, array: Y.Array)) |
| </code> |
| </b> |
| <dd></dd> |
| <b><code>map(function(T, number, YArray):M):Array<M></code></b> |
| <dd></dd> |
| <b><code>clone(): Y.Array</code></b> |
| <dd> |
| Clone all values into a fresh Y.Array instance. The returned type can be |
| included into the Yjs document. |
| </dd> |
| <b><code>toArray():Array<object|boolean|Array|string|number|null|Uint8Array|Y.Type></code></b> |
| <dd>Copies the content of this YArray to a new Array.</dd> |
| <b><code>toJSON():Array<Object|boolean|Array|string|number|null></code></b> |
| <dd> |
| Copies the content of this YArray to a new Array. It transforms all child types |
| to JSON using their <code>toJSON</code> method. |
| </dd> |
| <b><code>[Symbol.Iterator]</code></b> |
| <dd> |
| Returns an YArray Iterator that contains the values for each index in the array. |
| <pre>for (let value of yarray) { .. }</pre> |
| </dd> |
| <b><code>observe(function(YArrayEvent, Transaction):void)</code></b> |
| <dd> |
| Adds an event listener to this type that will be called synchronously every time |
| this type is modified. In the case this type is modified in the event listener, |
| the event listener will be called again after the current event listener returns. |
| </dd> |
| <b><code>unobserve(function(YArrayEvent, Transaction):void)</code></b> |
| <dd> |
| Removes an <code>observe</code> event listener from this type. |
| </dd> |
| <b><code>observeDeep(function(Array<YEvent>, Transaction):void)</code></b> |
| <dd> |
| Adds an event listener to this type that will be called synchronously every time |
| this type or any of its children is modified. In the case this type is modified |
| in the event listener, the event listener will be called again after the current |
| event listener returns. The event listener receives all Events created by itself |
| or any of its children. |
| </dd> |
| <b><code>unobserveDeep(function(Array<YEvent>, Transaction):void)</code></b> |
| <dd> |
| Removes an <code>observeDeep</code> event listener from this type. |
| </dd> |
| </dl> |
| </details> |
| <details> |
| <summary><b>Y.Map</b></summary> |
| <br> |
| <p> |
| A shareable Map type. |
| </p> |
| <pre><code>const ymap = new Y.Map()</code></pre> |
| <dl> |
| <b><code>parent:Y.AbstractType|null</code></b> |
| <dd></dd> |
| <b><code>size: number</code></b> |
| <dd>Total number of key/value pairs.</dd> |
| <b><code>get(key:string):object|boolean|string|number|null|Uint8Array|Y.Type</code></b> |
| <dd></dd> |
| <b><code>set(key:string, value:object|boolean|string|number|null|Uint8Array|Y.Type)</code></b> |
| <dd></dd> |
| <b><code>delete(key:string)</code></b> |
| <dd></dd> |
| <b><code>has(key:string):boolean</code></b> |
| <dd></dd> |
| <b><code>clear()</code></b> |
| <dd>Removes all elements from this YMap.</dd> |
| <b><code>clone():Y.Map</code></b> |
| <dd>Clone this type into a fresh Yjs type.</dd> |
| <b><code>toJSON():Object<string, Object|boolean|Array|string|number|null|Uint8Array></code></b> |
| <dd> |
| Copies the <code>[key,value]</code> pairs of this YMap to a new Object.It |
| transforms all child types to JSON using their <code>toJSON</code> method. |
| </dd> |
| <b><code>forEach(function(value:object|boolean|Array|string|number|null|Uint8Array|Y.Type, |
| key:string, map: Y.Map))</code></b> |
| <dd> |
| Execute the provided function once for every key-value pair. |
| </dd> |
| <b><code>[Symbol.Iterator]</code></b> |
| <dd> |
| Returns an Iterator of <code>[key, value]</code> pairs. |
| <pre>for (let [key, value] of ymap) { .. }</pre> |
| </dd> |
| <b><code>entries()</code></b> |
| <dd> |
| Returns an Iterator of <code>[key, value]</code> pairs. |
| </dd> |
| <b><code>values()</code></b> |
| <dd> |
| Returns an Iterator of all values. |
| </dd> |
| <b><code>keys()</code></b> |
| <dd> |
| Returns an Iterator of all keys. |
| </dd> |
| <b><code>observe(function(YMapEvent, Transaction):void)</code></b> |
| <dd> |
| Adds an event listener to this type that will be called synchronously every time |
| this type is modified. In the case this type is modified in the event listener, |
| the event listener will be called again after the current event listener returns. |
| </dd> |
| <b><code>unobserve(function(YMapEvent, Transaction):void)</code></b> |
| <dd> |
| Removes an <code>observe</code> event listener from this type. |
| </dd> |
| <b><code>observeDeep(function(Array<YEvent>, Transaction):void)</code></b> |
| <dd> |
| Adds an event listener to this type that will be called synchronously every time |
| this type or any of its children is modified. In the case this type is modified |
| in the event listener, the event listener will be called again after the current |
| event listener returns. The event listener receives all Events created by itself |
| or any of its children. |
| </dd> |
| <b><code>unobserveDeep(function(Array<YEvent>, Transaction):void)</code></b> |
| <dd> |
| Removes an <code>observeDeep</code> event listener from this type. |
| </dd> |
| </dl> |
| </details> |
| |
| <details> |
| <summary><b>Y.Text</b></summary> |
| <br> |
| <p> |
| A shareable type that is optimized for shared editing on text. It allows to |
| assign properties to ranges in the text. This makes it possible to implement |
| rich-text bindings to this type. |
| </p> |
| <p> |
| This type can also be transformed to the |
| <a href="https://quilljs.com/docs/delta">delta format</a>. Similarly the |
| YTextEvents compute changes as deltas. |
| </p> |
| <pre>const ytext = new Y.Text()</pre> |
| <dl> |
| <b><code>parent:Y.AbstractType|null</code></b> |
| <dd></dd> |
| <b><code>insert(index:number, content:string, [formattingAttributes:Object<string,string>])</code></b> |
| <dd> |
| Insert a string at <var>index</var> and assign formatting attributes to it. |
| <pre>ytext.insert(0, 'bold text', { bold: true })</pre> |
| </dd> |
| <b><code>delete(index:number, length:number)</code></b> |
| <dd></dd> |
| <b><code>format(index:number, length:number, formattingAttributes:Object<string,string>)</code></b> |
| <dd>Assign formatting attributes to a range in the text</dd> |
| <b><code>applyDelta(delta: Delta, opts:Object<string,any>)</code></b> |
| <dd> |
| See <a href="https://quilljs.com/docs/delta/">Quill Delta</a> |
| Can set options for preventing remove ending newLines, default is true. |
| <pre>ytext.applyDelta(delta, { sanitize: false })</pre> |
| </dd> |
| <b><code>length:number</code></b> |
| <dd></dd> |
| <b><code>toString():string</code></b> |
| <dd>Transforms this type, without formatting options, into a string.</dd> |
| <b><code>toJSON():string</code></b> |
| <dd>See <code>toString</code></dd> |
| <b><code>toDelta():Delta</code></b> |
| <dd> |
| Transforms this type to a <a href="https://quilljs.com/docs/delta/">Quill Delta</a> |
| </dd> |
| <b><code>observe(function(YTextEvent, Transaction):void)</code></b> |
| <dd> |
| Adds an event listener to this type that will be called synchronously every time |
| this type is modified. In the case this type is modified in the event listener, |
| the event listener will be called again after the current event listener returns. |
| </dd> |
| <b><code>unobserve(function(YTextEvent, Transaction):void)</code></b> |
| <dd> |
| Removes an <code>observe</code> event listener from this type. |
| </dd> |
| <b><code>observeDeep(function(Array<YEvent>, Transaction):void)</code></b> |
| <dd> |
| Adds an event listener to this type that will be called synchronously every time |
| this type or any of its children is modified. In the case this type is modified |
| in the event listener, the event listener will be called again after the current |
| event listener returns. The event listener receives all Events created by itself |
| or any of its children. |
| </dd> |
| <b><code>unobserveDeep(function(Array<YEvent>, Transaction):void)</code></b> |
| <dd> |
| Removes an <code>observeDeep</code> event listener from this type. |
| </dd> |
| </dl> |
| </details> |
| |
| <details> |
| <summary><b>Y.XmlFragment</b></summary> |
| <br> |
| <p> |
| A container that holds an Array of Y.XmlElements. |
| </p> |
| <pre><code>const yxml = new Y.XmlFragment()</code></pre> |
| <dl> |
| <b><code>parent:Y.AbstractType|null</code></b> |
| <dd></dd> |
| <b><code>firstChild:Y.XmlElement|Y.XmlText|null</code></b> |
| <dd></dd> |
| <b><code>insert(index:number, content:Array<Y.XmlElement|Y.XmlText>)</code></b> |
| <dd></dd> |
| <b><code>delete(index:number, length:number)</code></b> |
| <dd></dd> |
| <b><code>get(index:number)</code></b> |
| <dd></dd> |
| <b><code>slice(start:number, end:number):Array<Y.XmlElement|Y.XmlText></code></b> |
| <dd>Retrieve a range of content</dd> |
| <b><code>length:number</code></b> |
| <dd></dd> |
| <b><code>clone():Y.XmlFragment</code></b> |
| <dd>Clone this type into a fresh Yjs type.</dd> |
| <b><code>toArray():Array<Y.XmlElement|Y.XmlText></code></b> |
| <dd>Copies the children to a new Array.</dd> |
| <b><code>toDOM():DocumentFragment</code></b> |
| <dd>Transforms this type and all children to new DOM elements.</dd> |
| <b><code>toString():string</code></b> |
| <dd>Get the XML serialization of all descendants.</dd> |
| <b><code>toJSON():string</code></b> |
| <dd>See <code>toString</code>.</dd> |
| <b><code>createTreeWalker(filter: function(AbstractType<any>):boolean):Iterable</code></b> |
| <dd>Create an Iterable that walks through the children.</dd> |
| <b><code>observe(function(YXmlEvent, Transaction):void)</code></b> |
| <dd> |
| Adds an event listener to this type that will be called synchronously every time |
| this type is modified. In the case this type is modified in the event listener, |
| the event listener will be called again after the current event listener returns. |
| </dd> |
| <b><code>unobserve(function(YXmlEvent, Transaction):void)</code></b> |
| <dd> |
| Removes an <code>observe</code> event listener from this type. |
| </dd> |
| <b><code>observeDeep(function(Array<YEvent>, Transaction):void)</code></b> |
| <dd> |
| Adds an event listener to this type that will be called synchronously every time |
| this type or any of its children is modified. In the case this type is modified |
| in the event listener, the event listener will be called again after the current |
| event listener returns. The event listener receives all Events created by itself |
| or any of its children. |
| </dd> |
| <b><code>unobserveDeep(function(Array<YEvent>, Transaction):void)</code></b> |
| <dd> |
| Removes an <code>observeDeep</code> event listener from this type. |
| </dd> |
| </dl> |
| </details> |
| |
| <details> |
| <summary><b>Y.XmlElement</b></summary> |
| <br> |
| <p> |
| A shareable type that represents an XML Element. It has a <code>nodeName</code>, |
| attributes, and a list of children. But it makes no effort to validate its |
| content and be actually XML compliant. |
| </p> |
| <pre><code>const yxml = new Y.XmlElement()</code></pre> |
| <dl> |
| <b><code>parent:Y.AbstractType|null</code></b> |
| <dd></dd> |
| <b><code>firstChild:Y.XmlElement|Y.XmlText|null</code></b> |
| <dd></dd> |
| <b><code>nextSibling:Y.XmlElement|Y.XmlText|null</code></b> |
| <dd></dd> |
| <b><code>prevSibling:Y.XmlElement|Y.XmlText|null</code></b> |
| <dd></dd> |
| <b><code>insert(index:number, content:Array<Y.XmlElement|Y.XmlText>)</code></b> |
| <dd></dd> |
| <b><code>delete(index:number, length:number)</code></b> |
| <dd></dd> |
| <b><code>get(index:number)</code></b> |
| <dd></dd> |
| <b><code>length:number</code></b> |
| <dd></dd> |
| <b><code>setAttribute(attributeName:string, attributeValue:string)</code></b> |
| <dd></dd> |
| <b><code>removeAttribute(attributeName:string)</code></b> |
| <dd></dd> |
| <b><code>getAttribute(attributeName:string):string</code></b> |
| <dd></dd> |
| <b><code>getAttributes():Object<string,string></code></b> |
| <dd></dd> |
| <b><code>get(i:number):Y.XmlElement|Y.XmlText</code></b> |
| <dd>Retrieve the i-th element.</dd> |
| <b><code>slice(start:number, end:number):Array<Y.XmlElement|Y.XmlText></code></b> |
| <dd>Retrieve a range of content</dd> |
| <b><code>clone():Y.XmlElement</code></b> |
| <dd>Clone this type into a fresh Yjs type.</dd> |
| <b><code>toArray():Array<Y.XmlElement|Y.XmlText></code></b> |
| <dd>Copies the children to a new Array.</dd> |
| <b><code>toDOM():Element</code></b> |
| <dd>Transforms this type and all children to a new DOM element.</dd> |
| <b><code>toString():string</code></b> |
| <dd>Get the XML serialization of all descendants.</dd> |
| <b><code>toJSON():string</code></b> |
| <dd>See <code>toString</code>.</dd> |
| <b><code>observe(function(YXmlEvent, Transaction):void)</code></b> |
| <dd> |
| Adds an event listener to this type that will be called synchronously every |
| time this type is modified. In the case this type is modified in the event |
| listener, the event listener will be called again after the current event |
| listener returns. |
| </dd> |
| <b><code>unobserve(function(YXmlEvent, Transaction):void)</code></b> |
| <dd> |
| Removes an <code>observe</code> event listener from this type. |
| </dd> |
| <b><code>observeDeep(function(Array<YEvent>, Transaction):void)</code></b> |
| <dd> |
| Adds an event listener to this type that will be called synchronously every time |
| this type or any of its children is modified. In the case this type is modified |
| in the event listener, the event listener will be called again after the current |
| event listener returns. The event listener receives all Events created by itself |
| or any of its children. |
| </dd> |
| <b><code>unobserveDeep(function(Array<YEvent>, Transaction):void)</code></b> |
| <dd> |
| Removes an <code>observeDeep</code> event listener from this type. |
| </dd> |
| </dl> |
| </details> |
| |
| ### Y.Doc |
| |
| ```js |
| const doc = new Y.Doc() |
| ``` |
| |
| <dl> |
| <b><code>clientID</code></b> |
| <dd>A unique id that identifies this client. (readonly)</dd> |
| <b><code>gc</code></b> |
| <dd> |
| Whether garbage collection is enabled on this doc instance. Set `doc.gc = false` |
| in order to disable gc and be able to restore old content. See https://github.com/yjs/yjs#yjs-crdt-algorithm |
| for more information about gc in Yjs. |
| </dd> |
| <b><code>transact(function(Transaction):void [, origin:any])</code></b> |
| <dd> |
| Every change on the shared document happens in a transaction. Observer calls and |
| the <code>update</code> event are called after each transaction. You should |
| <i>bundle</i> changes into a single transaction to reduce the amount of event |
| calls. I.e. <code>doc.transact(() => { yarray.insert(..); ymap.set(..) })</code> |
| triggers a single change event. <br>You can specify an optional <code>origin</code> |
| parameter that is stored on <code>transaction.origin</code> and |
| <code>on('update', (update, origin) => ..)</code>. |
| </dd> |
| <b><code>toJSON():any</code></b> |
| <dd> |
| Deprecated: It is recommended to call toJSON directly on the shared types. |
| Converts the entire document into a js object, recursively traversing each yjs |
| type. Doesn't log types that have not been defined (using |
| <code>ydoc.getType(..)</code>). |
| </dd> |
| <b><code>get(string, Y.[TypeClass]):[Type]</code></b> |
| <dd>Define a shared type.</dd> |
| <b><code>getArray(string):Y.Array</code></b> |
| <dd>Define a shared Y.Array type. Is equivalent to <code>y.get(string, Y.Array)</code>.</dd> |
| <b><code>getMap(string):Y.Map</code></b> |
| <dd>Define a shared Y.Map type. Is equivalent to <code>y.get(string, Y.Map)</code>.</dd> |
| <b><code>getText(string):Y.Text</code></b> |
| <dd>Define a shared Y.Text type. Is equivalent to <code>y.get(string, Y.Text)</code>.</dd> |
| <b><code>getXmlElement(string, string):Y.XmlElement</code></b> |
| <dd>Define a shared Y.XmlElement type. Is equivalent to <code>y.get(string, Y.XmlElement)</code>.</dd> |
| <b><code>getXmlFragment(string):Y.XmlFragment</code></b> |
| <dd>Define a shared Y.XmlFragment type. Is equivalent to <code>y.get(string, Y.XmlFragment)</code>.</dd> |
| <b><code>on(string, function)</code></b> |
| <dd>Register an event listener on the shared type</dd> |
| <b><code>off(string, function)</code></b> |
| <dd>Unregister an event listener from the shared type</dd> |
| </dl> |
| |
| #### Y.Doc Events |
| |
| <dl> |
| <b><code>on('update', function(updateMessage:Uint8Array, origin:any, Y.Doc):void)</code></b> |
| <dd> |
| Listen to document updates. Document updates must be transmitted to all other |
| peers. You can apply document updates in any order and multiple times. Use `updateV2` |
| to receive V2 events. |
| </dd> |
| <b><code>on('beforeTransaction', function(Y.Transaction, Y.Doc):void)</code></b> |
| <dd>Emitted before each transaction.</dd> |
| <b><code>on('afterTransaction', function(Y.Transaction, Y.Doc):void)</code></b> |
| <dd>Emitted after each transaction.</dd> |
| <b><code>on('beforeAllTransactions', function(Y.Doc):void)</code></b> |
| <dd> |
| Transactions can be nested (e.g. when an event within a transaction calls another |
| transaction). Emitted before the first transaction. |
| </dd> |
| <b><code>on('afterAllTransactions', function(Y.Doc, Array<Y.Transaction>):void)</code></b> |
| <dd>Emitted after the last transaction is cleaned up.</dd> |
| </dl> |
| |
| ### Document Updates |
| |
| Changes on the shared document are encoded into *document updates*. Document |
| updates are *commutative* and *idempotent*. This means that they can be applied |
| in any order and multiple times. |
| |
| #### Example: Listen to update events and apply them on remote client |
| |
| ```js |
| const doc1 = new Y.Doc() |
| const doc2 = new Y.Doc() |
| |
| doc1.on('update', update => { |
| Y.applyUpdate(doc2, update) |
| }) |
| |
| doc2.on('update', update => { |
| Y.applyUpdate(doc1, update) |
| }) |
| |
| // All changes are also applied to the other document |
| doc1.getArray('myarray').insert(0, ['Hello doc2, you got this?']) |
| doc2.getArray('myarray').get(0) // => 'Hello doc2, you got this?' |
| ``` |
| |
| Yjs internally maintains a [state vector](#state-vector) that denotes the next |
| expected clock from each client. In a different interpretation it holds the |
| number of structs created by each client. When two clients sync, you can either |
| exchange the complete document structure or only the differences by sending the |
| state vector to compute the differences. |
| |
| #### Example: Sync two clients by exchanging the complete document structure |
| |
| ```js |
| const state1 = Y.encodeStateAsUpdate(ydoc1) |
| const state2 = Y.encodeStateAsUpdate(ydoc2) |
| Y.applyUpdate(ydoc1, state2) |
| Y.applyUpdate(ydoc2, state1) |
| ``` |
| |
| #### Example: Sync two clients by computing the differences |
| |
| This example shows how to sync two clients with the minimal amount of exchanged |
| data by computing only the differences using the state vector of the remote |
| client. Syncing clients using the state vector requires another roundtrip, but |
| can save a lot of bandwidth. |
| |
| ```js |
| const stateVector1 = Y.encodeStateVector(ydoc1) |
| const stateVector2 = Y.encodeStateVector(ydoc2) |
| const diff1 = Y.encodeStateAsUpdate(ydoc1, stateVector2) |
| const diff2 = Y.encodeStateAsUpdate(ydoc2, stateVector1) |
| Y.applyUpdate(ydoc1, diff2) |
| Y.applyUpdate(ydoc2, diff1) |
| ``` |
| |
| #### Example: Syncing clients without loading the Y.Doc |
| |
| It is possible to sync clients and compute delta updates without loading the Yjs |
| document to memory. Yjs exposes an API to compute the differences directly on the |
| binary document updates. |
| |
| ```js |
| // encode the current state as a binary buffer |
| let currentState1 = Y.encodeStateAsUpdate(ydoc1) |
| let currentState2 = Y.encodeStateAsUpdate(ydoc2) |
| // now we can continue syncing clients using state vectors without using the Y.Doc |
| ydoc1.destroy() |
| ydoc2.destroy() |
| |
| const stateVector1 = Y.encodeStateVectorFromUpdate(currentState1) |
| const stateVector2 = Y.encodeStateVectorFromUpdate(currentState2) |
| const diff1 = Y.diffUpdate(currentState1, stateVector2) |
| const diff2 = Y.diffUpdate(currentState2, stateVector1) |
| |
| // sync clients |
| currentState1 = Y.mergeUpdates([currentState1, diff2]) |
| currentState2 = Y.mergeUpdates([currentState2, diff1]) |
| ``` |
| |
| #### Obfuscating Updates |
| |
| If one of your users runs into a weird bug (e.g. the rich-text editor throws |
| error messages), then you don't have to request the full document from your |
| user. Instead, they can obfuscate the document (i.e. replace the content with |
| meaningless generated content) before sending it to you. Note that someone might |
| still deduce the type of content by looking at the general structure of the |
| document. But this is much better than requesting the original document. |
| |
| Obfuscated updates contain all the CRDT-related data that is required for |
| merging. So it is safe to merge obfuscated updates. |
| |
| ```javascript |
| const ydoc = new Y.Doc() |
| // perform some changes.. |
| ydoc.getText().insert(0, 'hello world') |
| const update = Y.encodeStateAsUpdate(ydoc) |
| // the below update contains scrambled data |
| const obfuscatedUpdate = Y.obfuscateUpdate(update) |
| const ydoc2 = new Y.Doc() |
| Y.applyUpdate(ydoc2, obfuscatedUpdate) |
| ydoc2.getText().toString() // => "00000000000" |
| ``` |
| |
| #### Using V2 update format |
| |
| Yjs implements two update formats. By default you are using the V1 update format. |
| You can opt-in into the V2 update format which provides much better compression. |
| It is not yet used by all providers. However, you can already use it if |
| you are building your own provider. All below functions are available with the |
| suffix "V2". E.g. `Y.applyUpdate` ⇒ `Y.applyUpdateV2`. Also when listening to updates |
| you need to specifically need listen for V2 events e.g. `yDoc.on('updateV2', …)`. |
| We also support conversion functions between both formats: |
| `Y.convertUpdateFormatV1ToV2` & `Y.convertUpdateFormatV2ToV1`. |
| |
| #### Update API |
| |
| <dl> |
| <b><code>Y.applyUpdate(Y.Doc, update:Uint8Array, [transactionOrigin:any])</code></b> |
| <dd> |
| Apply a document update on the shared document. Optionally you can specify |
| <code>transactionOrigin</code> that will be stored on |
| <code>transaction.origin</code> |
| and <code>ydoc.on('update', (update, origin) => ..)</code>. |
| </dd> |
| <b><code>Y.encodeStateAsUpdate(Y.Doc, [encodedTargetStateVector:Uint8Array]):Uint8Array</code></b> |
| <dd> |
| Encode the document state as a single update message that can be applied on the |
| remote document. Optionally specify the target state vector to only write the |
| differences to the update message. |
| </dd> |
| <b><code>Y.encodeStateVector(Y.Doc):Uint8Array</code></b> |
| <dd>Computes the state vector and encodes it into an Uint8Array.</dd> |
| <b><code>Y.mergeUpdates(Array<Uint8Array>)</code></b> |
| <dd> |
| Merge several document updates into a single document update while removing |
| duplicate information. The merged document update is always smaller than |
| the separate updates because of the compressed encoding. |
| </dd> |
| <b><code>Y.encodeStateVectorFromUpdate(Uint8Array): Uint8Array</code></b> |
| <dd> |
| Computes the state vector from a document update and encodes it into an Uint8Array. |
| </dd> |
| <b><code>Y.diffUpdate(update: Uint8Array, stateVector: Uint8Array): Uint8Array</code></b> |
| <dd> |
| Encode the missing differences to another update message. This function works |
| similarly to <code>Y.encodeStateAsUpdate(ydoc, stateVector)</code> but works |
| on updates instead. |
| </dd> |
| <b><code>convertUpdateFormatV1ToV2</code></b> |
| <dd> |
| Convert V1 update format to the V2 update format. |
| </dd> |
| <b><code>convertUpdateFormatV2ToV1</code></b> |
| <dd> |
| Convert V2 update format to the V1 update format. |
| </dd> |
| </dl> |
| |
| ### Relative Positions |
| |
| When working with collaborative documents, we often need to work with positions. |
| Positions may represent cursor locations, selection ranges, or even assign a |
| comment to a range of text. Normal index-positions (expressed as integers) are |
| not convenient to use because the index-range is invalidated as soon as a remote |
| change manipulates the document. Relative positions give you a powerful API to |
| express positions. |
| |
| A relative position is fixated to an element in the shared document and is not |
| affected by remote changes. I.e. given the document `"a|c"`, the relative |
| position is attached to `c`. When a remote user modifies the document by |
| inserting a character before the cursor, the cursor will stay attached to the |
| character `c`. `insert(1, 'x')("a|c") = "ax|c"`. When the relative position is |
| set to the end of the document, it will stay attached to the end of the |
| document. |
| |
| #### Example: Transform to RelativePosition and back |
| |
| ```js |
| const relPos = Y.createRelativePositionFromTypeIndex(ytext, 2) |
| const pos = Y.createAbsolutePositionFromRelativePosition(relPos, doc) |
| pos.type === ytext // => true |
| pos.index === 2 // => true |
| ``` |
| |
| #### Example: Send relative position to remote client (json) |
| |
| ```js |
| const relPos = Y.createRelativePositionFromTypeIndex(ytext, 2) |
| const encodedRelPos = JSON.stringify(relPos) |
| // send encodedRelPos to remote client.. |
| const parsedRelPos = JSON.parse(encodedRelPos) |
| const pos = Y.createAbsolutePositionFromRelativePosition(parsedRelPos, remoteDoc) |
| pos.type === remoteytext // => true |
| pos.index === 2 // => true |
| ``` |
| |
| #### Example: Send relative position to remote client (Uint8Array) |
| |
| ```js |
| const relPos = Y.createRelativePositionFromTypeIndex(ytext, 2) |
| const encodedRelPos = Y.encodeRelativePosition(relPos) |
| // send encodedRelPos to remote client.. |
| const parsedRelPos = Y.decodeRelativePosition(encodedRelPos) |
| const pos = Y.createAbsolutePositionFromRelativePosition(parsedRelPos, remoteDoc) |
| pos.type === remoteytext // => true |
| pos.index === 2 // => true |
| ``` |
| |
| <dl> |
| <b><code> |
| Y.createRelativePositionFromTypeIndex(type:Uint8Array|Y.Type, index: number |
| [, assoc=0]) |
| </code></b> |
| <dd> |
| Create a relative position fixated to the i-th element in any sequence-like |
| shared type (if <code>assoc >= 0</code>). By default, the position associates |
| with the character that comes after the specified index position. If |
| <code>assoc < 0</code>, then the relative position associates with the character |
| before the specified index position. |
| </dd> |
| <b><code> |
| Y.createAbsolutePositionFromRelativePosition(RelativePosition, Y.Doc): |
| { type: Y.AbstractType, index: number, assoc: number } | null |
| </code></b> |
| <dd> |
| Create an absolute position from a relative position. If the relative position |
| cannot be referenced, or the type is deleted, then the result is null. |
| </dd> |
| <b><code> |
| Y.encodeRelativePosition(RelativePosition):Uint8Array |
| </code></b> |
| <dd> |
| Encode a relative position to an Uint8Array. Binary data is the preferred |
| encoding format for document updates. If you prefer JSON encoding, you can |
| simply JSON.stringify / JSON.parse the relative position instead. |
| </dd> |
| <b><code>Y.decodeRelativePosition(Uint8Array):RelativePosition</code></b> |
| <dd>Decode a binary-encoded relative position to a RelativePosition object.</dd> |
| </dl> |
| |
| ### Y.UndoManager |
| |
| Yjs ships with an Undo/Redo manager for selective undo/redo of changes on a |
| Yjs type. The changes can be optionally scoped to transaction origins. |
| |
| ```js |
| const ytext = doc.getText('text') |
| const undoManager = new Y.UndoManager(ytext) |
| |
| ytext.insert(0, 'abc') |
| undoManager.undo() |
| ytext.toString() // => '' |
| undoManager.redo() |
| ytext.toString() // => 'abc' |
| ``` |
| |
| <dl> |
| <b><code>constructor(scope:Y.AbstractType|Array<Y.AbstractType> |
| [, {captureTimeout:number,trackedOrigins:Set<any>,deleteFilter:function(item):boolean}])</code></b> |
| <dd>Accepts either single type as scope or an array of types.</dd> |
| <b><code>undo()</code></b> |
| <dd></dd> |
| <b><code>redo()</code></b> |
| <dd></dd> |
| <b><code>stopCapturing()</code></b> |
| <dd></dd> |
| <b> |
| <code> |
| on('stack-item-added', { stackItem: { meta: Map<any,any> }, type: 'undo' |
| | 'redo' }) |
| </code> |
| </b> |
| <dd> |
| Register an event that is called when a <code>StackItem</code> is added to the |
| undo- or the redo-stack. |
| </dd> |
| <b> |
| <code> |
| on('stack-item-updated', { stackItem: { meta: Map<any,any> }, type: 'undo' |
| | 'redo' }) |
| </code> |
| </b> |
| <dd> |
| Register an event that is called when an existing <code>StackItem</code> is updated. |
| This happens when two changes happen within a "captureInterval". |
| </dd> |
| <b> |
| <code> |
| on('stack-item-popped', { stackItem: { meta: Map<any,any> }, type: 'undo' |
| | 'redo' }) |
| </code> |
| </b> |
| <dd> |
| Register an event that is called when a <code>StackItem</code> is popped from |
| the undo- or the redo-stack. |
| </dd> |
| <b> |
| <code> |
| on('stack-cleared', { undoStackCleared: boolean, redoStackCleared: boolean }) |
| </code> |
| </b> |
| <dd> |
| Register an event that is called when the undo- and/or the redo-stack is cleared. |
| </dd> |
| </dl> |
| |
| #### Example: Stop Capturing |
| |
| UndoManager merges Undo-StackItems if they are created within time-gap |
| smaller than `options.captureTimeout`. Call `um.stopCapturing()` so that the next |
| StackItem won't be merged. |
| |
| ```js |
| // without stopCapturing |
| ytext.insert(0, 'a') |
| ytext.insert(1, 'b') |
| undoManager.undo() |
| ytext.toString() // => '' (note that 'ab' was removed) |
| // with stopCapturing |
| ytext.insert(0, 'a') |
| undoManager.stopCapturing() |
| ytext.insert(0, 'b') |
| undoManager.undo() |
| ytext.toString() // => 'a' (note that only 'b' was removed) |
| ``` |
| |
| #### Example: Specify tracked origins |
| |
| Every change on the shared document has an origin. If no origin was specified, |
| it defaults to `null`. By specifying `trackedOrigins` you can |
| selectively specify which changes should be tracked by `UndoManager`. The |
| UndoManager instance is always added to `trackedOrigins`. |
| |
| ```js |
| class CustomBinding {} |
| |
| const ytext = doc.getText('text') |
| const undoManager = new Y.UndoManager(ytext, { |
| trackedOrigins: new Set([42, CustomBinding]) |
| }) |
| |
| ytext.insert(0, 'abc') |
| undoManager.undo() |
| ytext.toString() // => 'abc' (does not track because origin `null` and not part |
| // of `trackedTransactionOrigins`) |
| ytext.delete(0, 3) // revert change |
| |
| doc.transact(() => { |
| ytext.insert(0, 'abc') |
| }, 42) |
| undoManager.undo() |
| ytext.toString() // => '' (tracked because origin is an instance of `trackedTransactionorigins`) |
| |
| doc.transact(() => { |
| ytext.insert(0, 'abc') |
| }, 41) |
| undoManager.undo() |
| ytext.toString() // => 'abc' (not tracked because 41 is not an instance of |
| // `trackedTransactionorigins`) |
| ytext.delete(0, 3) // revert change |
| |
| doc.transact(() => { |
| ytext.insert(0, 'abc') |
| }, new CustomBinding()) |
| undoManager.undo() |
| ytext.toString() // => '' (tracked because origin is a `CustomBinding` and |
| // `CustomBinding` is in `trackedTransactionorigins`) |
| ``` |
| |
| #### Example: Add additional information to the StackItems |
| |
| When undoing or redoing a previous action, it is often expected to restore |
| additional meta information like the cursor location or the view on the |
| document. You can assign meta-information to Undo-/Redo-StackItems. |
| |
| ```js |
| const ytext = doc.getText('text') |
| const undoManager = new Y.UndoManager(ytext, { |
| trackedOrigins: new Set([42, CustomBinding]) |
| }) |
| |
| undoManager.on('stack-item-added', event => { |
| // save the current cursor location on the stack-item |
| event.stackItem.meta.set('cursor-location', getRelativeCursorLocation()) |
| }) |
| |
| undoManager.on('stack-item-popped', event => { |
| // restore the current cursor location on the stack-item |
| restoreCursorLocation(event.stackItem.meta.get('cursor-location')) |
| }) |
| ``` |
| |
| ## Yjs CRDT Algorithm |
| |
| *Conflict-free replicated data types* (CRDT) for collaborative editing are an |
| alternative approach to *operational transformation* (OT). A very simple |
| differentiation between the two approaches is that OT attempts to transform |
| index positions to ensure convergence (all clients end up with the same |
| content), while CRDTs use mathematical models that usually do not involve index |
| transformations, like linked lists. OT is currently the de-facto standard for |
| shared editing on text. OT approaches that support shared editing without a |
| central source of truth (a central server) require too much bookkeeping to be |
| viable in practice. CRDTs are better suited for distributed systems, provide |
| additional guarantees that the document can be synced with remote clients, and |
| do not require a central source of truth. |
| |
| Yjs implements a modified version of the algorithm described in [this |
| paper](https://www.researchgate.net/publication/310212186_Near_Real-Time_Peer-to-Peer_Shared_Editing_on_Extensible_Data_Types). |
| This [article](https://blog.kevinjahns.de/are-crdts-suitable-for-shared-editing/) |
| explains a simple optimization on the CRDT model and |
| gives more insight about the performance characteristics in Yjs. |
| More information about the specific implementation is available in |
| [INTERNALS.md](./INTERNALS.md) and in |
| [this walkthrough of the Yjs codebase](https://youtu.be/0l5XgnQ6rB4). |
| |
| CRDTs that are suitable for shared text editing suffer from the fact that they |
| only grow in size. There are CRDTs that do not grow in size, but they do not |
| have the characteristics that are beneficial for shared text editing (like |
| intention preservation). Yjs implements many improvements to the original |
| algorithm that diminish the trade-off that the document only grows in size. We |
| can't garbage collect deleted structs (tombstones) while ensuring a unique |
| order of the structs. But we can 1. merge preceding structs into a single |
| struct to reduce the amount of meta information, 2. we can delete content from |
| the struct if it is deleted, and 3. we can garbage collect tombstones if we |
| don't care about the order of the structs anymore (e.g. if the parent was |
| deleted). |
| |
| **Examples:** |
| |
| 1. If a user inserts elements in sequence, the struct will be merged into a |
| single struct. E.g. `text.insert(0, 'a'), text.insert(1, 'b');` is |
| first represented as two structs (`[{id: {client, clock: 0}, content: 'a'}, |
| {id: {client, clock: 1}, content: 'b'}`) and then merged into a single |
| struct: `[{id: {client, clock: 0}, content: 'ab'}]`. |
| 2. When a struct that contains content (e.g. `ItemString`) is deleted, the |
| struct will be replaced with an `ItemDeleted` that does not contain content |
| anymore. |
| 3. When a type is deleted, all child elements are transformed to `GC` structs. A |
| `GC` struct only denotes the existence of a struct and that it is deleted. |
| `GC` structs can always be merged with other `GC` structs if the id's are |
| adjacent. |
| |
| Especially when working on structured content (e.g. shared editing on |
| ProseMirror), these improvements yield very good results when |
| [benchmarking](https://github.com/dmonad/crdt-benchmarks) random document edits. |
| In practice they show even better results, because users usually edit text in |
| sequence, resulting in structs that can easily be merged. The benchmarks show |
| that even in the worst case scenario that a user edits text from right to left, |
| Yjs achieves good performance even for huge documents. |
| |
| ### State Vector |
| |
| Yjs has the ability to exchange only the differences when syncing two clients. |
| We use lamport timestamps to identify structs and to track in which order a |
| client created them. Each struct has an `struct.id = { client: number, clock: |
| number}` that uniquely identifies a struct. We define the next expected `clock` |
| by each client as the *state vector*. This data structure is similar to the |
| [version vectors](https://en.wikipedia.org/wiki/Version_vector) data structure. |
| But we use state vectors only to describe the state of the local document, so we |
| can compute the missing struct of the remote client. We do not use it to track |
| causality. |
| |
| ## License and Author |
| |
| Yjs and all related projects are [**MIT licensed**](./LICENSE). |
| |
| Yjs is based on my research as a student at the [RWTH |
| i5](http://dbis.rwth-aachen.de/). Now I am working on Yjs in my spare time. |
| |
| Fund this project by donating on [GitHub Sponsors](https://github.com/sponsors/dmonad) |
| or hiring [me](https://github.com/dmonad) as a contractor for your collaborative |
| app. |