|
| 1 | +# Documents |
| 2 | + |
| 3 | +> Create new documents and make them collaborative. |
| 4 | +
|
| 5 | + |
| 6 | + |
| 7 | +> ⚠ **This example only works on JupyterLab v3.1 or higher** |
| 8 | +
|
| 9 | +> Before starting this guide, it is strongly recommended to look at the documentation, precisely the section of [Documents](https://jupyterlab.readthedocs.io/en/stable/extension/documents.html#documents) |
| 10 | +
|
| 11 | +- [Documents](#documents) |
| 12 | + - [Introduction to documents](#introduction-to-documents) |
| 13 | + - [Factories](#factories) |
| 14 | + - [Registering new Documents](#registering-new-documents) |
| 15 | + - [Document Widget](#document-widget) |
| 16 | + - [Document Model](#document-model) |
| 17 | + - [Shared Model](#shared-model) |
| 18 | + |
| 19 | +## Introduction to documents |
| 20 | + |
| 21 | +In JupyterLab, we refer to a document to those widgets backed by a file stored on disk. These files are represented in the frontend by a `Context` which is the bridge between the file and its model, `DocumentModel` representing the data in the file and `DocumentWidget`, which is the view of the model. To make the documents API extensible to enable other developers to write new extensions to support different file types, JupyterLab introduces the `DocumentRegistry` to register new `FileType`s, `DocumentModel`s and `DocumentWidget`s. This way, when opening a new file, the `DocumentManager` will look into the file metadata and create an instance of `Context` with the right `DocumentModel` for this file. To register new documents, you can create factories, either a `ModelFactory` for the model or a `WidgetFactory` for the view. |
| 22 | + |
| 23 | +## Factories |
| 24 | + |
| 25 | +Factories are objects meant to create instances of the suitable widget/model given a file. For example, when the `DocumentManager` detects that the file is a notebook, it uses the notebook widget factory to create a new instance of `NotebookPanel`. On the other hand, if you want to make a new `DocumentModel` or `DocumentWidget` for a specific file type, you have to create a factory and register it to the `DocumentRegister`. When registering a factory, you tell the `DocumentManager` that you added a new model or widget for a specific file. Then, the `DocumentManager` will use those factories to create instances of the new `DocumentModel` or `DocumentWidget`. |
| 26 | + |
| 27 | +The easiest way of creating a new widget factory is extending from the `ABCWidgetFactory<T, U>` and overwrite its method `createNewWidget`. The `DocumentManager` calls `createNewWidget` to create a new widget for a given file. This method receives as an argument the context which includes the model. In this method, you can create and pass as an argument all the objects your `DocumentWidget` needs. Usually, the `DocumentWidget` needs context and the content. The content is the main view of the `DocumentWidget` (you can find more information on the section for the [Document Widget](#document-widget)). |
| 28 | + |
| 29 | +<!-- prettier-ignore-start --> |
| 30 | +```ts |
| 31 | +// src/factory.ts#L33-L40 |
| 32 | + |
| 33 | +protected createNewWidget( |
| 34 | + context: DocumentRegistry.IContext<ExampleDocModel> |
| 35 | +): ExampleDocWidget { |
| 36 | + return new ExampleDocWidget({ |
| 37 | + context, |
| 38 | + content: new ExamplePanel(context), |
| 39 | + }); |
| 40 | +} |
| 41 | +``` |
| 42 | +<!-- prettier-ignore-end --> |
| 43 | + |
| 44 | +On the other hand, to create a `ModelFactory`, you need to implement the interface `IModelFactory<T>` specifying the name of your model, which type of files represents and its format. |
| 45 | + |
| 46 | +<!-- prettier-ignore-start --> |
| 47 | +```ts |
| 48 | +// src/factory.ts#L46-L47 |
| 49 | + |
| 50 | +export class ExampleDocModelFactory |
| 51 | + implements DocumentRegistry.IModelFactory<ExampleDocModel> |
| 52 | +``` |
| 53 | +<!-- prettier-ignore-end --> |
| 54 | +
|
| 55 | +At the same time, you need to implement the method `createNew`. The `DocumentManager` will call this method when opening a file that uses your custom `DocumentModel`. |
| 56 | +
|
| 57 | +<!-- prettier-ignore-start --> |
| 58 | +```ts |
| 59 | +// src/factory.ts#L100-L102 |
| 60 | + |
| 61 | +createNew(languagePreference?: string, modelDB?: IModelDB): ExampleDocModel { |
| 62 | + return new ExampleDocModel(languagePreference, modelDB); |
| 63 | +} |
| 64 | +``` |
| 65 | +<!-- prettier-ignore-end --> |
| 66 | + |
| 67 | +## Registering new Documents |
| 68 | + |
| 69 | +When registering a new document, first of all, you need to know for what file type is your new `DocumentModel`. If the file type is already registered, you won't need to register it again. You could register a new `DocumentModel` for an existing file type. If the file type you want to support is not registered, you will need to register it. To do that, you can use the API `addFileType` from the `DocumentRegistry`. The essential arguments are `extensions` to indicate the extension of the file, `fileFormat` that specifies the data format, and `contentType` to determine if it is a notebook, file or directory. |
| 70 | + |
| 71 | +<!-- prettier-ignore-start --> |
| 72 | +```ts |
| 73 | +// src/index.ts#L73-L81 |
| 74 | + |
| 75 | +// register the filetype |
| 76 | +app.docRegistry.addFileType({ |
| 77 | + name: 'example', |
| 78 | + displayName: 'Example', |
| 79 | + mimeTypes: ['text/json', 'application/json'], |
| 80 | + extensions: ['.example'], |
| 81 | + fileFormat: 'text', |
| 82 | + contentType: 'file', |
| 83 | +}); |
| 84 | +``` |
| 85 | +<!-- prettier-ignore-end --> |
| 86 | + |
| 87 | +Once the file type is registered, you can register a `DocumentModel` for a specific file type. The `DocumentModel` represents the content of the file. For example, JupyterLab has two models registered for the notebook. When you open a notebook with the Notebook editor, the `DocumentManager` creates an instance of the `NotebookModel` that loads the notebook as a JSON object and offers a complex API to manage cells and metadata independently (treats the content of the notebook as a structured data). When opening a notebook with the plain text editor the `DocumentManager` creates an instance of the base `DocumentModel` class which treats the content of the notebook as a string. Note that you can register multiple models for the same file type. Still, these models are not in sync when the user opens two editors for the same file that use different models (like opening a notebook with the notebook editor and the plain text editor). These editors are not in sync because they use different models. At some point, they will show different content. |
| 88 | + |
| 89 | +To register a new `DocumentModel` we can use the API `addModelFactory` from the `DocumentRegistry`. In this case, we created the model factory without arguments, but you can add the argument that you need. |
| 90 | + |
| 91 | +<!-- prettier-ignore-start --> |
| 92 | +```ts |
| 93 | +// src/index.ts#L70-L71 |
| 94 | + |
| 95 | +const modelFactory = new ExampleDocModelFactory(); |
| 96 | +app.docRegistry.addModelFactory(modelFactory); |
| 97 | +``` |
| 98 | +<!-- prettier-ignore-end --> |
| 99 | + |
| 100 | +The last step is to register the `DocumentWidget`. As with the `DocumentModel`, you can register a widget for an existing model or a new model if the existing ones fit your needs. In this case, different widgets using the same model will stay in sync. The `DocumentWidget` is the view for the model, and it is only the layer that allows users to interact with the content of the file. |
| 101 | + |
| 102 | +To register a new `DocumentWidget` we can use the API `addWidgetFactory` from the `DocumentRegistry`. The main arguments you need to add to the factory are the widget's name, the name of the model that this widget uses, a list of file types that the widget can open, and the list of file types that the widget is the default view. |
| 103 | + |
| 104 | +<!-- prettier-ignore-start --> |
| 105 | +```ts |
| 106 | +// src/index.ts#L49-L67 |
| 107 | + |
| 108 | +// Creating the widget factory to register it so the document manager knows about |
| 109 | +// our new DocumentWidget |
| 110 | +const widgetFactory = new ExampleWidgetFactory({ |
| 111 | + name: FACTORY, |
| 112 | + modelName: 'example-model', |
| 113 | + fileTypes: ['example'], |
| 114 | + defaultFor: ['example'], |
| 115 | +}); |
| 116 | + |
| 117 | +// Add the widget to the tracker when it's created |
| 118 | +widgetFactory.widgetCreated.connect((sender, widget) => { |
| 119 | + // Notify the instance tracker if restore data needs to update. |
| 120 | + widget.context.pathChanged.connect(() => { |
| 121 | + tracker.save(widget); |
| 122 | + }); |
| 123 | + tracker.add(widget); |
| 124 | +}); |
| 125 | +// Registering the widget factory |
| 126 | +app.docRegistry.addWidgetFactory(widgetFactory); |
| 127 | +``` |
| 128 | +<!-- prettier-ignore-end --> |
| 129 | + |
| 130 | +## Document Widget |
| 131 | + |
| 132 | +The `DocumentWidget` is the view that will open when opening the file. The `DocumentWidget` contains four main attributes: |
| 133 | + |
| 134 | +- `context`: The context is the bridge between the file on disk and its representation on the frontend. This context includes all the information about the file and some methods to handle the file as its content. Some other attributes you can find in the context are the `DocumentModel` and the `sessionContext`, which handles the communication with the backend. |
| 135 | +- `title`: Which handles the content of the tab. |
| 136 | +- `toolbar`: The editor's toolbar, where you can add different widgets to trigger actions on the document. |
| 137 | +- `contentHeader`: This is a panel between the toolbar and the main content area. You can see this header as a second toolbar or as a notification area. |
| 138 | +- `content`: The content is the main area of the `DocumentWidget` when you add the view for your document. |
| 139 | + |
| 140 | +## Document Model |
| 141 | + |
| 142 | +The `DocumentModel` represents the file in the frontend. Through the model, you can listen to changes in the state of the file like its metadata or some other properties like `dirty` that indicates that the content differs from disk, and you can modify and listen to changes on the content. The main methods on the `DocumentModel` are `toString` and `fromString`, every file but the notebook is loaded/saved to disk as a string using these methods. |
| 143 | + |
| 144 | +## Shared Model |
| 145 | + |
| 146 | +In JupyterLab v3.1, we introduced the package `@jupyterlab/shared-models` to swap `ModelDB` as a data storage to make the notebooks collaborative. We implemented these shared models using [Yjs](https://yjs.dev), a high-performance CRDT for building collaborative applications that automatically sync. You can find all the documentation of Yjs [here](https://docs.yjs.dev). |
| 147 | + |
| 148 | +Yjs documents (`Y.Doc`) are the main class on Yjs. They represent a shared document between clients and hold multiple shared objects. Yjs documents enable you to share different [data types like text, Array, Map or set](https://docs.yjs.dev/getting-started/working-with-shared-types), which makes it possible to create not only collaborative text editors but diagrams, drawings and much more applications. |
| 149 | + |
| 150 | +To sync content between clients, Yjs uses providers. Providers abstract Yjs from the network technology your application uses. They sync Yjs documents through a communication protocol or a database. Most providers have in common that they use the concept of room names to connect Yjs documents. In JupyterLab, we created a package called `@jupyterlab/docprovider` with a WebSocket provider that syncs documents through a new end-point (`api/yjs`) in the JupyterLab server. |
| 151 | + |
| 152 | +Another critical component of Yjs is Awareness. Every Yjs document has an `awareness` attribute that enables you to share user's information like its name, cursor, mouse pointer position, etc. The `awareness` attribute doesn't persist across sessions. Instead, Yjs uses a tiny state-based Awareness CRDT that propagates JSON objects to all users. When you go offline, your awareness state is automatically deleted and notifies all users that you went offline. |
| 153 | + |
| 154 | +After a short explanation of Yjs' features, now it's time to start with the implementation. You can create a new shared model by extending from `YDocument<T>`. [YDocument](https://jupyterlab.readthedocs.io/en/stable/api/classes/shared_models.ydocument.html) is a generic implementation of a shared model that handles the initialization of the `YDoc` and already implements some functionalities like the changes history. |
| 155 | + |
| 156 | +To create a new shared object, you have to use the `ydoc`. The new attribute will be linked to the `ydoc` and sync between the different clients automatically. You can also listen to changes on the shared attributes to propagate them to the `DocumentWidget`. |
| 157 | + |
| 158 | +<!-- prettier-ignore-start --> |
| 159 | +```ts |
| 160 | +// src/model.ts#L327-L328 |
| 161 | + |
| 162 | +this._content = this.ydoc.getMap('content'); |
| 163 | +this._content.observe(this._contentObserver); |
| 164 | +``` |
| 165 | +<!-- prettier-ignore-end --> |
| 166 | + |
| 167 | +To access the information about the different users connected, you can use the `awareness` attribute on the shared model. The `awareness` keeps the state of every user as a map with the user's id as a key and a JSON object as the value for the state. You could add new information to the user's state by using the method `setLocalStateField` and access to the state of all users with `getStates`. To listen for changes on the state of the users, you can use the method `on('change', () => {})`. |
| 168 | + |
| 169 | +<!-- prettier-ignore-start --> |
| 170 | +```ts |
| 171 | +// src/model.ts#L261-L261 |
| 172 | + |
| 173 | +this.sharedModel.awareness.setLocalStateField('mouse', pos); |
| 174 | +``` |
| 175 | +<!-- prettier-ignore-end --> |
| 176 | + |
| 177 | +<!-- prettier-ignore-start --> |
| 178 | +```ts |
| 179 | +// src/model.ts#L289-L289 |
| 180 | + |
| 181 | +const clients = this.sharedModel.awareness.getStates(); |
| 182 | +``` |
| 183 | +<!-- prettier-ignore-end --> |
| 184 | + |
| 185 | +<!-- prettier-ignore-start --> |
| 186 | +```ts |
| 187 | +// src/model.ts#L41-L41 |
| 188 | + |
| 189 | +this.sharedModel.awareness.on('change', this._onClientChanged); |
| 190 | +``` |
| 191 | +<!-- prettier-ignore-end --> |
| 192 | + |
| 193 | +Every time you modify a shared property, this property triggers an event in all the clients to notify them. Still, sometimes you will need to apply a series of modifications as a single transaction to trigger the event only when it has applied all the changes. In this case, you can use the `transaction` method to group all the operations. |
| 194 | + |
| 195 | +<!-- prettier-ignore-start --> |
| 196 | +```ts |
| 197 | +// src/model.ts#L167-L170 |
| 198 | + |
| 199 | +this.sharedModel.transact(() => { |
| 200 | + this.sharedModel.setContent('position', { x: obj.x, y: obj.y }); |
| 201 | + this.sharedModel.setContent('content', obj.content); |
| 202 | +}); |
| 203 | +``` |
| 204 | +<!-- prettier-ignore-end --> |
0 commit comments