Specification draft #2
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,2 +1,124 @@ | ||
| # unified-user-data-format | ||
|
|
||
| Specification for the unified user data format across the various "Tube" apps (NewPipe, FreeTube, Invidious, etc..) | ||
|
|
||
| ## Intro | ||
|
|
||
| * This specification is a very early draft, and prone to regular changes | ||
| * Unless otherwise specified, all fields are assumed to be `Strings` | ||
|
|
||
|
|
||
| ## Main file (`main.json`) | ||
|
|
||
| #### `"version"` | ||
|
|
||
| The [semantic version](https://semver.org/) of the specification. | ||
|
|
||
| Conditions for bumping major/minor/patch version: TBD | ||
| Potential suffixes: `-draft`, TBD | ||
|
|
||
| #### `"subscriptions"` | ||
|
|
||
| An array of [subscription objects](spec/subscription_object). | ||
|
|
||
| #### `"playlists"` | ||
|
|
||
| An array of Strings. | ||
|
|
||
| Each string refers to the name of a playlist file (without the `playlist_` prefix) | ||
| that was included in the export. | ||
|
|
||
| #### `"saved_playlists_present"` | ||
|
|
||
| A Boolean. | ||
|
|
||
| Indicates whether the `saved_playlists.json` file was included in the export. | ||
|
|
||
| #### `"watch_history_present"` | ||
|
|
||
| A Boolean. | ||
|
|
||
| Indicates whether the special playlist `watch_history` was included in the export. | ||
|
|
||
| ### Overview | ||
|
|
||
| ```javascript | ||
| { | ||
| "version": "X.Y.Z[-draft/-rc]", | ||
| "saved_playlists_present": Boolean, | ||
| "watch_history_present": Boolean, | ||
| "subscriptions": SubscriptionObject[], | ||
| "playlists": String[], | ||
| } | ||
| ``` | ||
|
|
||
|
|
||
| ## Playlist file (`playlist_xxx.json`) | ||
|
|
||
| A playlist file MUST contain the data of a single playlist created by the user. | ||
|
|
||
| #### > `"name"` | ||
|
|
||
| The title of the playlist. | ||
|
|
||
| #### > `"visibility"` | ||
|
|
||
| The visibility of the playlist. | ||
|
|
||
| Accepted values: "public", "unlisted", "private" | ||
SamantazFox marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| #### > `"description"` | ||
|
|
||
| The description of the playlist. | ||
| Format: plain text (? TBD) | ||
|
|
||
| CAN be nil; In this case, the parser MUST assume an empty `String`. | ||
|
There was a problem hiding this comment. Why is it assuming an empty string if the value is There was a problem hiding this comment. Agree, I think it's perfectly reasonable to assume
Contributor
Author
There was a problem hiding this comment. I probably need to improve the wording... The idea is that nil and empty string have to be considered the same (that is, no description is present). There was a problem hiding this comment. Ideally there should only be null if there is no description, and otherwise there is a string. |
||
|
|
||
| #### > `"videos"` | ||
|
|
||
| An array of [video objects](spec/video_object). | ||
|
|
||
| ### Overview | ||
|
|
||
| ```javascript | ||
| { | ||
| "name": String, | ||
| "visibility": String, | ||
| "description": String, | ||
| "videos": VideoObject[], | ||
| } | ||
| ``` | ||
|
|
||
|
|
||
| ## Saved playlist file (`saved_playlists.json`) | ||
|
|
||
| This file contains the playlists from other services that the user bookmarked. | ||
|
|
||
| #### > `"playlists"` | ||
|
|
||
| An array of [saved playlist objects](spec/saved playlist_object). | ||
|
|
||
| ### Overview | ||
|
|
||
| ```javascript | ||
| { | ||
| "playlists": SavedPlaylist[], | ||
| } | ||
| ``` | ||
|
|
||
|
|
||
| ## Watch history file (`watch_history.json`) | ||
|
|
||
| The watch history is nothing more than a private paylist, without metadata | ||
|
|
||
| #### > `"videos"` | ||
|
|
||
| An array of [video objects](spec/video_object). | ||
|
|
||
| ### Overview | ||
|
|
||
| ```javascript | ||
| { | ||
| "videos": VideoObject[], | ||
| } | ||
| ``` | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| { | ||
| "version": "0.0.5-draft", | ||
| "subscriptions": [ | ||
| { | ||
| "type": "youtube", | ||
| "id": "UCXuqSBlHAE6Xw-yeJA0Tunw", | ||
| "name": "Linus Tech Tips", | ||
| "url": "https://www.youtube.com/channel/UCXuqSBlHAE6Xw-yeJA0Tunw", | ||
| "thumbnail": "https://yt3.ggpht.com/ytc/[snip]" | ||
| } | ||
| ], | ||
| "saved_playlists_present": true, | ||
| "watch_history_present": true, | ||
| "playlists": [ | ||
| "abc123" | ||
| ] | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| { | ||
| "name": "My fav Music!", | ||
| "visibility": "public", | ||
| "description": "The playlist I listen to every day", | ||
| "videos": [ | ||
| { | ||
| "type": "youtube", | ||
| "id": "djV11Xbc914", | ||
| "url": "https://www.youtube.com/v/djV11Xbc914", | ||
| "thumbnail": "https://i.ytimg.com/vi/djV11Xbc914/default.jpg" | ||
| }, | ||
| { | ||
| "type": "youtube", | ||
| "id": "HEXWRTEbj1I", | ||
| "url": "https://www.youtube.com/v/HEXWRTEbj1I", | ||
| "thumbnail": "https://i.ytimg.com/vi/HEXWRTEbj1I/default.jpg" | ||
| }, | ||
| { | ||
| "type": "youtube", | ||
| "id": "sRl02nXVMhw", | ||
| "url": "https://www.youtube.com/v/sRl02nXVMhw", | ||
| "thumbnail": "https://i.ytimg.com/vi/sRl02nXVMhw/default.jpg" | ||
| }, | ||
| { | ||
| "type": "youtube", | ||
| "id": "dQw4w9WgXcQ", | ||
| "url": "https://www.youtube.com/v/dQw4w9WgXcQ", | ||
| "thumbnail": "https://i.ytimg.com/vi/dQw4w9WgXcQ/default.jpg" | ||
| }, | ||
| { | ||
| "type": "youtube", | ||
| "id": "Hy8kmNEo1i8", | ||
| "url": "https://www.youtube.com/v/Hy8kmNEo1i8", | ||
| "thumbnail": "https://i.ytimg.com/vi/Hy8kmNEo1i8/default.jpg" | ||
| }, | ||
| { | ||
| "type": "youtube", | ||
| "id": "zA52uNzx7Y4", | ||
| "url": "https://www.youtube.com/v/zA52uNzx7Y4", | ||
| "thumbnail": "https://i.ytimg.com/vi/zA52uNzx7Y4/default.jpg" | ||
| }, | ||
| { | ||
| "type": "youtube", | ||
| "id": "9bZkp7q19f0", | ||
| "url": "https://www.youtube.com/v/9bZkp7q19f0", | ||
| "thumbnail": "https://i.ytimg.com/vi/9bZkp7q19f0/default.jpg" | ||
| } | ||
| ] | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| { | ||
| "playlists": [ | ||
| { | ||
| "type": "youtube", | ||
| "id": "PLYT4vq6pQVSvgqA8Qsm8TXjYJMTT2XyNV", | ||
| "title": "World Of Walker (Full Album)", | ||
| "author": "Alan Walker", | ||
| "url": "https://www.youtube.com/playlist?list=PLYT4vq6pQVSvgqA8Qsm8TXjYJMTT2XyNV", | ||
| "thumbnail": "https://yt3.ggpht.com/ytc/[snip]" | ||
| } | ||
| ] | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| { | ||
| "videos": [ | ||
| { | ||
| "type": "youtube", | ||
| "id": "djV11Xbc914", | ||
| "url": "https://www.youtube.com/v/djV11Xbc914", | ||
| "thumbnail": "https://i.ytimg.com/vi/djV11Xbc914/default.jpg" | ||
| }, | ||
| { | ||
| "type": "youtube", | ||
| "id": "HEXWRTEbj1I", | ||
| "url": "https://www.youtube.com/v/HEXWRTEbj1I", | ||
| "thumbnail": "https://i.ytimg.com/vi/HEXWRTEbj1I/default.jpg" | ||
| }, | ||
| { | ||
| "type": "youtube", | ||
| "id": "sRl02nXVMhw", | ||
| "url": "https://www.youtube.com/v/sRl02nXVMhw", | ||
| "thumbnail": "https://i.ytimg.com/vi/sRl02nXVMhw/default.jpg" | ||
| }, | ||
| { | ||
| "type": "youtube", | ||
| "id": "dQw4w9WgXcQ", | ||
| "url": "https://www.youtube.com/v/dQw4w9WgXcQ", | ||
| "thumbnail": "https://i.ytimg.com/vi/dQw4w9WgXcQ/default.jpg" | ||
| }, | ||
| { | ||
| "type": "youtube", | ||
| "id": "Hy8kmNEo1i8", | ||
| "url": "https://www.youtube.com/v/Hy8kmNEo1i8", | ||
| "thumbnail": "https://i.ytimg.com/vi/Hy8kmNEo1i8/default.jpg" | ||
| }, | ||
| { | ||
| "type": "youtube", | ||
| "id": "zA52uNzx7Y4", | ||
| "url": "https://www.youtube.com/v/zA52uNzx7Y4", | ||
| "thumbnail": "https://i.ytimg.com/vi/zA52uNzx7Y4/default.jpg" | ||
| }, | ||
| { | ||
| "type": "youtube", | ||
| "id": "9bZkp7q19f0", | ||
| "url": "https://www.youtube.com/v/9bZkp7q19f0", | ||
| "thumbnail": "https://i.ytimg.com/vi/9bZkp7q19f0/default.jpg" | ||
| } | ||
| ] | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| # SavedPlaylist object | ||
|
|
||
| #### > `"type"` | ||
|
|
||
| The type of the service this entry refers to. | ||
|
|
||
| This field is a short and meaningful identifier that prevents the parser from | ||
| trying to guess the name of the service from the `url` present in this object. | ||
|
|
||
| #### > `"id"` | ||
|
|
||
| The ID of the playlist. | ||
|
|
||
| The format if this ID is service-dependant. | ||
|
|
||
| The only requirement being the uniqueness of the ID in the service. | ||
| E.g: All youtube playlist IDs (starting with `PL`) are unique to eachother. | ||
|
|
||
| #### > `"title"` | ||
|
|
||
| The playlist's title. | ||
|
|
||
| #### > `"author"` | ||
|
|
||
| The name of the channel/user who created the playlist. | ||
|
|
||
| #### > `"url"` | ||
|
|
||
| The URL used to access the video. | ||
|
|
||
| This URL SHOULD NOT contain query parameters, unless required by the service | ||
| (e.g youtube playlists). | ||
|
|
||
| #### > `"thumbnail"` | ||
|
|
||
| The URL to the playlist's thumbnail. | ||
|
|
||
| This field CAN be omitted, e.g if the playlist doesn't have a thumbnail, or | ||
| if the exporter doesn't store this metadata. | ||
|
|
||
| If present, it MUST represent the original URL to the thumbnail as found on | ||
| the service mentioned in `type`. | ||
|
|
||
| ### Overview | ||
|
|
||
| ```javascript | ||
| { | ||
| "type": String, | ||
| "id": String, | ||
| "title": String, | ||
| "author": String, | ||
| "url": String, | ||
| "thumbnail": String | null | ||
| } | ||
| ``` |
SamantazFox marked this conversation as resolved.
Show resolved
Hide resolved
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| # Subscription object | ||
|
|
||
| #### > `"type"` | ||
|
|
||
| The type of the service this entry refers to. | ||
|
|
||
| This field is a short and meaningful identifier that prevents the parser from | ||
| trying to guess the name of the service from the `url` present in this object. | ||
|
|
||
| #### > `"id"` | ||
|
|
||
| The ID of the subscribed channel. | ||
|
|
||
| The format if this ID is service-dependant. | ||
|
|
||
| The only requirement being the uniqueness of the ID in the service. | ||
| E.g: All youtube channel IDs (starting with `UC`) are unique to eachother. | ||
|
|
||
| #### > `"name"` | ||
|
|
||
| The name of the subscribed channel. | ||
|
|
||
| #### > `"url"` | ||
|
|
||
| The URL used to access the user subscribed channel. | ||
|
|
||
| This URL SHOULD NOT contain query parameters. | ||
|
|
||
| #### > `"thumbnail"` | ||
|
|
||
| The URL to the user/channel's avatar. | ||
|
|
||
| This field CAN be omitted, e.g if the channel doesn't have an avatar, or | ||
| if the exporter doesn't store this metadata. | ||
|
|
||
| If present, it MUST represent the original URL to the avatar as found on | ||
| the service mentioned in `type`. | ||
|
|
||
| ### Overview | ||
|
|
||
| ```javascript | ||
| { | ||
| "type": String, | ||
| "id": String, | ||
| "name": String, | ||
| "url": String, | ||
| "thumbnail": String | null | ||
| } | ||
| ``` |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we really be doing this? Watch history can have additional fields such as the watched till duration, and additional metadata. I would suggest an additional format to store this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This field is only here to indicate if the
watch_historyfile is part of the export.The format of the watch history can still evolve regardless!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@FireMasterK if that confuses you, I can reword that part!
Right now, videos in history and regular playlists are the same objects, but that's definitely going to change ^^
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That makes more sense! I think they can be reworded once we finalize a specification for watch history too.
But, in such a scenario, can't we remove this field altogether?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was thinking to make some files optional during the export (like the watch history).
This field's purpose is ti tell the importer whether that file is supposed to be present in the export.