-
Notifications
You must be signed in to change notification settings - Fork 293
OpenTimelineIO Application Integrator's Guide
NOTE: This documentation is work in progress and will be in a high state of flux. Once it has stabilized we'll remove this notice.
OpenTimelineIO (OTIO for short) is an interchange format and API for editorial cut information that strives to be a superset of data models like EDL, ALE, AAF, and FCP 7 XML while providing a data and mental model that is accessible for people familiar with NLE (Non-Linear Editing) software.
Integrating OpenTimelineIO with your applications allows for richer, clearer, and more accessible communication about creative decisions and technical metadata through production, finishing, and delivery. Preserving information like lens metadata, rough color treatments, and notes from the director are just a few ways leveraging OpenTimelineIO can have direct impact on the filmmaking process. What metadata does your application generate that would aid realization of the creative vision down the pipeline?
TODO: give an overview of the sections in this document
This document aims to cover common elements used in the OTIO schema and some typical patterns for how they're composed - there is more depth to the full schema than is covered in this document. The OpenTimelineIO Data Model Specification covers the schema objects in much deeper detail.
Applications integrating OTIO should rely on one of the implementations provided by the project to read and write .otio files, but to illustrated the core concepts, we'll include some JSON serialization as examples.
OTIO files are designed to be human readable JSON (with some more explicit constraints around numeric types) and work is in progress on the OpenTimelineIO Data Model Specification to ensure they are easy to understand and useful beyond years to come and in to archival status.
While the specification provides in-depth detail about specific schema objects, their usage, and semantic meaning; integrators should avoid creating their own implementation unless there is a really compelling reason to do so. The core implementation provides a lot of subtle behavior that can be time consuming to reproduce and could impact an application's ability to effectively interoperate with OpenTimelineIO.
Beyond the main C++ implementation with Python bindings, language bindings are available for:
OTIO's implementation is in two layers:
- The serialization method for storing structured data along with a schema name and version for interpreting the data
- A set of schema which define types of objects and the fields they provide
When interacting with OTIO, integrators will be dealing almost entirely deserialized schema objects. The implementation will handle mechanics of translating between in-memory objects and their serialized form.
In addition to the "first-class" fields offered by otio schema objects, every object in OTIO has a dictionary field called metadata that allows applications and users to attach any arbitrary metadata they like. This is a unique feature of OTIO and provides some powerful uses in a production pipeline. Studios attach identifiers for internal production tracking systems, adapters store format-specific contextual metadata about the file they derived the OTIO from, and some users even attach metadata like source script page number or notes from a review system. The metadata dictionary is also a powerful tool in driving the evolution of first-class schema in OTIO.
OTIO allows for any arbitrarily deep collection of other dictionaries, lists, scalars, and even SerializableObjects in the metadata dictionary - the implementation enforces no structure, but there are some strong conventions implementers should follow.
Using a unique namespace helps avoid collision with metadata populated by others. The "namespace" key in this case is just the top-level key in the metadata dictionary under which an implementor places the dictionary containing their data. These top-level "namespace" keys denote informal sorts of sub-schema.
For example, a clip may have a metadata dictionary like the following:
{
"myAnimStudio": {
"sequence": "ABC",
"shot": "ABC010",
"shot_id": "7b3aaa14-8305-4fdd-87c2-b0b9d3f9dac7"
},
"cdl": {
"asc_sat": 0.9,
"asc_sop": {
"offset": [
1,
-0.0122,
0.0305
],
"power": [
1,
0,
1
],
"slope": [
0.1,
0.2,
0.3
]
}
},
"cmx_3600": {
"comments": [
"SOURCE FILE: ABC010.LAYOUT3.01"
]
}
}Note that information for studio production tracking, CDL values, and a comment field from the original CMX 3600 EDL all coexist as separated by their namespace keys.
In reading and writing OTIO, make a best effort to preserve the data in metadata dictionaries. If possible, only mutate the data under the namespaces your application controls. This ensures that metadata populated upstream of your application will flow downstream without loss. In the previous example, the identifiers under the myAnimStudio namespace provide critical value to the pipeline and avoid using brittle file-path based matching. Likewise, critical creative decisions are stored in the cdl namespace, preserving them helps us ensure we maintain the artistic vision of the piece.
Part of OTIO is that all schema objects are serializable. .otio files are a JSON serialization of an object tree from OTIO with one root.
While there are no current plans to create alternate serialization formats, the implementation is built to allow creation of a new serialization format if needed. If this were to happen, a new file extension would be used.
Any object available in the OTIO schema can be serialized and de-serialized as the root object in a .otio file. This means implementers should check what kind of schema object results from a deserialization process and handle it appropriately. This also means there is a great degree of freedom in what kind of data can be communicated with OTIO. Whole timelines, bins, and individual clips are among the possible objects that can be imported and exported via OTIO.
While any OTIO schema object can be serialized at the top-level, most OTIO files will have either a Timeline or SerializableCollection as the root object.
For instance, in cases where older workflows might use an .edl file, an .otio file with a Timeline would be used. In cases where an .ale file was used, an .otio file with a SerializableCollection of Clip objects would be used.
OTIO has no schema for a project, this would typically be represented as a SerializableCollection containing Timelines, other SerializableCollections for folder or bin structure, and Clips for various media in the project.
OTIO offers a lot of the essentials for the most common types of cut information used in pipelines, but the team is also actively expanding the schema to enable more and more use cases. Schema development is driven by real-world use in production for a period of time when possible.
The file format and implementation provide a number of useful tools to help implementors easily progress with OTIO's evolution.
Each object type has an independent schema version that is stored as an integer that increments each time there is a breaking change. This means non-breaking changes, like addition of fields, won't interfere with loading in older implementations, they just may not be available.
When the library encounters a serialized object with an older schema, it automatically migrates that object to the newest available schema, even "up-converting" data if needed. This means application implementors do not need to worry about having multiple versions of schema in-memory for a given type of object.
metadata dictionaries are a good place to store information that is not yet part of the "first-class" schema
By making data available early in a metadata dictionary, it allows pipelines to start making use of it and aid in the design of an "first-class" schema to be added to OTIO. Most new schema in OTIO starts as some extra information in a metadata dictionary, once it has been proven useful and the form has been refined, an update to OTIO will then promote that to new "first-class" schema.
Upgrading the OTIO release used in your software ensures you have maximum read compatibility with other applications, moving forward with the format helps users from having to make special concessions for you. There are currently no mechanisms provided for downgrading the schema versions used within an OTIO file, so keeping up-to-date ensures you stay relevant.
OpenTimelineIO's Schema provides familiar objects meant to directly correlate to the units editors interact with in an NLE while providing advanced compositions. Below are some of the most common schema objects and some guidance on using them.
Timeline objects represent one edit, sequence, or presentation. Timelines are named - as you might see a sequence named in an editor's project.
The Timeline object
Using an appropriate namespace key, the Timeline metadata attribute is a great place to store timeline/sequence settings such as edit rate, video resolution, and audio sample rate. In the future many of these concepts will be given first-class attributes.
SerializableCollection objects store an ordered collection of other OTIO schema objects. They are commonly used to represent bins or folders in a project and can be nested to create deeper project structure. The name property would be the name you see for the folder or bin within a project and the children property is a collection of the items within the collection.
When creating an editorial composition, there are two important relationships items have to one another, sequential (clip B comes after clip A), and simultaneously (clip B is composited over clip A). In OTIO, a Track is used to compose items in sequential order and a Stack is used to compose items that are simultaneous. A Timeline object's tracks attribute references a Stack containing the timeline's audio, video, and other Tracks.
This Stack of Track items is directly analogous to the timeline interface where editors compose their clips in NLE software.
Commonly the only Stack used in an OTIO document is the tracks stack on the timeline. The children of the Stack is an ordered list of all the items (typically Tracks) that are coincident in "compositing" order.
In general, it is assumed that visual media uses alpha compositing, overlaying items one over the other. Audio media is assumed to use additive compositing, mixing all the tracks together so you hear them all. If another compositing method is used, somewhere under your application's namespace key the metadata on the Stack is a great place to store that information.
The children of the Track is a list of items in the order they should be played. A track is, in a way, like a playlist. As opposed to formats like EDL, items do not specify a range in the output they occupy, they are simply one after another.
The items in a timeline are either Clip or Gap schema objects. Clip objects specify media to use in the composition, whereas Gap objects can be used to offset Clip objects temporally.
These Items are single-use, meaning any given Item instance can be used in a composition once. The APIs provide ways of copying Items to create any number of identical instances for use.
A Clip represents an instance of using some subsegment of media in the timeline. This is directly analogous to clips you see in a standard NLE.
The source in and out points on a Clip are set using source_range. The source_range is specified as the time in the source media to select.
Gap
Gap represents an absence of media - in other words, a Gap does not contribute to the composition media, it only serves to offset other items in time.
Following is a compilation of the guidelines scattered throughout the guide as a reference:
- Use a unique "namespace" for data your application stores within object
metadatadictionaries. - Preserve metadata to the best of your abilities.
- Check the type of top-level object when deserializing.
-
metadatadictionaries should be used to store important information that is not yet a "first-class" part of the OTIO schema. - Keep up-to-date with the OTIO implementation.
- Find all schema object name references and link out to appropriate documentation
- Create illustrations showing examples of the schema objects, maybe from otioview?
- Continue reorganization pass through specific schema objects section