forked from w3c/openscreenprotocol
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathindex.bs
2940 lines (2312 loc) · 131 KB
/
index.bs
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<pre class='metadata'>
Title: Open Screen Protocol
Shortname: openscreenprotocol
Level: None
Status: w3c/ED
ED: https://w3c.github.io/openscreenprotocol/
TR: https://www.w3.org/TR/openscreenprotocol/
Canonical URL: ED
Editor: Mark Foltz, Google, https://github.com/mfoltzgoogle, w3cid 68454
Repository: w3c/openscreenprotocol
Abstract: The Open Screen Protocol is a suite of network protocols that allow
user agents to implement the [[PRESENTATION-API|Presentation API]] and
the [[REMOTE-PLAYBACK|Remote Playback API]] in an interoperable
fashion.
Group: secondscreenwg
Markup Shorthands: markdown yes, dfn yes, idl yes, markup yes
</pre>
<pre class="anchors">
url: https://html.spec.whatwg.org/multipage/media.html#concept-mediaerror-code; type: dfn; spec: html
text: media error code
urlPrefix: https://html.spec.whatwg.org/multipage/media.html#; type: dfn; spec: html
text: official playback position
text: poster frame
text: timeline offset
text: media resource
text: media timeline
urlPrefix: https://www.w3.org/TR/presentation-api/#dfn-; type: dfn; spec: PRESENTATION-API
text: available presentation display
text: controlling user agent
text: presentation; url: receiving-browsing-context
text: presentation identifier
text: presentation request url
text: receiving user agent
urlPrefix: https://w3c.github.io/remote-playback/#dfn-; type: dfn; spec: REMOTE-PLAYBACK
text: availability sources set
text: compatible remote playback device
text: initiate remote playback
text: media element state
text: remote playback devices
text: remote playback source
url: https://datatracker.ietf.org/doc/html/rfc9000#name-transport-parameter-encodin; type: dfn; spec: RFC9000; text: Transport Parameter Encoding
url: https://datatracker.ietf.org/doc/html/rfc9000#name-connection_close-frames; type: dfn; spec: RFC9000; text: CONNECTION_CLOSE Frames
url: https://datatracker.ietf.org/doc/html/rfc9000#name-variable-length-integer-enc; type: dfn; spec: RFC9000; text: Variable-Length Integer Encoding
url: https://datatracker.ietf.org/doc/html/rfc9000#name-variable-length-integer-enc; type: dfn; spec: RFC9000; text: variable-length integer
url: https://tools.ietf.org/html/rfc6762#section-9; type: dfn; spec: RFC6762; text: conflict resolution
url: https://tools.ietf.org/html/rfc6763#section-7; type: dfn; spec: RFC6763; text: service name
url: https://tools.ietf.org/html/rfc6763#section-4.1.1; type: dfn; spec: RFC6763; text: instance name
url: https://tools.ietf.org/html/rfc5646#section-2; type: dfn; spec: RFC5646; text: language tag
url: https://tools.ietf.org/html/rfc4122#section-4.4; type: dfn; spec: RFC4122; text: UUID
url: https://tools.ietf.org/html/rfc8122#section-5; type: dfn; spec: RFC8122; text: sha-256
url: https://tools.ietf.org/html/rfc8122#section-5; type: dfn; spec: RFC8122; text: sha-512
url: https://tools.ietf.org/html/rfc8122#section-5; type: dfn; spec: RFC8122; text: md2
url: https://tools.ietf.org/html/rfc8122#section-5; type: dfn; spec: RFC8122; text: md5
url: https://tools.ietf.org/html/rfc6381#section-3; type: dfn; spec: RFC6381; text: codecs parameter
url: https://tools.ietf.org/html/rfc8610#section-3; type: dfn; spec: RFC8610; text: concise data definition language
url: https://www.iso.org/standard/62021.html#; type: dfn; spec: iso18004; text: QR code
url: https://tools.ietf.org/html/rfc5280#section-4.2.1.3; type: dfn; spec: RFC5280; text: digitalSignature
url: https://datatracker.ietf.org/doc/html/rfc8446#section-4.2.3; type: dfn; spec: RFC8446; text: signature scheme
</pre>
Introduction {#introduction}
============================
The Open Screen Protocol connects browsers to devices capable of rendering Web
content for a shared audience. Typically, these are devices like
Internet-connected TVs, HDMI dongles, and smart speakers.
This spec defines a suite of network protocols that enable two user agents to
implement the [[PRESENTATION-API|Presentation API]] and
[[REMOTE-PLAYBACK|Remote Playback API]] in an interoperable fashion. This means
that a Web developer can expect these APIs to work as intended when connecting
two devices from independent implementations of the Open Screen Protocol.
The Open Screen Protocol is a specific implementation of these two APIs, meaning
that it does not handle all possible ways that browsers and presentation
displays could support these APIs. The Open Screen Protocol specifically
supports browsers and displays that are connected via the same local area
network. It allows a browser to present a URL, initiate remote playback of an
HTML <l spec=html>[=media element=]</l>, and stream media data to another
device.
The Open Screen Protocol is intended to be extensible, so that additional
capabilities can be added over time. This may include additions to existing Web
APIs or new Web APIs.
The accompanying
[explainer](https://w3c.github.io/openscreenprotocol/explainer.html) provides
more background on the protocol.
Terminology {#terminology}
--------------------------
An <dfn lt="open screen protocol agent|osp agent">Open Screen Protocol
agent</dfn> (or OSP agent) is any implementation of this protocol (browser,
display, speaker, or other software).
Some OSP agents support the [[PRESENTATION-API|Presentation API]]. The
API allows a [=controlling user agent=] to initiate presentation of Web content
on another device. We call this agent a <dfn>controller</dfn> for short. A
[=receiving user agent=] is responsible for rendering the Web content, which we
will call a <dfn>receiver</dfn> for short. The the Web content itself is called
a [=presentation=].
Some OSP agents also support the [[REMOTE-PLAYBACK|Remote Playback API]]. That
API allows an agent to render a <l spec="html">[=media element=]</l> on a
[=remote playback device=]. In this document, we refer to it as a [=receiver=]
because it is shorter and keeps terminology consistent between presentations and
remote playbacks. Similarly, we use the term [=controller=] to refer to the
agent that starts, terminates, and controls the remote playback.
For media streaming, we refer to an OSP agent that sends a media stream
as a <dfn>media sender</dfn> and an agent that receives a media stream as a
<dfn>media receiver</dfn>. Note that an agent can be both a media sender and a
media receiver.
For additional terms and idioms specific to the
[[PRESENTATION-API|Presentation API]] or [[REMOTE-PLAYBACK|Remote Playback API]],
please consult the respective specifications.
Requirements {#requirements}
============================
General Requirements {#requirements-general}
--------------------------------------------------------------
1. An [=Open Screen Protocol agent=] must be able to discover the presence of
another OSP agent connected to the same IPv4 or IPv6 subnet and reachable by
IP multicast.
2. An OSP agent must be able to obtain the IPv4 or IPv6 address of
the agent, a display name for the agent, and an IP port number for
establishing a network transport to the agent.
Presentation API Requirements {#requirements-presentation-api}
--------------------------------------------------------------
1. A [=controller=] must be able to determine if a [=receiver=] is reasonably
capable of rendering a specific [=presentation request URL=].
2. A controller must be able to start a new [=presentation=] on a
receiver given a [=presentation request URL=] and [=presentation
identifier=].
3. A controller must be able to create a new {{PresentationConnection}} to an
existing presentation on the receiver, given its [=presentation request
URL=] and [=presentation identifier=].
4. It must be possible to close a {{PresentationConnection}} between a
controller and a presentation, and signal both parties with the reason why
the connection was closed.
5. Multiple controllers must be able to connect to a single presentation
simultaneously.
6. Messages sent by the controller must be delivered to the presentation (or
vice versa) in a reliable and in-order fashion.
7. If a message cannot be delivered, then the controller must be
able to signal the receiver (or vice versa) that the connection should be
closed with reason `error`.
8. The controller and presentation must be able to send and receive `DOMString`
messages (represented as `string` type in ECMAScript).
9. The controller and presentation must be able to send and receive binary
messages (represented as `Blob` objects in HTML5, or `ArrayBuffer` or
`ArrayBufferView` types in ECMAScript).
10. The controller must be able to signal to the receiver to
terminate a presentation, given its [=presentation request URL=] and
[=presentation identifier=].
11. The receiver must be able to signal all connected controllers
when a presentation is terminated.
Remote Playback API Requirements {#requirements-remote-playback}
----------------------------------------------------------------
1. A [=controller=] must be able to find out whether there is at least one
compatible [=receiver=] for a given {{HTMLMediaElement}}, both
instantaneously and continuously.
2. A controller must be able to to [=initiate remote playback=] of
an {{HTMLMediaElement}} to a compatible receiver.
3. The controller must be able send media sources as URLs and text
tracks from an {{HTMLMediaElement}} to a compatible receiver.
4. The controller must be able send media data from an {{HTMLMediaElement}} to
a compatible receiver.
5. During remote playback, the controller and the remote playback
device must be able to synchronize the [=media element state=] of the
{{HTMLMediaElement}}.
6. During remote playback, either the controller or the receiver must be able
to disconnect from the other party.
7. The controller should be able to pass locale and text direction information
to the receiver to assist in rendering text during remote playback.
Non-Functional Requirements {#requirements-non-functional}
----------------------------------------------------------
1. It should be possible to implement an OSP agent using modest
hardware requirements, similar to what is found in a low end smartphone,
smart TV or streaming device. See the [Device
Specifications](https://w3c.github.io/openscreenprotocol/device_specs.html)
document for agent hardware specifications.
2. The discovery and connection protocols should minimize power consumption,
especially on a [=listening agent=] which is likely to be battery
powered.
3. The protocol should minimize the amount of information provided to a passive
network observer about the identity of the user or activities on the agent, including
presentations, remote playbacks, or the content of media streams.
4. The protocol should prevent active network attackers from impersonating a
display and observing or altering data intended for the controller or
receiver.
5. A listening agent should be able to discover quickly when an [=advertising
agent=] becomes available or unavailable (i.e., when it connects or
disconnects from the network).
6. Agents should present sensible information to the user when a protocol
operation fails. For example, if a controller is unable to start a
presentation, it should be possible to report in the controller interface if
it was a network error, authentication error, or the presentation content
failed to load.
7. Agents should be able to remember that a user authenticated another agent.
This means it is not required for the user to intervene and re-authenticate
each time an agent wants to connect to an agent that the user has already
authenticated.
8. Message latency between agents should be minimized to permit interactive
use. For example, it should be comfortable to type in a form in one agent
and have the text appear in the presentation in real time. Real-time
latency for gaming or mouse use is ideal, but not a requirement.
9. The controller initiating a presentation or remote playback should
communicate its preferred locale to the receiver, so it can render the
content in that locale.
10. It should be possible to extend the control protocol (above the discovery and
transport levels) with optional features not defined explicitly by the
specification, to facilitate experimentation and enhancement of the base
APIs.
Discovery with mDNS {#discovery}
===============================
[=Open Screen Protocol agents=] discover one another by advertising and
listening for information identifying themselves along with an IP service
endpoint. Agent advertisement and discovery through [[RFC6763|DNS-SD]] and
[[RFC6762|mDNS]] is defined by this specification and is mandatory to implement
by all agents. However, agents are free to implement additional discovery
mechanisms, such as querying for the same DNS-SD records via unicast DNS.
OSP agents must use the DNS-SD [=Service Name=] `_openscreen._udp`.
An <dfn noexport>advertising agent</dfn> is one that responds to mDNS queries
for `_openscreen._udp.local`. Such an agent should have a <dfn noexport>display
name</dfn> (a non-empty string) that is a human readable description of the
presentation display, e.g. "Living Room TV."
A <dfn noexport>listening agent</dfn> is one that sends mDNS queries for
`_openscreen._udp.local`. Listening agents may have a display name.
Advertising agents must use a DNS-SD [=Instance Name=] that is a prefix of the
agent's display name. If the Instance Name is not the complete display name, it
must be terminated by a null (`\000`) character, so that a listening agent knows
it has been truncated.
Advertising agents must follow the mDNS [=conflict resolution=] procedure, to
prevent multiple advertising agents from using the same DNS-SD Instance Name.
Agents should be careful when displaying Instance Names to users; see
[[#instance-names]] for guidelines on Instance Name display.
Advertising agents must include DNS TXT records with the following
keys and values:
: fp
:: The [=agent fingerprint=] of the advertising agent. The steps to compute
the agent fingerprint are defined below.
: mv
:: An unsigned integer value that indicates that metadata has changed. The
advertising agent must update it to a greater value. This signals to the
listening agent that it should connect to the advertising agent to discover
updated metadata. The value should be encoded as a
[=variable-length integer=].
: at
:: An alphanumeric, unguessable token consisting of characters from the set
`[A-Za-z0-9+/]`.
Note: `at` prevents off-LAN parties from attempting authentication; see
[[#remote-active-mitigations]]. `at` should have at least 32 bits of true
entropy to make brute force attacks impractical.
NOTE: If an OSP agent suspends its network connectivity (e.g. for power saving
reasons) it should attempt to retain cached and valid mDNS records so that
discovery state is preserved when the network connection is resumed.
<!-- TODO: Add examples of sample mDNS records. -->
Future extensions to this QUIC-based protocol can use the same metadata
discovery process to indicate support for those extensions, through a
capabilities mechanism to be determined. If a future version of the Open Screen
Protocol uses mDNS but breaks compatibility with the metadata discovery process,
it should change the DNS-SD service name to a new value, indicating a new
mechanism for metadata discovery.
Computing the Agent Fingerprint {#computing-agent-fingerprint}
-------------------------------
The <dfn>agent fingerprint</dfn> of an agent is computed by following these
steps:
1. Compute the [[RFC7469#section-2.4|SKPI Fingerprint]] of the [=agent certificate=]
according to [[!RFC7469]] using [[RFC6234|SHA-256]] as the hash algorithm.
2. base64 encode the result of Step 1 according to [[!RFC4648]].
Note: The resulting string will be 44 bytes in length.
Transport and metadata discovery with QUIC {#transport}
=======================================================
If a [=listening agent=] wants to connect to or learn further metadata about an
[=advertising agent=], it initiates a [[!RFC9000|QUIC]] connection to the IP and port
from its SRV record. Prior to authentication, a message may be exchanged (such
as further metadata), but such info should be treated as unverified (such as
indicating to a user that a display name of an unauthenticated agent is
unverified).
The connection IDs used both by agents should be zero length. If zero length
connection IDs are chosen, agents are restricted from changing IP or port
without establishing a new QUIC connection. In such cases, agents must
establish a new QUIC connection in order to change IP or port.
TLS 1.3 {#tls-13}
-----------------
When an [=OSP Agent=] makes a QUIC connection to another agent, it must use
[[!RFC8446|TLS 1.3]] to secure the connection. TLS 1.3 should be used with the
following application-specific parameters to indicate that the connection will
be used to communicate with a specific OSP Agent using OSP. An OSP Agent may
refuse incoming connections that lack these parameters.
* The [[!RFC7301|ALPN]] used must be "osp".
* The [[!RFC6066|server_name extension]] must be set to the following `host_name`:
`<fp>._openscreen._udp`.
* `<fp>` must be substituted with the [=agent fingerprint=] as used in mDNS TXT.
An OSP Agent must not send TLS early data.
Issue(228): Register ALPN with IANA.
Agent Certificates {#certificates}
----------------------------------
Each OSP Agent must generate an [[!RFC5280|X.509 v3]] <dfn>agent
certificate</dfn> containing a public key to be used with the TLS 1.3
certificate exchange. Both [=advertising agents=] and [=listening agents=] must
use the [=agent certificate=] in TLS 1.3 `Certificate` messages when making a
QUIC connection.
The [=agent certificate=] must have the following characteristics:
* 256-bit ECDSA public key.
* Self-signed.
* Support the `ecdsa_secp256r1_sha256` [=signature scheme=] as defined in TLS 1.3.
* The `AlgorithmIdentifier` values are as defined in [[!RFC5480]] (for public
keys) and [[!RFC5758]] (for signature schemes).
* [[!X690]] specifies the Distinguished Encoding Rules (DER) representation
used to encode the identifiers.
* Valid for signing.
Let the <dfn>certificate serial number</dfn> be the result of the following steps:
<ol>
<li>If the agent has never generated an agent certificate:
<ol>
<li>Let the <dfn>certificate serial number base</dfn> be a 32-bit
pseudorandom integer value.</il>
<li>Let the <dfn>certificate serial number counter</dfn> be a 32-bit
unsigned integer, initially set to 0.</li>
</ol>
</li>
<li>Generate a 64-bit value as follows:
<ol>
<li>Increment the [=certificate serial number counter=] by one.</li>
<li>Assign the upper 32 bits to the [=certificate serial number base=].</li>
<li>Assign the lower 32 bits to the [=certificate serial number counter=].</il>
</ol>
</ol>
The following X.509 v3 fields are to be set as follows:
<div class="assertion">
<table>
<thead>
<th>Field</th>
<th>Value</th>
</thead>
<tbody>
<tr>
<td>Version Number</td>
<td>3</td>
</tr>
<tr>
<td>Serial Number</td>
<td>The [=certificate serial number=].</td>
</tr>
<tr>
<td>Public Key `AlgorithmIdentifier`</td>
<td>
<ul>
<li>ECC OID: `1.2.840.10045.2.1`</li>
<li>ECDSA 256 OID: `1.2.840.10045.3.1.7`</li>
<li>DER representation: `301306072a8648ce3d020106082a8648ce3d030107`</li>
</ol>
</td>
</tr>
<tr>
<td>Signature `AlgorithmIdentifier`</td>
<td>
<ul>
<li>OID: `1.2.840.10045.4.3.2`</li>
<li>DER representation: `300a06082a8648ce3d040302`</li>
</ul>
</td>
</tr>
<tr>
<td>Issuer Name</td>
<td>CN = The `model-name` from the `agent-info` message.<br/>
O = See note.<br/>
L = See note.<br/>
ST = See note.<br/>
C = See note.<br/>
</td>
</tr>
<tr>
<td>Subject Name</td>
<td>CN = `<fp>`._openscreen._udp<br/>
O = See note.<br/>
</td>
</tr>
<tr>
<td>Subject Public Key Algorithm</td>
<td>Elliptic Curve Public Key</td>
</tr>
<tr>
<td>Certificate Key usage</td>
<td>[=digitalSignature=]</td>
</tr>
</tbody>
</table>
</div>
Mandatory fields not mentioned above should be set according to [[!RFC5280]].
The value `<sn>` above should be substituted with the [=certificate serial
number=].
Note: The OSP agent may use the implementer or device model name as the value
for the `O` key for user interface and debugging purposes. It may use the agent
implementer's or device manufacturer's location as the value for the location
keys (`L`, `ST`, and `C`) for user interface and debugging purposes.
If an OSP agent sees an [=agent certificate=] it has not yet verified through
[[#authentication]], it must treat that agent as unverified and initiate
authentication with that agent before allowing additional messages to be
exchanged with that agent (apart from the messages described in [[#metadata]]).
If an OSP agent sees a valid [=agent certificate=] it has verified through
authentication, it is not required to initiate authentication with that agent
before sending further messages.
Metadata Discovery {#metadata}
------------------------------
To learn further metadata, an agent may send an [=agent-info-request=] message
and receive back an [=agent-info-response=] message. Any agent may send this
request at any time to learn about the state and capabilities of another device,
which are described by the [=agent-info=] message in the
[=agent-info-response=].
If an agent changes any information in its [=agent-info=] message, it should
send an [=agent-info-event=] message to all other connected agents with the new
[=agent-info=] (without waiting for an [=agent-info-request=]).
The [=agent-info=] message contains the following fields:
: display-name (required)
:: The display name of the agent, intended to be displayed to a user by the
requester. The requester should indicate through the UI if the responder
is not authenticated or if the display name changes.
: model-name (optional)
:: If the agent is a hardware device, the model name of
the device. This is used mainly for debugging purposes, but may be
displayed to the user of the requesting agent.
: capabilities (required)
:: The control protocols, roles, and media types the agent supports.
Presence indicates a capability and absence indicates lack of a
capability. Capabilities should should affect how an agent is
presented to a user, such as drawing a different icon depending on
the whether it receives audio, video or both.
: state-token (required)
:: A random alphanumeric value consisting of 8 characters in the range
[0-9A-Za-z]. This value is set before the agent makes its first connection
and must be set to a new value when the agent is reset or otherwise lost all
of its state related to this protocol.
: locales (required)
:: The agent's preferred locales for display of localized content, in the order
of user preference. Each entry is an RFC5646 [=language tag=].
The various capabilities have the following meanings:
: receive-audio
:: The agent can render audio via the other protocols it supports. Those other
protocols may report more specific capabilities, such as support for
certain audio codecs in the streaming protocol.
: receive-video
:: The agent can receive video via the other protocols it supports. Those other
protocols may report more specific capabilities, such as support for
certain video codecs in the streaming protocol.
: receive-presentation
:: The agent can receive presentations using the presentation protocol.
: control-presentation
:: The agent can control presentations using the presentation protocol.
: receive-remote-playback
:: The agent can receive remote playback using the remote playback
protocol.
: control-remote-playback
:: The agent can control remote playback using the remote playback
protocol.
: receive-streaming
:: The agent can receiving streaming using the streaming protocol.
: send-streaming
:: The agent can send streaming using the streaming protocol.
NOTE: See the [Capabilities Registry](https://github.com/w3c/openscreenprotocol/blob/main/capabilities.md)
for a list of all known capabilities (both defined by this specification, and
through [[#protocol-extensions]]).
If a listening agent wishes to receive messages from an advertising agent or an
advertising agent wishes to send messages to a listening agent, it may wish to
keep the QUIC connection alive. Once neither side needs to keep the connection
alive for the purposes of sending or receiving messages, the connection should
be closed with an error code of 5139. In order to keep a QUIC connection alive, an
agent may send an [=agent-status-request=] message, and any agent that receives an
[=agent-status-request=] message should send an [=agent-status-response=] message. Such
messages should be sent more frequently than the QUIC idle_timeout transport
parameter (see [=Transport Parameter Encoding=] in [[!RFC9000|QUIC]]) and QUIC PING
frames should not be used. An idle_timeout transport parameter of 25 seconds is
recommended. The agent should behave as though a timer less than the
idle_timeout were reset every time a message is sent on a QUIC stream. If the
timer expires, a [=agent-status-request=] message should be sent.
If a listening agent wishes to send messages to an advertising agent, the
listening agent can connect to the advertising agent "on demand"; it does not
need to keep the connection alive.
If an OSP agent suspends its network connectivity (e.g. for power saving
reasons), it should attempt to resume QUIC connections to the OSP agents to
which it was previously connected once network connectivity is restored. Once
reconnected, it should send `agent-status-request` messages to those agents.
The [=agent-info=] and [=agent-status-response=] messages may be extended to
include additional information not defined in this spec, as described in
[[#protocol-extension-fields]].
Messages delivery using CBOR and QUIC streams {#messages}
========================================================
Messages are serialized using [[!RFC8949|CBOR]]. To send a group of messages in
order, that group of messages must be sent in one QUIC stream. Independent
groups of messages (with no ordering dependency across groups) should be sent in
different QUIC streams. In order to put multiple CBOR-serialized messages into
the the same QUIC stream, the following is used.
For each message, the [=OSP agent=] must write into a unidirectional QUIC stream
the following:
1. A type key representing the type of the message, encoded as a [=variable-length
integer=] (see [[#appendix-a]] for type keys)
2. The message encoded as CBOR.
If an agent receives a message for which it does not recognize a type key, it
must close the QUIC connection with an application error code of 404 and should
include the unknown type key in the reason phrase of the [=CONNECTION_CLOSE
frame=].
Variable-length integers are encoded in the [=Variable-Length Integer Encoding=]
used by [[!RFC9000|QUIC]].
Many messages are requests and responses, so a common format is defined for
those. A request and a response includes a request ID which is an unsigned
integer chosen by the requester. Responses must include the request ID of the
request they are associated with.
Type Key Backwards Compatibility {#message-compatibility}
--------------------------------
As messages are modified or extended over time, certain rules must be followed
to maintain backwards compatibiilty with agents that understand older versions
of messages.
1. If a required field is added to or removed from a message (either to/from the
message directly or indirectly through the field of a field), a new type key
must be assigned to the message. Is is effectively a new message and must not
be sent unless the receiving agent is known to understand the new type key.
1. If an optional field is added to a message (either to the message directly
or indirectly through the field of a field), the type key may remain unchanged
if the behavior of older receiving agents that do not understand the added field
is compatible with newer sending agents that include the field.
Otherwise, a new type key must be assigned.
1. If an optional field is removed from a message (either from the message
directly or indirectly through the field of a field), the type key may remain
unchanged if the behavior of newer receiving agents that do not understand the
removed field is compatible with older sending agents that include the field.
Otherwise, a new type key must be assigned.
1. Required fields may not be added or removed from array-based messages, such
as audio-frame.
Authentication {#authentication}
================================
Each supported authentication method is implemeted via authentication messages
specific to that method. The authentication method is explicitly specified by
the message itself. The authentication status message is common for all authentication
methods. Any new authentication method added must define new authentication messages.
[=Open Screen Protocol agents=] must implement [[#authentication-with-spake2]]
with pre-shared keys.
Prior to authentication, agents exchange [=auth-capabilities=] messages specifying
pre-shared key (PSK) ease of input for the user and supported PSK input methods.
The agent with the lowest PSK ease of input presents a PSK to the user when the agent
either sends or receives an authentication request. In case both agents have the same
PSK ease of input value, the server presents the PSK to the user. The same pre-shared key
is used by both agents. The agent presenting the PSK to the user is the PSK presenter,
the agent requiring the user to input the PSK is the PSK consumer.
PSK ease of input is an integer in the range from 0 to 100 inclusive, where 0 means
it is not possible for the user to input PSK on this device and 100 means
that it's easy for the user to input PSK on the device. Supported PSK input methods
are numeric and scanning a QR-code. Devices with non-zero PSK ease of input must
support the numeric PSK input method.
Any authentication method may require an [=auth-initiation-token=] before
showing a PSK to the user or requesting PSK input from the user. For an
[=advertising agent=], the `at` field in its mDNS TXT record must be used as the
`auth-initation-token` in the the first authentication message sent to or from
that agent. Agents should discard any authentication message whose
`auth-initation-token` is set and does not match the `at` provided by the
advertising agent.
In the `psk-min-bits-of-entropy` field of the [=auth-capabilities=] messsage,
agents may specify the minimum bits of entropy it requires for a PSK, in the
range of 20 to 60 bits inclusive, with a default of 20. The PSK presenter must
generate a PSK that has at least as many bits of entropy as it receives in this
field, and at least as many bits of entropy as it sends in this field.
If an agent chooses to show a user a PSK in more than one way (such as both a
QR-code and a numeric PSK), they should be for the same PSK. If they were
different, the PSK presenter would not know which one the user chose to use, and
that may lead to authentication failures.
[[#appendix-c]] describes two encoding schemes for PSKs that agents may
support to produce either a string or a [=QR code=] for display to the user.
Authentication with SPAKE2 {#authentication-with-spake2}
--------------------------
Issue(242): [Meta] Track CFRG PAKE competition outcome
For all messages and objects defined in this section, see [[#appendix-a]] for
the full CDDL definitions.
The default authentication method is
\[SPAKE2](https://tools.ietf.org/html/draft-irtf-cfrg-spake2-26) with
the following cipher suite:
1. Elliptic curve is \[edwards25519](https://tools.ietf.org/html/rfc7748#page-4).
2. Hash function is \[SHA-256](https://tools.ietf.org/html/rfc6234).
3. Key derivation function is \[HKDF](https://tools.ietf.org/html/rfc5869).
4. Message authentication code is \[HMAC](https://tools.ietf.org/html/rfc2104).
5. Password hash function is \[SHA-512](https://tools.ietf.org/html/rfc6234).
Open Screen Protocol does not use a memory-hard hash function to hash PSKs with
SPAKE2 and uses SHA-512 instead, as the PSK is one-time use and is not stored in
any form.
SPAKE2 provides explicit mutual authentication.
This authentication method assumes the agents share a low-entropy secret,
such as a number or a short password that could be entered by a user on a
phone, a keyboard or a TV remote control.
SPAKE2 is not symmetric and has two roles, Alice (A) and Bob (B).
The messages used in this authentication method are: [=auth-spake2-handshake=],
[=auth-spake2-confirmation=] and [=auth-status=]. \[SPAKE2] describes in detail
how [=auth-spake2-handshake=] and [=auth-spake2-confirmation=] are computed.
The values `A` and `B` used in SPAKE2 are the [=agent fingerprints=] of the
client and server, respectively. `pw` is the PSK presented to the user.
The PSK presenter or the PSK consumer may initiate authentication (assuming the
role of Alice in SPAKE2).
If the PSK presenter wants to initiate authentication, it starts the
authentication process by presenting the PSK to the user and sending a
[=auth-spake2-handshake=] message. The `public-value` field of the
[=auth-spake2-handshake=] message must be set to the value of `pA` from SPAKE2
and the `psk-status` field must be set to `psk-shown`.
When the PSK consumer receives the [=auth-spake2-handshake=] message, the PSK
consumer prompts the user for the PSK input if it has not done so yet. Once it
receives the PSK, it sends an [=auth-spake2-handshake=] message with the
`public-value` field set to the value of `pB` from SPAKE2 and the `psk-status`
field set to `psk-input`.
If the PSK consumer wants to initiate authentication, the PSK consumer sends a
[=auth-spake2-handshake=] message to the PSK presenter with the `psk-status`
field set to `psk-needs-presentation` and the `public-value` field set to
`pA`. The PSK presenter, on receiving this message, creates a PSK and presents
it to the the user. Once that is done, it sends an [=auth-spake2-handshake=]
message to the PSK consumer with `psk-status` set to `psk-input` and the
`public-value` field set to `pB`.
Once an agent knows both `pA` and `pB` from [=auth-spake2-handshake=] messages,
it computes and sends a [=auth-spake2-confirmation=] with the
`confirmation-value` field set to `cA` (for Alice) or `cB` (for Bob) to the
other agent.
Once an agent receives an [=auth-spake2-confirmation=] message, it validates
that message using the procedure in \[SPAKE2] and then replies with an
[=auth-status=] authenticated message to the other agent. Any value of `result`
other than `authenticated` means that authentication failed, and the agent must
immediately disconnect.
NOTE: The [=auth-status=] message is merely informative as each agent
independently computes the outcome of SPAKE2 through key confirmation
verification.
[[#appendix-d]] shows the entire process when agents have not
authenticated each other, including discovery, QUIC connection establishment,
metadata exchange and authentication. When agents have completed authentication,
the authentication phase can be omitted.
Presentation Protocol {#presentation-protocol}
=====================
This section defines the use of the Open Screen Protocol for starting,
terminating, and controlling presentations as defined by
[[PRESENTATION-API|Presentation API]]. [[#presentation-api]]
defines how APIs in [[PRESENTATION-API|Presentation API]] map to the
protocol messages defined in this section.
To learn which receivers are [=available presentation displays=] for a
particular [=presentation request URL=] or set of URLs, the controller may send
a [=presentation-url-availability-request=] message with the following values:
: urls
:: A list of presentation URLs. Must not be empty.
: watch-duration
:: The period of time that the controller is interested in receiving updates
about their URLs, should the availability change.
: watch-id
:: An identifier the receiver must use when sending updates about URL
availability so that the controller knows which URLs the receiver is referring
to.
In response, the receiver should send one [=presentation-url-availability-response=]
message with the following values:
: url-availabilities
:: A list of URL availability states. Each state must correspond to the matching URL
from the request by list index.
While the watch is valid (the watch-duration has not expired), the receivers
should send [=presentation-url-availability-event=] messages when URL
availabilities change. Such events contain the following values:
: watch-id
:: The watch-id given in the [=presentation-url-availability-response=],
used to refer to the presentation URLs whose availability has changed.
: url-availabilities
:: A list of URL availability states. Each state must correspond to the URLs from the
request referred to by the watch-id.
Note that these messages are not broadcasted to all controllers. They are sent
individually to controllers that have requested availability for the URLs that
have changed in availability state within the watch duration of the original
availability request.
To save power, the controller may disconnect the QUIC connection and
later reconnect to send availability requests and receive availability
responses and updates. The QUIC connection ID may or may not be the same
when reconnecting.
To start a presentation, the controller may send a
[=presentation-start-request=] message to the receiver with the following
values:
: presentation-id
:: The [=presentation identifier=]
: url
:: The selected presentation URL
: headers
:: headers that the receiver should use to fetch the presentation URL. For example,
[[PRESENTATION-API#creating-a-receiving-browsing-context|section 6.6.1]] of
the Presentation API says that the HTTP `Accept-Language` header should be
provided.
The [=presentation identifier=] must follow the restrictions defined by
[[PRESENTATION-API#common-idioms|section 6.1]] of the Presentation API, in that
it must consist of at least 16 ASCII characters.
When the receiver receives the [=presentation-start-request=], it should send back a
[=presentation-start-response=] message after either the presentation URL has been
fetched and loaded, or the receiver has failed to do so. If it has failed, it
must respond with the appropriate result (such as invalid-url or timeout). If
it has succeeded, it must reply with a success result.
Additionally, the response must include the following:
: connection-id
:: An ID that both agents can use to send connection messages
to each other. It is chosen by the receiver for ease of implementation: if
the message receiver chooses the connection-id, it may keep the ID unique
across connections, thus making message demuxing/routing easier.
The response should include the following:
: http-response-code
:: The numeric HTTP response code that was returned from fetching the
presentation URL (after redirects).
To send a presentation message, the controller or receiver may send a
[=presentation-connection-message=] with the following values:
: connection-id
:: The ID from the [=presentation-start-response=] or
[=presentation-connection-open-response=] messages.
: message
:: The presentation message data.
NOTE: An OSP agent should minimize buffering and processing of messages sent or
received via the QUIC connection beyond what is strictly necessary (i.e., CBOR
serialization). Message payloads should be treated as real-time data, as they
may be used to synchronize playback of media streams between agents or other low
latency use cases. The synchronization thresholds recommended in
[[ITU-R-BT.1359-1]] imply that the total agent-to-agent processing latency
(including serialization, buffering, QUIC processing, and network latency) must
be no greater than 45 ms to permit effective lip sync during media playback.
To terminate a presentation, the controller may send a
[=presentation-termination-request=] message with the following values:
: presentation-id
:: The ID of the presentation to terminate.
: reason
:: Set to `application-request` if the application requested termination,
or `user-request` if the user requested termination. (These are the only
valid values for `reason` in a [=presentation-termination-request=].)
When a [=receiver=] receives a [=presentation-termination-request=], it should
send back a [=presentation-termination-response=] message to the requesting
controller.
It should also notify other controllers about the termination by sending
a [=presentation-termination-event=] message. And it can send the same message if
it terminates a presentation without a request from a controller to do so. This
message contains the following values:
: presentation-id
:: The ID of the presentation that was terminated.
: source
:: Set to `controller` when the termination was in response to a
[=presentation-termination-request=], or `receiver` otherwise.
: reason
:: The detailed reason why the presentation was terminated.
To accept incoming connection requests from controller, a receiver must receive
and process the [=presentation-connection-open-request=] message which contains the
following values:
: presentation-id
:: The ID of the presentation to connect to.
: url
:: The URL of the presentation to connect to.
The receiver should, upon receipt of a
[=presentation-connection-open-request=] message, send back a
[=presentation-connection-open-response=] message which contains the
following values:
: result
:: a code indicating success or failure, and the reason for the failure
: connection-id
:: An ID that both agents can use to send connection messages
to each other. It is chosen by the receiver for ease of implementation (if
the message receiver chooses the connection-id, it may keep the ID unique
across connections, thus making message demuxing/routing easier).
: connection-count
:: The new number of open connections to the presentation that received
the incoming connection request.
If the [=presentation-connection-open-response=] message indicates success, the
receiver should also send a [=presentation-change-event=] to all other endpoints
that have an active presentation connection to that presentation with the
values:
: presentation-id
:: The ID of the presentation that just received a new presentation connection.
: connection-count
:: The new total number of open connections to that presentation.
A controller may close a connection without terminating the presentation by
sending a [=presentation-connection-close-event=] message to the receiver with the
following values:
: connection-id
:: The ID of the connection that was closed.
: reason
:: Set to `close-method-called` or `connection-object-discarded`.
The receiver may also close a connection without terminating a presentation. If
it does so, it should send a [=presentation-connection-close-event=] message to the
controller with the following values:
: connection-id
:: The ID of the connection that was closed.
: reason
:: Set to `close-method-called` or `connection-object-discarded`.
: connection-count
:: The number of open presentation connections that remain.
If a receiver closes a presentation connection (for any reason), it should send
a [=presentation-change-event=] to all other controllers with an open connection
to that presentation with the values:
: presentation-id
:: The ID of the presentation that just closed a connection.
: connection-count
:: The number of open presentation connections that remain.
Note: When an agent closes a presentation connection, it is always successful,
so request and response messages are not needed. A request to terminate a
presentation may succeed or fail, so a response message is required.