Policy Requirements: Implementing Disallowed Items

[schnuerle]

Per our Working Group discussion, there are number of ways that disallowed fields and endpoints could be implemented in the program Policy Requirements feature. See this comment for how it could be implemented, and the notes from this meeting. Let's discuss them here to reach consensus.

Questions (reference by number in your replies):

1. Should disallowed endpoints be specified, and how? Since no endpoints are required by MDS (ie, its kit-of-parts approach to ask for only what you need) is it enough to specify only the endpoints you need and be explicit that other endpoints will not be returned? What happens with endpoints that are not required nor disallowed - how do you handle this third state?
2. What does it mean to disallow an endpoint? For providers to shut it off completely, or ensure the agency has no authenticated access to the endpoint, or some other option?
3. Are there legal or auditing reasons for agencies to explicitly disallow endpoints by enumeration?
4. What burden is placed on providers implementing this? Is it hard to leave out fields (per agency) that have always been required to be returned in MDS? Is there any burden in requiring optional fields?

Note that while in beta and in this minor, non-breaking MDS 1.2.0 release, items listed as required or disallowed will be treated as a 'request' through the Requirements endpoint (precluding intentional formal agency communications with providers) to prevent an unintentional burden on providers.

[jrheard]

I haven't been in the calls lately, so please excuse me if consensus has already been reached about this subtopic - I'm also curious about 5) what should be done if an agency specifies that a particular field should be disallowed, but a provider sends down filled-out values for that field anyway. Should the agency treat the provider's feed as "broken"/"noncompliant", and refuse to use it until the feed has become compliant (i.e. no longer sends down the disallowed field)? I think that it could be useful if the spec offered some general high-level guidance about what to do in this particular situation.

Edit: I didn't notice the italicized portion at the bottom of the prompt, my mistake. It sounds like for now there are no consequences if a feed doesn't comply with an agency's policy requirements (presumably just some emails back and forth until the feed comes into compliance), is that correct?

[quicklywilliam]

Yes, that's my understanding @jrheard. The goal is to make it explicit that providing disallowed fields is non-compliant, but there is nothing in the spec itself that dictates what steps an Agency or API consumer would take when given a non-compliant feed.

[schnuerle]

For the 1.2.0 release since it's a minor release and must be non-breaking, yes it is not required for someone implementing the spec to add or remove the fields listed in the new Requirements endpoint. We are going to make it a 'request' only until the next major release. At that time though, anyone implementing 2.0.0 who is not requiring or leaving out parts of the data feeds based on the agency's Requirements endpoint would not be in compliance with MDS. It's up to the agency to determine the repercussions of non-compliance, as always.

[ezmckinn]

Thanks for driving this discussion @quicklywilliam @jrheard and @schnuerle.

We agree that disallowing entire APIs doesn’t make much sense, especially since the default is for cities to not receive it unless they ask for it.

Disallowing individual fields is both technically feasible, and a useful way for cities to protect rider privacy. We already do this in an affirmative sense (i.e. designating what fields to turn on; inverting this process would be simple). As for fields that are neither disallowed nor expressly required, we think it is fine to leave them as "optional."

One point of caution: we hope the standard MDS feeds (and these working groups) will maintain the privacy focus within the standard specs. We’d like to avoid a scenario where, by default, MDS requires overly sensitive information, legitimate privacy concerns are side stepped on the ground that any field can be disallowed. In this scenario, the burden would be on all cities using the standard to both pay close attention, and implement a Requirements API to explicitly disallow data.

Another issue concerns the international context where  countries have differing privacy laws which would disallow the collection and transmission of certain data elements.

Some guidance from OMF on the following would be helpful:

1. Intended use of the "disallow" feature — e.g. that privacy remains a priority for the general standard, and cities ought to use this to go above and beyond

2. Guidance/recommendations on implementation in international contexts.