Inconsistency: DNR modifyHeaders supported headers

[sammacbeth]

The DNR modifyHeaders rule action allows for request headers to be added, removed or modified. The current Chrome docs do not specify any limitation on which headers these rules can modify.

In fact, in Chrome and Firefox arbitrary header names are supported (apart from a list of forbidden header names) and can be modified by these rules. However, in Safari, only 'recognized' headers can specified in these rules. For example, take the following rule:

{
  id: 1,
  priority: 1,
  action: {
    type: "modifyHeaders",
    requestHeaders: [
      { header: "Sec-GPC", operation: "set", value: "1" },
    ],
  },
  condition: {
    urlFilter: "||global-privacy-control.glitch.me/",
    resourceTypes: ["main_frame", "sub_frame"],
  },
}

This rule is accepted and works as expected in Firefox and Chrome. However, in Safari, the following exception is thrown:

Invalid call to declarativeNetRequest.updateDynamicRules(). Error with rule at index 0: Rule with id 1 is invalid. The header `Sec-GPC` is not recognized.

This issue has also been submitted to Safari as Radar #FB12074761

[erosman]

While on the subject, there are additional considerations:

• Forbidden header name
• Modification of cookie, host, origin, & referer as required for User scripts in Manifest V3 #279
  (usually implemented with arbitrary header names)

[bershanskiy]

In fact, in Chrome and Firefox it seems that arbitrary header names are supported and can be modified by these rules.

No, unfortunately Chrome and Firefox do have some limitations, these limitations are just not documented. There is a list somewhere in source code of both projects,but I couldn't find it in a quick glance.

Forbidden header name

Please note that different context may have different forbidden header names, e.g., https://bugs.chromium.org/p/chromium/issues/detail?id=595993

[sammacbeth]

In fact, in Chrome and Firefox it seems that arbitrary header names are supported and can be modified by these rules.

No, unfortunately Chrome and Firefox do have some limitations, these limitations are just not documented. There is a list somewhere in source code of both projects,but I couldn't find it in a quick glance.

Forbidden header name

Please note that different context may have different forbidden header names, e.g., https://bugs.chromium.org/p/chromium/issues/detail?id=595993

Thanks, I updated the issue description to add that context.

[Rob--W]

I had added more context in a comment on crbug "Issue 1117475: DNR rules which add new headers are accepted inconsistently"

The "append" operation is now supported for a subset of changes since 108, as part of (non-public) issue 1355626.
The relevant changes are at https://chromium-review.googlesource.com/q/Ie5ef2bae40e7c3d1d04f284775d9a9c1044790ea and https://source.chromium.org/chromium/chromium/src/+/d377adec687d1d631fe7fea246706b51fdfea368

The test case from this bug report is still not addressed, because the "append" operation is only supported for a subset of explicitly supported headers, defined at https://cs.chromium.org/kDNRRequestHeaderAppendAllowList . The changes have not been documented.

While this is not documented in Chrome's DNR docs, I did document the inconsistency on MDN at: https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/declarativeNetRequest/ModifyHeaderInfo#header_limits

[xeenon]

Safari is planning to add Sec-GPC to the allow list. We do have an allow list of headers that we check for any operation type.

[Rob--W]

follow-up: safari Needs a response from a Safari representative and follow-up: chrome Needs a response from a Chrome representative reflect the action item from today's meeting:

post a comment with the current behaviors (and intended behavior if any), so that we can have a common ground to start the discussion with

Firefox 113+ supports DNR without restrictions other than the Host request header: specifying a new value requires host permissions for the new host value (documented at https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/declarativeNetRequest/ModifyHeaderInfo#header_limits ). In particular, DNR extensions in Firefox can specify Sec- headers.

While not documented, usual restrictions from (request/response) headers apply. E.g. some headers don't permit multiple values, and an attempt to specify multiple results in unspecified behavior. E.g. appending Content-Type twice results in the value being set to the last specified value.

[rdcronin]

From the Chrome side, using a blocklist instead of an allowlist is intentional. We heard strong developer feedback that there were use cases where developers needed to add or modify custom headers, which could never be reasonably contained in an allowlist. While, in principle, we acknowledge that allowlists are preferable, we felt this was an important use case to support.

[oliverdunk]

@xeenon, my understanding is that the implementation in Safari was based on the original behaviour in Chrome (which has now changed). Given that Chrome uses a deny list, would you be open to aligning?

[oliverdunk]

In this case, we are using supportive/implemented to mean using a deny list for header names.

[xeenon]

Yeah, a deny list makes more sense to me.

[derjanb]

[...] We do have an allow list of headers that we check for any operation type.

In case someone stumbles upon it, this is Safari's allow list