`Sitemap.from_xml_string` cannot opt into enqueue host filtering

[janbuchar]

Found while triaging GHSA-c533-9w4q-667c (which I'm closing as informative - a raw XML string has no origin, so there's no host policy to bypass). It does point at a real API gap though.

• Sitemap.from_xml_string takes only content and hardcodes SitemapSource(type='raw', content=content) - no url, no ParseSitemapOptions
• host filtering is conditional on source.get('url')
• so if you have raw sitemap XML and you know where it came from, there's no way to say so - filtering is silently skipped and the same-hostname default never applies
• suggestion: from_xml_string(content, *, sitemap_url: str | None = None, parse_sitemap_options: ParseSitemapOptions | None = None), forwarded as SitemapSource(type='raw', content=content, url=sitemap_url)
• explicitly not worth doing: rejecting <loc> values when the origin is unknown - that makes the default behaviour return an empty list
• while in there: _get_origin_url stamps raw sources with raw://<sha256>, so SitemapUrl.origin_sitemap_url is a pseudo-URL that would fail scheme validation if anyone fed it back into filter_url - harmless today, footgun later

[arpitjain099]

While tracing the advisory I noticed most of the plumbing for this already exists, in case it helps whoever picks it up.

SitemapSource already declares url as NotRequired, and the filter in _process_sitemap_item keys on source.get('url') rather than on the source type, so a raw source that carries a url would be filtered today with no change to the filtering code. What blocks it is only that from_xml_string hardcodes the source:

async def from_xml_string(cls, content: str) -> Sitemap:
    return await cls.parse([SitemapSource(type='raw', content=content)])

so the shape could be as small as an optional origin plus the options passthrough that is currently missing:

@classmethod
async def from_xml_string(
    cls,
    content: str,
    url: str | None = None,
    parse_sitemap_options: ParseSitemapOptions | None = None,
) -> Sitemap:
    source = SitemapSource(type='raw', content=content)
    if url is not None:
        source['url'] = url
    return await cls.parse([source], parse_sitemap_options=parse_sitemap_options)

Two things worth deciding before anyone writes it, since they change behaviour beyond filtering:

• _get_origin_url() returns raw://<sha256> for any raw source. If a caller declares an origin, should SitemapUrl.origin_sitemap_url become that URL instead, or keep the content hash so raw sources stay distinguishable in output?
• That same value is the dedup key for nested sitemaps. Switching it for url-bearing raw sources means a raw sitemap and the fetched copy of the same sitemap would dedup against each other, which may be what you want or may not.

Happy to send the PR if you tell me which way you want those two, otherwise feel free to use any of this.