Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test the encoding sniffing algorithm (aka meta prescan) #130

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Test the encoding sniffing algorithm (aka meta prescan) #130

wants to merge 2 commits into from

Commits on Aug 24, 2020

  1. Test the (meta) prescan algorithm 

    This change adds a `preparsed` subdirectory in the `encoding` directory,
with tests for which the result of the *encoding sniffing algorithm* at
https://html.spec.whatwg.org/#encoding-sniffing-algorithm is the
expected result — that is, tests for which the expected result is the
output of running *only* the encoding sniffing algorithm (of which the
main sub-algorithm is the so-called “meta prescan”) — without
also running the tokenization state machine and tree-construction stage.

This change also adds a README file that explicitly documents what the
expected results for the encoding tests are, based on whether or not
they’re in the `preparsed` subdirectory.

Without those changes, it’s unclear whether the expected results shown
in the existing tests are for the output of fully parsing the test data —
through the tokenization state machine and tree-construction stage — or
instead just the output of the encoding sniffing algorithm only. And
without those changes, we also don’t have any tests a system can use for
testing only the output from the encoding sniffing algorithm.

Fixes #28
    @sideshowbarker
    sideshowbarker committed Aug 24, 2020
    Configuration menu
    Copy the full SHA
    1e10bdb View commit details
    Browse the repository at this point in the history

  2. Note preparsed encoding tests are first 1024 bytes

    @sideshowbarker
    sideshowbarker committed Aug 24, 2020
    Configuration menu
    Copy the full SHA
    b6c4e3f View commit details
    Browse the repository at this point in the history