Replace dead reference URLs found behind bot walls - #9774
Merged
Merged
Conversation
Repoints dead reference URLs to the same document at a live official URL, an official page with the same content, or a pinned Wayback capture of the dead URL. Only URLs change. Lists the dead prefixes in known_dead_reference_urls/wave2.toml for the offline guard. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
These mappings were verified earlier but skipped by the first build because their mechanical liveness check had not run yet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Virginia's 5% copayment cap from July 2025 now cites the enacted 2025 Appropriation Act, Item 125.10 B.4, instead of a 2026 HB30 amendment for later fiscal years. - Drop the six Oklahoma OAC 340:40 replacements: five pointed at the Cornell LII mirror and one at an archive of a different OKDHS URL, neither of which is the issuing agency or a capture of the dead URL. Those citations and guard entries go back to how main has them. - Utah's 2025 military retirement credit rate cites page 24, where its worksheet is. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # policyengine_us/variables/gov/states/al/tax/income/al_agi.py # policyengine_us/variables/gov/states/al/tax/income/al_income_tax_before_non_refundable_credits.py # policyengine_us/variables/gov/states/al/tax/income/al_income_tax_before_refundable_credits.py # policyengine_us/variables/gov/states/md/tax/income/agi/subtractions/md_two_income_subtraction.py # policyengine_us/variables/gov/states/md/tax/income/deductions/md_deductions.py # policyengine_us/variables/gov/states/md/tax/income/deductions/md_itemized_deductions.py
The guard added in #9786 rejects a reference that is not one bare URL. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The merge of main took this branch's side of each conflicted file, which put back URLs main had already replaced. Restore main's replacements; the branch's own reference changes in these files stay. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
Merged by the US + core hub at head Gates:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Part of the repo-wide audit of dead reference URLs outside tax.ny.gov (follow-up to #9711 and #9730). This PR covers URLs that only showed as dead from inside a browser (bot-walled hosts): New Hampshire Revenue, Utah forms, Louisiana Legislature over http, Westlaw Maryland Code, elaws.us rule mirrors, Illinois www2, Iowa Revenue and others.
#page.parameters/,variables/anddocs/differs from the old line only inside URLs. The exceptions: twelve Python files that ruff re-wrapped once their URLs changed length (their ASTs match apart from the URL strings), and the Virginia reference title described under "Changes after review".policyengine_us/tests/code_health/known_dead_reference_urls/wave2.tomllists 48 of the dead prefixes, so the offline guard from Replace dead NY tax department reference URLs #9711 fails if they return.How each URL was checked
curl -sL -A "Mozilla/5.0"on 2026-09-30, rechecked with a Chrome user agent. A 403, 429 or bot challenge counted as unknown, not dead, and was rechecked inside a real browser page; URLs still behind a bot wall were left unchanged. A 200 that landed on a home, search, login or not-found page counted as dead.pdftotexton the cited#pagefor PDFs). Disputed items went to an adjudicator under fixed rules: the agency's own site, then another official publisher, then a Wayback capture of the dead URL. Third-party mirrors (taxformfinder.org, NBER and the like) were not accepted for official citations.#pageexists in its PDF. A few replacements behind bot walls were confirmed in a browser instead.Changes
http://dcrules.elaws.us/dcmr/5-a199https://osse.dc.gov/sites/default/files/dc/sites/osse/publication/attachments/Final%20Rulemaking%20for%20the%20Licensing%20of%20Child%20Development%20Facilities.pdfhttp://epolicy.dpss.lacounty.gov/epolicy/epolicy/server/general/projects_responsive/ePolicyMaster/index.htm?&area=general&type=responsivehelp&ctxid=&project=ePolicyMasterhttps://my.dpss.lacounty.gov/public/en/home/epolicy/program/calworks/composition/minimum-basic-standard-of-adequate-care.htmlhttps://my.dpss.lacounty.gov/public/en/home/epolicy/program/calworks/income/earned-income-disregards.htmlhttps://my.dpss.lacounty.gov/public/en/home/epolicy/program/calworks/property/property.htmlhttps://my.dpss.lacounty.gov/public/en/home/epolicy/program/child-care/overview.htmlhttps://my.dpss.lacounty.gov/public/en/home/epolicy/program/child-care/regional-market-rate-ceilings.htmlhttps://my.dpss.lacounty.gov/public/en/home/epolicy/program/gr/residence/immigrant-eligibility-chart.htmlhttp://ilrules.elaws.us/iac/t23_pt235_sec.235.10https://www.ilga.gov/commission/jcar/admincode/023/023020500A00100R.htmlhttp://legis.la.gov/Legis/Law.aspx?d=101769https://legis.la.gov/Legis/Law.aspx?d=101769http://legis.la.gov/Legis/Law.aspx?d=453233https://legis.la.gov/Legis/Law.aspx?d=453233http://txrules.elaws.us/rule/title1_chapter372_sec.372.408https://texas-sos.appianportalsgov.com/rules-and-meetings?interface=VIEW_TAC_SUMMARY&recordId=141498http://txrules.elaws.us/rule/title40_chapter809_sec.809.41https://texas-sos.appianportalsgov.com/rules-and-meetings?interface=VIEW_TAC_SUMMARY&recordId=229534http://txrules.elaws.us/rule/title40_chapter809_sec.809.44https://texas-sos.appianportalsgov.com/rules-and-meetings?interface=VIEW_TAC_SUMMARY&recordId=210290http://www.legis.la.gov/legis/Law.aspx?d=101760https://www.legis.la.gov/legis/Law.aspx?d=101760https://alisondb.legislature.state.al.us/alison/CodeOfAlabama/1975/Coatoc.htmhttps://alison.legislature.state.al.us/code-of-alabamahttps://budget.lis.virginia.gov/amendment/2024/2/HB30/Enrolled/CR/126/2h/https://budget.lis.virginia.gov/item/2025/1/HB1600/Chapter/1/125.10/https://capriverside.org/sites/g/files/aldnop136/files/2024-12/2025%20LIHEAP%20CAP%20APPLICATION%20ENGLISH.pdfhttps://capriverside.org/sites/g/files/aldnop136/files/2024-12/2025%20LIHEAP%20CAP%20APPLICATION%20ENGLISH.pdf?archive-consent=truehttps://codes.findlaw.com/il/chapter-35-revenue/il-st-sect-35-5-212-1.htmlhttps://www.ilga.gov/Documents/legislation/publicacts/102/PDF/102-0700.pdfhttps://des.az.gov/sites/default/files/dl/CCA-1210B.pdfhttps://web.archive.org/web/20251031002516/https://des.az.gov/sites/default/files/dl/CCA-1210B.pdfhttps://doe.virginia.gov/home/showpublisheddocument/56270https://ris.dls.virginia.gov/uploads/22VAC40/dibr/VDOE%20Child%20Care%20Program%20Guidance%20Manual%205.3.2023-20240822104447.pdfhttps://emanuals.jfs.ohio.gov/CashFoodAssist/CAM/ACT/https://web.archive.org/web/20240806164713/https://emanuals.jfs.ohio.gov/CashFoodAssist/CAM/ACT/https://epolicy.dpss.lacounty.gov/epolicy/epolicy/server/general/projects_responsive/ePolicyMaster/index.htm?&area=general&type=responsivehelp&ctxid=&project=ePolicyMasterhttps://my.dpss.lacounty.gov/public/en/home/epolicy/program/calworks/composition/minimum-basic-standard-of-adequate-care.htmlhttps://my.dpss.lacounty.gov/public/en/home/epolicy/program/calworks/income/earned-income-disregards.htmlhttps://my.dpss.lacounty.gov/public/en/home/epolicy/program/calworks/payments/calworks-maximum-aid-payment.htmlhttps://my.dpss.lacounty.gov/public/en/home/epolicy/program/calworks/property/determining-property-value-vehicles.htmlhttps://my.dpss.lacounty.gov/public/en/home/epolicy/program/calworks/property/property.htmlhttps://www.cdss.ca.gov/ord/entres/getinfo/pdf/10EAS.pdfhttps://fns-prod.azureedge.net/sites/default/files/resource-files/WICPC2018FoodPackage-Summary.pdfhttps://www.fns.usda.gov/sites/default/files/resource-files/WICPC2018FoodPackage-Summary.pdfhttps://fns-prod.azureedge.net/sites/default/files/resource-files/fna-2008-amended-through-pl-116-94.pdfhttps://www.govinfo.gov/content/pkg/USCODE-2019-title7/html/USCODE-2019-title7-chap51-sec2014.htmhttps://govt.westlaw.com/mdc/Document/N05479690A64A11DBB5DDAC3692B918BC?viewType=FullText&originationContext=documenttoc&transitionType=CategoryPageItem&contextData=(sc.Default)https://mgaleg.maryland.gov/2022RS/Statute_Web/gtg/10-218.pdfhttps://govt.westlaw.com/mdc/Document/N5C14CC91D9D311E2A5EFA1428CB399FF?viewType=FullText&originationContext=documenttoc&transitionType=CategoryPageItem&contextData=(sc.Default)https://mgaleg.maryland.gov/2024RS/Statute_Web/gtg/10-105.pdfhttps://govt.westlaw.com/mdc/Document/N9D9D6B70E6C811ECB80FD99D7E19CB35?viewType=FullText&originationContext=documenttoc&transitionType=CategoryPageItem&contextData=(sc.Default)https://mgaleg.maryland.gov/mgawebsite/Laws/StatuteText?article=gtg§ion=10-754&enactments=falsehttps://govt.westlaw.com/mdc/Document/NC8EB19606F6911E8A99BCF2C90B83D38?viewType=FullText&originationContext=documenttoc&transitionType=CategoryPageItem&contextData=(sc.Default)https://mgaleg.maryland.gov/2024RS/Statute_Web/gtg/10-217.pdfhttps://govt.westlaw.com/mdc/Document/NF93A7BD2E6C811ECA065A3F5EAA0E5C9?viewType=FullText&originationContext=documenttoc&transitionType=CategoryPageItem&contextData=(sc.Default)https://mgaleg.maryland.gov/mgawebsite/Laws/StatuteText?article=gtg§ion=10-207&enactments=falsehttps://housingforhouston.com/residents/housing-choice-voucher/payment-standards/https://web.archive.org/web/20251109130455/https://housingforhouston.com/residents/housing-choice-voucher/payment-standards/https://revenue.iowa.gov/press-release/2023-10-25/idr-announces-2024-individual-income-tax-brackets-and-interest-rateshttps://content.govdelivery.com/accounts/IACIO/bulletins/3779769https://tax.idaho.gov/wp-content/uploads/forms/EFO00088/https://tax.idaho.gov/wp-content/uploads/forms/EFO00088/EFO00088_03-02-2026.pdfhttps://tax.utah.gov/forms/2021/tc-40.pdfhttps://files.tax.utah.gov/tax/forms/2021/tc-40.pdfhttps://tax.utah.gov/forms/2021/tc-40inst.pdfhttps://files.tax.utah.gov/tax/forms/2021/tc-40inst.pdfhttps://tax.utah.gov/forms/2022/tc-40inst.pdfhttps://files.tax.utah.gov/tax/forms/2022/tc-40inst.pdfhttps://tax.utah.gov/forms/2023/tc-40inst.pdfhttps://files.tax.utah.gov/tax/forms/2023/tc-40inst.pdfhttps://tax.utah.gov/forms/drafts/tc-40inst.pdfhttps://files.tax.utah.gov/tax/forms/2025/tc-40inst.pdfhttps://trackbill.com/bill/maryland-house-bill-186-income-tax-subtraction-modification-for-centenarians/2173534/https://mgaleg.maryland.gov/mgawebsite/Legislation/Details/hb0186?ys=2022RShttps://www.capitol.hawaii.gov/hrscurrent/Vol04_Ch0201-0257/HRS0235/HRS_0235-0055_0075.htmhttps://www.capitol.hawaii.gov/hrscurrent/Vol04_Ch0201-0257/HRS0235/HRS_0235-0055_0007_0005.htmhttps://www.cde.ca.gov/sp/cd/ci/mb2505.asphttps://web.archive.org/web/20260125002512/https://www.cde.ca.gov/sp/cd/ci/mb2505.asphttps://www.huduser.gov/hudapi/https://www.huduser.gov/portal/dataset/fmr-api.htmlhttps://www.revenue.nh.gov/forms/2021/documents/dp-10-2020-print.pdfhttps://www.revenue.nh.gov/sites/g/files/ehbemt736/files/documents/dp-10-2020-print.pdfhttps://www.revenue.nh.gov/forms/2023/documents/dp-10-2022-print.pdfhttps://www.revenue.nh.gov/sites/g/files/ehbemt736/files/documents/dp-10-2022-print.pdfhttps://www.revenue.nh.gov/forms/2024/documents/dp-10-instructions-2023.pdfhttps://www.revenue.nh.gov/sites/g/files/ehbemt736/files/documents/dp-10-instructions-2023.pdfhttps://www.revenue.nh.gov/sites/g/files/ehbemt736/files/documents/dp-10-2023_pro.pdfhttps://www.revenue.nh.gov/sites/g/files/ehbemt736/files/documents/dp-10-2023-form.pdfhttps://www.revenue.nh.gov/sites/g/files/ehbemt736/files/documents/dp-10-instructions-2023_0.pdfhttps://www.revenue.nh.gov/sites/g/files/ehbemt736/files/documents/dp-10-instructions-2023.pdfhttps://www.revenue.nh.gov/sites/g/files/ehbemt736/files/documents/ed-02-2023_PRO.pdfhttps://www.revenue.nh.gov/sites/g/files/ehbemt736/files/documents/ed-02-form.pdfhttps://www.romney.senate.gov/wp-content/uploads/2022/06/family-security-act-2.0_one-pager_appendix.pdfhttps://web.archive.org/web/20250101020512/https://www.romney.senate.gov/wp-content/uploads/2022/06/family-security-act-2.0_one-pager_appendix.pdfhttps://www.romney.senate.gov/wp-content/uploads/2024/09/FSA-Scanned-and-Final.pdfhttps://www.govinfo.gov/content/pkg/BILLS-118s5256is/pdf/BILLS-118s5256is.pdfhttps://www.sos.state.tx.us/texreg/archive/December272024/Adopted%20Rules/10.COMMUNITY%20DEVELOPMENT.htmlhttps://web.archive.org/web/20250625061143/https://www.sos.state.tx.us/texreg/archive/December272024/Adopted%20Rules/10.COMMUNITY%20DEVELOPMENT.htmlhttps://www2.illinois.gov/rev/forms/incometax/Documents/currentyear/individual/il-1040-schedule-m.pdfhttps://taxarchive.illinois.gov/content/dam/soi/en/web/taxarchive/forms/income-tax/2019/individual/il-1040-schedule-m.pdfhttps://www2.illinois.gov/rev/programs/EIC/Pages/default.aspxhttps://tax.illinois.gov/programs/eitc.htmlhttps://www2.illinois.gov/rev/research/taxrates/Pages/income.aspxhttps://tax.illinois.gov/research/taxrates/income.htmlhttps://www3.montgomerycountymd.gov/311/Solutions.aspx?SolutionId=1-4DAM0Ihttps://www2.montgomerycountymd.gov/mcgportalapps/Press_Detail.aspx?Item_ID=44684https://wwwa.fileyourtaxes.com/www17/cgi-bin/static/displayBlankPDF.pl?form=MT-FORM2.pdf&year=2023https://revenue.mt.gov/files/Forms/Montana-Individual-Income-Tax-Return-Form-2/2023_Montana_Individual_Income_Tax_Return_Form_2.pdfLeft unchanged
https://okrules.elaws.us/oac/340:40-5-1,340:40-7-1,340:40-7-3,340:40-7-8,340:40-7-12,340:40-7-13(20 citing files): dropped after review. Five candidates were the Cornell LII mirror. The sixth was an archive of a different OKDHS URL. Neither is the issuing agency or a capture of the dead URL. OKDHS's policy library paths now return 404. The official code at rules.ok.gov is a single-page app, and no stable per-section link to it was verified. These citations and their guard entries stay as on main for a follow-up.These URLs are not changed here; each needs more than a URL swap or turned out not to be dead.
http://epolicy.dpss.lacounty.gov/epolicy/epolicy/server/general/projects_responsive/ePolicyMaster/index.htm?&area=general&type=responsivehelp&ctxid=&project=ePolicyMaster#t=mergedProjects%2FCalWORKs%2FCalWORKs%2F44-101_Income_Definitions%2F44-101_Income_Definitions.htm%23Definitionsbc-4&rhtocid=_3_1_6_0_3: adjudicated unresolved: DPSS's CalWORKs 44-101 'Income Definitions' page is not live or archived. CDSS MPP 44-101 (10EAS.pdf p.9) matches the content but is another agency's instrument. Harmonize with the https #Policybc-2 vhttps://revenue.iowa.gov/2023-changes-iowa-individual-income-tax: mechanical check: 200 text/html; charset=UTF-8https://www.cthousedems.com/2026/02/04/hb-5134/: unresolved (The URL looks fabricated. cthousedems.com has no WHOIS registration, and Wayback CDX has zero captures anywhere on the domain (matchType=domain, 2026-10-01). Th)https://www.oregon.gov/dor/programs/businesses/Pages/: not-dead (False positive from URL extraction. or_lane_transit_district_total_payroll_tax.py lines 14-17 set reference = ("https://www.oregon.gov/dor/programs/businesses/P)https://www.twc.texas.gov/programs/child-care-services: adjudicated unresolved: The URL probably never existed, so no original document can be identified. The landing page lacks the age groups, and the BCY25 rate PDFs are different documents. Drop the reference; the variable alreNot listed in the guard
http://epolicy.dpss.lacounty.gov/epolicy/epolicy/server/general/projects_responsive/ePolicyMaster/index.htm?&area=general&type=responsivehelp&ctxid=&project=ePolicyMaster: still matches repo text after the rewrite (policyengine_us/variables/gov/states/ca/cdss/tanf/cash/income/ca_tanf_earned_income.py: http://epolicy.dpss.lacounty.gov/epolicy/epolicy/server/general/projects).http://legis.la.gov/Legis/Law.aspx?d=101769: replacement differs only in scheme or www., which the guard treats as the same URL.http://legis.la.gov/Legis/Law.aspx?d=453233: replacement differs only in scheme or www., which the guard treats as the same URL.http://www.legis.la.gov/legis/Law.aspx?d=101760: replacement differs only in scheme or www., which the guard treats as the same URL.https://capriverside.org/sites/g/files/aldnop136/files/2024-12/2025%20LIHEAP%20CAP%20APPLICATION%20ENGLISH.pdf: replacement differs only in scheme or www., which the guard treats as the same URL.https://epolicy.dpss.lacounty.gov/epolicy/epolicy/server/general/projects_responsive/ePolicyMaster/index.htm?&area=general&type=responsivehelp&ctxid=&project=ePolicyMaster: still matches repo text after the rewrite (policyengine_us/variables/gov/states/ca/cdss/tanf/cash/income/ca_tanf_earned_income.py: http://epolicy.dpss.lacounty.gov/epolicy/epolicy/server/general/projects).https://tax.idaho.gov/wp-content/uploads/forms/EFO00088/: replacement differs only in scheme or www., which the guard treats as the same URL.https://www.huduser.gov/hudapi/: still matches repo text after the rewrite (policyengine_us/tools/download_hud_fmr.py: https://www.huduser.gov/hudapi/).Pre-existing citation gaps (not fixed here)
The replacement keeps the original document and page, but the cited page does not show every value the file uses. These need a separate look because one URL is shared by files that cite different pages.
http://epolicy.dpss.lacounty.gov/epolicy/epolicy/server/general/projects_responsive/ePolicyMaster/index.htm?&area=general&type=responsivehelp&ctxid=&project=ePolicyMaster#t=mergedProjects%2FChild%20Care%2FChild_Care%2F1210_8_Regional_Market_Rate_Ceilings%2F1210_8_Regional_Market_Rate_Ceilings.htm%23Contactbc-13&rhtocid=_3_3_8_12: Same AR and release on DPSS's current site. It covers the 11 files citing factors, age categories and time rules. The standard/{younger,middle,older} 'Exhibit I' dollar ceilings are in neither page'sChanges after review
The independent review (GPT-6.1 Sol, tier hard) of
0609268e96requested changes. Commit16770c8179addresses them:va/dss/ccsp/copay/income_cap_rate.yaml(5% from 2025-07-01) cited a 2026-session HB30 amendment for FY2027–28. It now cites the enacted 2025 Appropriation Act (HB1600, Chapter 725), Item 125.10. Paragraph B.4 reads: "Family copayment rates for fiscal year 2026 shall be ... up to 5% of annual income ... with no household exceeding 5% of their income." The title is changed to match.ut/.../military_retirement/rate.yaml(2025) now cites page 24, the Military Retirement Credit Worksheet ("Credit percentage - 4.5%"). The Social Security files keep page 23.Verification
pytest policyengine_us/tests/code_health/test_known_dead_reference_urls.pypasses locally, including the new entries.ruff format --check .(locked ruff 0.15.5) passes.axiom: n/a: reference-only
🤖 Generated with Claude Code