diff --git a/claude-plugins/sent/skills/messaging-performance-analyzer/SKILL.md b/claude-plugins/sent/skills/messaging-performance-analyzer/SKILL.md index 3898887..6776c59 100644 --- a/claude-plugins/sent/skills/messaging-performance-analyzer/SKILL.md +++ b/claude-plugins/sent/skills/messaging-performance-analyzer/SKILL.md @@ -140,8 +140,3 @@ Use the `sent` skill for shared Sent terminology and routing. | `scripts/analyze_mdr_funnel.py` | Validation script | Reads an MDR export (CSV or JSON), groups channel × direction outcomes, separates delivery transitions from engagement, and retains malformed/unknown rows. Run from the skill root: `python scripts/analyze_mdr_funnel.py path/to/mdr.csv` (use `--threshold N`, `--show-errors`, or `--format json`). Exit `0` means no observed transition breach, `2` means bad input/no usable cohort, and `3` means an observed breach. JSON uses `null` where a denominator is unavailable; text uses `N/A`. | | `scripts/fixtures/good.json` | Fixture | Synthetic healthy-funnel MDR export. | | `scripts/fixtures/bad.json` | Fixture | Synthetic MDR export with deliberate >50% SENT→DELIVERED drop. | - -## Unverified claims to confirm or remove - -- Any fixed cohort-size threshold such as “1,000 messages minimum” is an analyst heuristic, not a documented Sent API rule. -- External provider identifiers such as carrier message IDs, WhatsApp `wamid`, and RCS message IDs are not in the v3 docs as join keys; use Sent `message_id` and treat provider IDs as escalation-only context. diff --git a/claude-plugins/sent/skills/messaging-performance-analyzer/references/mdr-status-codes.md b/claude-plugins/sent/skills/messaging-performance-analyzer/references/mdr-status-codes.md index 44d8175..5d18234 100644 --- a/claude-plugins/sent/skills/messaging-performance-analyzer/references/mdr-status-codes.md +++ b/claude-plugins/sent/skills/messaging-performance-analyzer/references/mdr-status-codes.md @@ -3,10 +3,22 @@ Supporting reference for `messaging-performance-analyzer`. "MDR" is the human term for Sent's per-message status stream; the v3 surfaces are `GET /v3/messages/{id}` and `GET /v3/messages/{id}/activities`. The codes below are Sent's own normalized catalog as documented at docs.sent.dm — not raw provider codes. Authoritative upstream sources (for the downstream provider codes that may appear in `error.details` after Sent normalization): + - WhatsApp: [Cloud API Error Codes](https://developers.facebook.com/docs/whatsapp/cloud-api/support/error-codes) - SMS: TCR + carrier-specific reject reasons (T-Mobile, AT&T, Verizon each publish their own list) - RCS: [RBM API errors](https://developers.google.com/business-communications/rcs-business-messaging/reference/rest) +## Table of contents + +- [Message status lifecycle](#message-status-lifecycle) +- [Synchronous errors](#synchronous-errors-http-response-body) +- [Send-time per-message errors](#send-time-per-message-errors) +- [Webhook payload shape](#webhook-payload-shape) +- [Webhook event lifecycle](#webhook-event-lifecycle) +- [Provider-level codes](#provider-level-codes-referential-not-sent-normalized) +- [Counting rules](#counting-rules) +- [Source notes](#source-notes) + ## Message status lifecycle Sent normalizes channels into a shared delivery state machine. The latest status per `message_id` determines its current outcome, while an activity history supplies the evidence for transition counts. Do not backfill earlier stages from a latest-only record. @@ -214,3 +226,10 @@ Carriers don't share an enum; the categories you actually need to triage on: - **Stop SMS at `DELIVERED`.** Calculate `READ` engagement only for WhatsApp and RCS. - **Separate channel fan-out.** `POST /v3/messages` with `"channel": ["sms","whatsapp","rcs"]` creates one message per channel; each has its own `message_id` and its own lifecycle. Don't double-count at the recipient level unless the user explicitly asks for recipient-level rollup. - **Honor a minimum cohort size** before drawing conclusions about small rate shifts. A working heuristic is ≥1,000 messages per cohort; below that, noise dominates. This is an analyst rule of thumb, not a Sent API rule. + +## Source notes + +- Sent lifecycle, error-envelope, template, message-activity, and webhook claims were last checked on 2026-08-09 against the [Sent v3 OpenAPI](https://api.sent.dm/swagger/v3/swagger.json), [message status guide](https://docs.sent.dm/llms/start/guides/message-status-tracking.txt), and [webhook event reference](https://docs.sent.dm/llms/start/webhooks/event-types.txt). +- Provider-code tables are referential aids sourced from the linked provider documentation; they are not Sent-normalized enums. SMS carriers do not publish one shared reject enum. +- The 1,000-message cohort threshold is an analyst heuristic, not a documented Sent API requirement. +- Provider identifiers such as carrier message IDs, WhatsApp `wamid`, and RCS message IDs are not documented Sent v3 join keys. Use Sent `message_id` and treat provider IDs as escalation context. diff --git a/claude-plugins/sent/skills/messaging-performance-analyzer/references/performance-diagnosis-playbook.md b/claude-plugins/sent/skills/messaging-performance-analyzer/references/performance-diagnosis-playbook.md index a8b5787..1e5a7ba 100644 --- a/claude-plugins/sent/skills/messaging-performance-analyzer/references/performance-diagnosis-playbook.md +++ b/claude-plugins/sent/skills/messaging-performance-analyzer/references/performance-diagnosis-playbook.md @@ -4,6 +4,14 @@ Supporting reference for `messaging-performance-analyzer`. The SKILL.md tells yo For the full catalog of codes referenced below, see `references/mdr-status-codes.md`. +## Table of contents + +- [Symptom-driven diagnosis](#symptom-driven-diagnosis) +- [Cross-skill handoff matrix](#cross-skill-handoff-matrix) +- [When to escalate to Sent support](#when-to-escalate-to-sent-support) +- [Diagnostic loop](#diagnostic-loop) +- [Source notes](#source-notes) + ## Symptom-driven diagnosis Every entry follows the same pattern: **observable symptom -> where the failure code lives -> what to check first -> handoff if rooted elsewhere.** @@ -137,3 +145,9 @@ Repeat until the symptom is explained or scoped: 7. If the symptom is missing customer-side data: prove webhook health via `is_active`, `consecutive_failures`, and `/v3/webhooks/{id}/events` before blaming delivery. 8. Hand off via the matrix above, or escalate to Sent support with the required evidence. 9. Quantify the diagnosis — never "looks better now" without a recomputed funnel. + +## Source notes + +- This playbook is operational guidance synthesized from the [Sent v3 OpenAPI](https://api.sent.dm/swagger/v3/swagger.json), [message status guide](https://docs.sent.dm/llms/start/guides/message-status-tracking.txt), [webhook event reference](https://docs.sent.dm/llms/start/webhooks/event-types.txt), and [channel-routing reference](https://docs.sent.dm/llms/reference/channel-routing.txt), last checked on 2026-08-09. +- Dominance thresholds, cohort-size guidance, comparison windows, and escalation timing are analyst heuristics unless a cited Sent source states otherwise. +- Re-check the repository documentation source catalog before relying on exact endpoints, enums, or limits after its `last_verified` date. diff --git a/packages/sent/skills/messaging-performance-analyzer/SKILL.md b/packages/sent/skills/messaging-performance-analyzer/SKILL.md index 3898887..6776c59 100644 --- a/packages/sent/skills/messaging-performance-analyzer/SKILL.md +++ b/packages/sent/skills/messaging-performance-analyzer/SKILL.md @@ -140,8 +140,3 @@ Use the `sent` skill for shared Sent terminology and routing. | `scripts/analyze_mdr_funnel.py` | Validation script | Reads an MDR export (CSV or JSON), groups channel × direction outcomes, separates delivery transitions from engagement, and retains malformed/unknown rows. Run from the skill root: `python scripts/analyze_mdr_funnel.py path/to/mdr.csv` (use `--threshold N`, `--show-errors`, or `--format json`). Exit `0` means no observed transition breach, `2` means bad input/no usable cohort, and `3` means an observed breach. JSON uses `null` where a denominator is unavailable; text uses `N/A`. | | `scripts/fixtures/good.json` | Fixture | Synthetic healthy-funnel MDR export. | | `scripts/fixtures/bad.json` | Fixture | Synthetic MDR export with deliberate >50% SENT→DELIVERED drop. | - -## Unverified claims to confirm or remove - -- Any fixed cohort-size threshold such as “1,000 messages minimum” is an analyst heuristic, not a documented Sent API rule. -- External provider identifiers such as carrier message IDs, WhatsApp `wamid`, and RCS message IDs are not in the v3 docs as join keys; use Sent `message_id` and treat provider IDs as escalation-only context. diff --git a/packages/sent/skills/messaging-performance-analyzer/references/mdr-status-codes.md b/packages/sent/skills/messaging-performance-analyzer/references/mdr-status-codes.md index 44d8175..5d18234 100644 --- a/packages/sent/skills/messaging-performance-analyzer/references/mdr-status-codes.md +++ b/packages/sent/skills/messaging-performance-analyzer/references/mdr-status-codes.md @@ -3,10 +3,22 @@ Supporting reference for `messaging-performance-analyzer`. "MDR" is the human term for Sent's per-message status stream; the v3 surfaces are `GET /v3/messages/{id}` and `GET /v3/messages/{id}/activities`. The codes below are Sent's own normalized catalog as documented at docs.sent.dm — not raw provider codes. Authoritative upstream sources (for the downstream provider codes that may appear in `error.details` after Sent normalization): + - WhatsApp: [Cloud API Error Codes](https://developers.facebook.com/docs/whatsapp/cloud-api/support/error-codes) - SMS: TCR + carrier-specific reject reasons (T-Mobile, AT&T, Verizon each publish their own list) - RCS: [RBM API errors](https://developers.google.com/business-communications/rcs-business-messaging/reference/rest) +## Table of contents + +- [Message status lifecycle](#message-status-lifecycle) +- [Synchronous errors](#synchronous-errors-http-response-body) +- [Send-time per-message errors](#send-time-per-message-errors) +- [Webhook payload shape](#webhook-payload-shape) +- [Webhook event lifecycle](#webhook-event-lifecycle) +- [Provider-level codes](#provider-level-codes-referential-not-sent-normalized) +- [Counting rules](#counting-rules) +- [Source notes](#source-notes) + ## Message status lifecycle Sent normalizes channels into a shared delivery state machine. The latest status per `message_id` determines its current outcome, while an activity history supplies the evidence for transition counts. Do not backfill earlier stages from a latest-only record. @@ -214,3 +226,10 @@ Carriers don't share an enum; the categories you actually need to triage on: - **Stop SMS at `DELIVERED`.** Calculate `READ` engagement only for WhatsApp and RCS. - **Separate channel fan-out.** `POST /v3/messages` with `"channel": ["sms","whatsapp","rcs"]` creates one message per channel; each has its own `message_id` and its own lifecycle. Don't double-count at the recipient level unless the user explicitly asks for recipient-level rollup. - **Honor a minimum cohort size** before drawing conclusions about small rate shifts. A working heuristic is ≥1,000 messages per cohort; below that, noise dominates. This is an analyst rule of thumb, not a Sent API rule. + +## Source notes + +- Sent lifecycle, error-envelope, template, message-activity, and webhook claims were last checked on 2026-08-09 against the [Sent v3 OpenAPI](https://api.sent.dm/swagger/v3/swagger.json), [message status guide](https://docs.sent.dm/llms/start/guides/message-status-tracking.txt), and [webhook event reference](https://docs.sent.dm/llms/start/webhooks/event-types.txt). +- Provider-code tables are referential aids sourced from the linked provider documentation; they are not Sent-normalized enums. SMS carriers do not publish one shared reject enum. +- The 1,000-message cohort threshold is an analyst heuristic, not a documented Sent API requirement. +- Provider identifiers such as carrier message IDs, WhatsApp `wamid`, and RCS message IDs are not documented Sent v3 join keys. Use Sent `message_id` and treat provider IDs as escalation context. diff --git a/packages/sent/skills/messaging-performance-analyzer/references/performance-diagnosis-playbook.md b/packages/sent/skills/messaging-performance-analyzer/references/performance-diagnosis-playbook.md index a8b5787..1e5a7ba 100644 --- a/packages/sent/skills/messaging-performance-analyzer/references/performance-diagnosis-playbook.md +++ b/packages/sent/skills/messaging-performance-analyzer/references/performance-diagnosis-playbook.md @@ -4,6 +4,14 @@ Supporting reference for `messaging-performance-analyzer`. The SKILL.md tells yo For the full catalog of codes referenced below, see `references/mdr-status-codes.md`. +## Table of contents + +- [Symptom-driven diagnosis](#symptom-driven-diagnosis) +- [Cross-skill handoff matrix](#cross-skill-handoff-matrix) +- [When to escalate to Sent support](#when-to-escalate-to-sent-support) +- [Diagnostic loop](#diagnostic-loop) +- [Source notes](#source-notes) + ## Symptom-driven diagnosis Every entry follows the same pattern: **observable symptom -> where the failure code lives -> what to check first -> handoff if rooted elsewhere.** @@ -137,3 +145,9 @@ Repeat until the symptom is explained or scoped: 7. If the symptom is missing customer-side data: prove webhook health via `is_active`, `consecutive_failures`, and `/v3/webhooks/{id}/events` before blaming delivery. 8. Hand off via the matrix above, or escalate to Sent support with the required evidence. 9. Quantify the diagnosis — never "looks better now" without a recomputed funnel. + +## Source notes + +- This playbook is operational guidance synthesized from the [Sent v3 OpenAPI](https://api.sent.dm/swagger/v3/swagger.json), [message status guide](https://docs.sent.dm/llms/start/guides/message-status-tracking.txt), [webhook event reference](https://docs.sent.dm/llms/start/webhooks/event-types.txt), and [channel-routing reference](https://docs.sent.dm/llms/reference/channel-routing.txt), last checked on 2026-08-09. +- Dominance thresholds, cohort-size guidance, comparison windows, and escalation timing are analyst heuristics unless a cited Sent source states otherwise. +- Re-check the repository documentation source catalog before relying on exact endpoints, enums, or limits after its `last_verified` date. diff --git a/plugins/sent/skills/messaging-performance-analyzer/SKILL.md b/plugins/sent/skills/messaging-performance-analyzer/SKILL.md index 3898887..6776c59 100644 --- a/plugins/sent/skills/messaging-performance-analyzer/SKILL.md +++ b/plugins/sent/skills/messaging-performance-analyzer/SKILL.md @@ -140,8 +140,3 @@ Use the `sent` skill for shared Sent terminology and routing. | `scripts/analyze_mdr_funnel.py` | Validation script | Reads an MDR export (CSV or JSON), groups channel × direction outcomes, separates delivery transitions from engagement, and retains malformed/unknown rows. Run from the skill root: `python scripts/analyze_mdr_funnel.py path/to/mdr.csv` (use `--threshold N`, `--show-errors`, or `--format json`). Exit `0` means no observed transition breach, `2` means bad input/no usable cohort, and `3` means an observed breach. JSON uses `null` where a denominator is unavailable; text uses `N/A`. | | `scripts/fixtures/good.json` | Fixture | Synthetic healthy-funnel MDR export. | | `scripts/fixtures/bad.json` | Fixture | Synthetic MDR export with deliberate >50% SENT→DELIVERED drop. | - -## Unverified claims to confirm or remove - -- Any fixed cohort-size threshold such as “1,000 messages minimum” is an analyst heuristic, not a documented Sent API rule. -- External provider identifiers such as carrier message IDs, WhatsApp `wamid`, and RCS message IDs are not in the v3 docs as join keys; use Sent `message_id` and treat provider IDs as escalation-only context. diff --git a/plugins/sent/skills/messaging-performance-analyzer/references/mdr-status-codes.md b/plugins/sent/skills/messaging-performance-analyzer/references/mdr-status-codes.md index 44d8175..5d18234 100644 --- a/plugins/sent/skills/messaging-performance-analyzer/references/mdr-status-codes.md +++ b/plugins/sent/skills/messaging-performance-analyzer/references/mdr-status-codes.md @@ -3,10 +3,22 @@ Supporting reference for `messaging-performance-analyzer`. "MDR" is the human term for Sent's per-message status stream; the v3 surfaces are `GET /v3/messages/{id}` and `GET /v3/messages/{id}/activities`. The codes below are Sent's own normalized catalog as documented at docs.sent.dm — not raw provider codes. Authoritative upstream sources (for the downstream provider codes that may appear in `error.details` after Sent normalization): + - WhatsApp: [Cloud API Error Codes](https://developers.facebook.com/docs/whatsapp/cloud-api/support/error-codes) - SMS: TCR + carrier-specific reject reasons (T-Mobile, AT&T, Verizon each publish their own list) - RCS: [RBM API errors](https://developers.google.com/business-communications/rcs-business-messaging/reference/rest) +## Table of contents + +- [Message status lifecycle](#message-status-lifecycle) +- [Synchronous errors](#synchronous-errors-http-response-body) +- [Send-time per-message errors](#send-time-per-message-errors) +- [Webhook payload shape](#webhook-payload-shape) +- [Webhook event lifecycle](#webhook-event-lifecycle) +- [Provider-level codes](#provider-level-codes-referential-not-sent-normalized) +- [Counting rules](#counting-rules) +- [Source notes](#source-notes) + ## Message status lifecycle Sent normalizes channels into a shared delivery state machine. The latest status per `message_id` determines its current outcome, while an activity history supplies the evidence for transition counts. Do not backfill earlier stages from a latest-only record. @@ -214,3 +226,10 @@ Carriers don't share an enum; the categories you actually need to triage on: - **Stop SMS at `DELIVERED`.** Calculate `READ` engagement only for WhatsApp and RCS. - **Separate channel fan-out.** `POST /v3/messages` with `"channel": ["sms","whatsapp","rcs"]` creates one message per channel; each has its own `message_id` and its own lifecycle. Don't double-count at the recipient level unless the user explicitly asks for recipient-level rollup. - **Honor a minimum cohort size** before drawing conclusions about small rate shifts. A working heuristic is ≥1,000 messages per cohort; below that, noise dominates. This is an analyst rule of thumb, not a Sent API rule. + +## Source notes + +- Sent lifecycle, error-envelope, template, message-activity, and webhook claims were last checked on 2026-08-09 against the [Sent v3 OpenAPI](https://api.sent.dm/swagger/v3/swagger.json), [message status guide](https://docs.sent.dm/llms/start/guides/message-status-tracking.txt), and [webhook event reference](https://docs.sent.dm/llms/start/webhooks/event-types.txt). +- Provider-code tables are referential aids sourced from the linked provider documentation; they are not Sent-normalized enums. SMS carriers do not publish one shared reject enum. +- The 1,000-message cohort threshold is an analyst heuristic, not a documented Sent API requirement. +- Provider identifiers such as carrier message IDs, WhatsApp `wamid`, and RCS message IDs are not documented Sent v3 join keys. Use Sent `message_id` and treat provider IDs as escalation context. diff --git a/plugins/sent/skills/messaging-performance-analyzer/references/performance-diagnosis-playbook.md b/plugins/sent/skills/messaging-performance-analyzer/references/performance-diagnosis-playbook.md index a8b5787..1e5a7ba 100644 --- a/plugins/sent/skills/messaging-performance-analyzer/references/performance-diagnosis-playbook.md +++ b/plugins/sent/skills/messaging-performance-analyzer/references/performance-diagnosis-playbook.md @@ -4,6 +4,14 @@ Supporting reference for `messaging-performance-analyzer`. The SKILL.md tells yo For the full catalog of codes referenced below, see `references/mdr-status-codes.md`. +## Table of contents + +- [Symptom-driven diagnosis](#symptom-driven-diagnosis) +- [Cross-skill handoff matrix](#cross-skill-handoff-matrix) +- [When to escalate to Sent support](#when-to-escalate-to-sent-support) +- [Diagnostic loop](#diagnostic-loop) +- [Source notes](#source-notes) + ## Symptom-driven diagnosis Every entry follows the same pattern: **observable symptom -> where the failure code lives -> what to check first -> handoff if rooted elsewhere.** @@ -137,3 +145,9 @@ Repeat until the symptom is explained or scoped: 7. If the symptom is missing customer-side data: prove webhook health via `is_active`, `consecutive_failures`, and `/v3/webhooks/{id}/events` before blaming delivery. 8. Hand off via the matrix above, or escalate to Sent support with the required evidence. 9. Quantify the diagnosis — never "looks better now" without a recomputed funnel. + +## Source notes + +- This playbook is operational guidance synthesized from the [Sent v3 OpenAPI](https://api.sent.dm/swagger/v3/swagger.json), [message status guide](https://docs.sent.dm/llms/start/guides/message-status-tracking.txt), [webhook event reference](https://docs.sent.dm/llms/start/webhooks/event-types.txt), and [channel-routing reference](https://docs.sent.dm/llms/reference/channel-routing.txt), last checked on 2026-08-09. +- Dominance thresholds, cohort-size guidance, comparison windows, and escalation timing are analyst heuristics unless a cited Sent source states otherwise. +- Re-check the repository documentation source catalog before relying on exact endpoints, enums, or limits after its `last_verified` date. diff --git a/schemas/sent/documentation-sources.json b/schemas/sent/documentation-sources.json new file mode 100644 index 0000000..351a6ff --- /dev/null +++ b/schemas/sent/documentation-sources.json @@ -0,0 +1,74 @@ +{ + "schema_version": 1, + "sources": [ + { + "id": "llms-index", + "url": "https://docs.sent.dm/llms.txt", + "kind": "documentation-index", + "affected_skills": ["sent"], + "last_verified": "2026-08-09" + }, + { + "id": "openapi", + "url": "https://api.sent.dm/swagger/v3/swagger.json", + "kind": "openapi", + "affected_skills": [ + "sent-account-readiness", + "sent-analytics", + "sent-contacts", + "sent-messaging", + "sent-templates", + "sms-10dlc-registration", + "waba-embedded-signup", + "waba-template-author" + ], + "last_verified": "2026-08-09" + }, + { + "id": "templates", + "url": "https://docs.sent.dm/llms/reference/api/template-definition.txt", + "kind": "documentation", + "affected_skills": ["sent-templates", "template-builder-ui", "waba-template-author"], + "last_verified": "2026-08-09" + }, + { + "id": "webhooks", + "url": "https://docs.sent.dm/llms/start/webhooks/event-types.txt", + "kind": "documentation", + "affected_skills": [ + "messaging-performance-analyzer", + "sender-profile-architect", + "waba-embedded-signup" + ], + "last_verified": "2026-08-09" + }, + { + "id": "sender-profiles", + "url": "https://docs.sent.dm/llms/start/concepts/sender-profiles.txt", + "kind": "documentation", + "affected_skills": ["sender-profile-architect", "sms-10dlc-registration", "waba-embedded-signup"], + "last_verified": "2026-08-09" + }, + { + "id": "waba", + "url": "https://docs.sent.dm/llms/start/guides/connect-whatsapp.txt", + "kind": "documentation", + "affected_skills": ["waba-embedded-signup", "waba-template-author"], + "last_verified": "2026-08-09" + }, + { + "id": "routing-rcs", + "url": "https://docs.sent.dm/llms/reference/channel-routing.txt", + "kind": "documentation", + "affected_skills": ["messaging-performance-analyzer", "rcs-agent-onboarding", "sent-messaging"], + "last_verified": "2026-08-09" + }, + { + "id": "10dlc", + "url": "https://docs.sent.dm/llms/start/advanced/10dlc-registration.txt", + "kind": "documentation", + "affected_skills": ["sms-10dlc-registration"], + "last_verified": "2026-08-09" + } + ] +} diff --git a/scripts/test_contracts.py b/scripts/test_contracts.py index 311ec22..9bc822e 100644 --- a/scripts/test_contracts.py +++ b/scripts/test_contracts.py @@ -3,6 +3,7 @@ from __future__ import annotations +import datetime import importlib.util import json import re @@ -122,6 +123,31 @@ def test_marketplace_prompts_include_specialists(self) -> None: self.assertEqual(catalog["website_url"], "https://github.com/sentdm/sent-plugin#readme") +class DocumentationSourceContractTests(unittest.TestCase): + def test_source_catalog_covers_freshness_domains(self) -> None: + metadata = REPOSITORY_METADATA.load_repository_metadata(ROOT) + path = ROOT / "schemas" / "sent" / "documentation-sources.json" + catalog = json.loads(path.read_text(encoding="utf-8")) + sources = {source["id"]: source for source in catalog["sources"]} + self.assertEqual( + set(sources), + { + "llms-index", + "openapi", + "templates", + "webhooks", + "sender-profiles", + "waba", + "routing-rcs", + "10dlc", + }, + ) + for source in sources.values(): + self.assertTrue(source["url"].startswith("https://")) + self.assertTrue(set(source["affected_skills"]) <= set(metadata.skills)) + datetime.date.fromisoformat(source["last_verified"]) + + class BundledExampleTests(unittest.TestCase): def test_every_markdown_json_block_parses(self) -> None: for path in SKILLS.rglob("*.md"): diff --git a/scripts/test_validation_gates.py b/scripts/test_validation_gates.py index ef11b05..1640a3f 100644 --- a/scripts/test_validation_gates.py +++ b/scripts/test_validation_gates.py @@ -161,6 +161,37 @@ def mutation_annotation_drift(root: Path) -> None: path.write_text(json.dumps(submission, indent=2) + "\n", encoding="utf-8") +def missing_long_reference_toc(root: Path) -> None: + path = root / "packages" / "sent" / "skills" / "messaging-performance-analyzer" / "references" / "mdr-status-codes.md" + content = path.read_text(encoding="utf-8") + path.write_text(content.replace("## Table of contents", "## Navigation", 1), encoding="utf-8") + + +def broken_reference_anchor(root: Path) -> None: + path = root / "packages" / "sent" / "skills" / "messaging-performance-analyzer" / "references" / "performance-diagnosis-playbook.md" + with path.open("a", encoding="utf-8") as handle: + handle.write("\n[Broken navigation](#missing-section)\n") + + +def deprecated_reference_heading(root: Path) -> None: + path = root / "packages" / "sent" / "skills" / "sent" / "references" / "sent-glossary.md" + with path.open("a", encoding="utf-8") as handle: + handle.write("\n## Suggested bundled references\n") + + +def unresolved_bundled_resource(root: Path) -> None: + path = root / "packages" / "sent" / "skills" / "messaging-performance-analyzer" / "SKILL.md" + content = path.read_text(encoding="utf-8") + path.write_text(content.replace("scripts/fixtures/good.json", "scripts/fixtures/missing.json"), encoding="utf-8") + + +def invalid_documentation_source_skill(root: Path) -> None: + path = root / "schemas" / "sent" / "documentation-sources.json" + catalog = json.loads(path.read_text(encoding="utf-8")) + catalog["sources"][0]["affected_skills"] = ["missing-skill"] + path.write_text(json.dumps(catalog, indent=2) + "\n", encoding="utf-8") + + def retired_brand_endpoint(root: Path) -> None: path = root / "packages" / "sent" / "skills" / "sms-10dlc-registration" / "SKILL.md" with path.open("a", encoding="utf-8") as handle: @@ -213,6 +244,11 @@ def contract_manifest_drift(root: Path) -> None: ("unexpected MCP tool", unexpected_public_tool, "OpenAI submission tool set mismatch"), ("invalid MCP tool owner", invalid_tool_owner, "is not a canonical skill"), ("mutation annotation consistency", mutation_annotation_drift, "public mutation class destructive"), + ("long reference TOC", missing_long_reference_toc, "requires linked table of contents"), + ("internal reference anchors", broken_reference_anchor, "broken internal anchor"), + ("deprecated reference headings", deprecated_reference_heading, "deprecated 'Suggested bundled...' title"), + ("bundled resource paths", unresolved_bundled_resource, "unresolved skill-local reference"), + ("documentation source skills", invalid_documentation_source_skill, "invalid affected_skills"), ("retired brand endpoint", retired_brand_endpoint, "retired endpoint"), ("template webhook envelope", template_sub_type, "template webhook example uses sub_type"), ("ordered channel fallback", ordered_channel_fallback, "explicit RCS/SMS array"), diff --git a/scripts/validate.py b/scripts/validate.py index 94bf6f5..7acafce 100644 --- a/scripts/validate.py +++ b/scripts/validate.py @@ -3,11 +3,13 @@ from __future__ import annotations +import datetime import json import re import shutil import subprocess import sys +from collections import Counter from collections.abc import Iterable from pathlib import Path from urllib.parse import urlparse @@ -28,6 +30,7 @@ SCHEMAS = ROOT / "schemas" / "agent-plugins" / "1.0.0" OPENAI_SUBMISSION_SCHEMA = ROOT / "schemas" / "openai" / "chatgpt-app-submission.v1.json" CONTRACT_MANIFEST = ROOT / "schemas" / "sent" / "v3-contract-manifest.json" +DOCUMENTATION_SOURCES = ROOT / "schemas" / "sent" / "documentation-sources.json" MCP_URL = "https://mcp.sent.dm/mcp" PLUGIN_SCHEMA_ID = "https://agent-plugins.org/schemas/1.0.0/plugin.schema.json" MCP_SCHEMA_ID = "https://agent-plugins.org/schemas/1.0.0/mcp.schema.json" @@ -97,6 +100,7 @@ FORBIDDEN_KEY_SUFFIXES = {".key", ".pem", ".p12", ".pfx"} LEGACY_PRIVATE_NAMESPACE = "sent" + "-ops-skills:" MARKDOWN_LINK = re.compile(r"\[[^\]]*\]\(([^)]+)\)") +MARKDOWN_HEADING = re.compile(r"^(#{1,6})\s+(.+?)\s*#*\s*$", re.MULTILINE) LOCAL_RESOURCE = re.compile(r"(? None: def validate_reference(skill_root: Path, target: str, source: Path, validation: Validation) -> None: target = target.strip().strip("`<>") - if not target or target.startswith("#"): + if not target: return parsed = urlparse(target) if parsed.scheme or target.startswith("//"): if parsed.scheme in {"http", "https"}: validation.check(parsed.scheme == "https", f"{source.relative_to(ROOT)}: external URLs must use HTTPS") return - clean = target.split("#", 1)[0].split("?", 1)[0] - if not clean: - return - candidate = Path(clean) - validation.check(not candidate.is_absolute(), f"{source.relative_to(ROOT)}: absolute path reference {target}") - validation.check(".." not in candidate.parts, f"{source.relative_to(ROOT)}: sibling/root path reference {target}") - if candidate.is_absolute() or ".." in candidate.parts: - return - resolved = (skill_root / candidate).resolve() + path_target, _, anchor = target.partition("#") + clean = path_target.split("?", 1)[0] + if clean: + candidate = Path(clean) + validation.check(not candidate.is_absolute(), f"{source.relative_to(ROOT)}: absolute path reference {target}") + validation.check(".." not in candidate.parts, f"{source.relative_to(ROOT)}: sibling/root path reference {target}") + if candidate.is_absolute() or ".." in candidate.parts: + return + resolved = (skill_root / candidate).resolve() + else: + resolved = source.resolve() root = skill_root.resolve() validation.check(root == resolved or root in resolved.parents, f"{source.relative_to(ROOT)}: path escapes skill root: {target}") validation.check(resolved.exists(), f"{source.relative_to(ROOT)}: unresolved skill-local reference {target}") + if anchor and resolved.is_file() and resolved.suffix.lower() == ".md": + anchors = markdown_anchors(resolved.read_text(encoding="utf-8")) + validation.check( + anchor in anchors, + f"{source.relative_to(ROOT)}: broken internal anchor #{anchor} in {target}", + ) + + +def heading_slug(title: str) -> str: + title = re.sub(r"[`*_~]", "", title.strip().casefold()) + title = re.sub(r"[^\w\s-]", "", title) + return re.sub(r"\s+", "-", title) + + +def markdown_anchors(text: str) -> set[str]: + anchors: set[str] = set() + occurrences: Counter[str] = Counter() + for _, title in MARKDOWN_HEADING.findall(text): + base = heading_slug(title) + count = occurrences[base] + occurrences[base] += 1 + anchors.add(base if count == 0 else f"{base}-{count}") + return anchors + + +def validate_reference_hygiene(validation: Validation) -> None: + for path in sorted(SKILLS.glob("*/references/**/*.md")): + text = path.read_text(encoding="utf-8") + headings = MARKDOWN_HEADING.findall(text) + validation.check( + bool(headings) and headings[0][0] == "#" and text.startswith("# "), + f"{path.relative_to(ROOT)}: reference must start with one H1 title", + ) + validation.check( + "suggested bundled" not in text.lower(), + f"{path.relative_to(ROOT)}: deprecated 'Suggested bundled...' title", + ) + if len(text.splitlines()) <= 100: + continue + validation.check( + "## Table of contents" in text, + f"{path.relative_to(ROOT)}: reference over 100 lines requires linked table of contents", + ) + toc_match = re.search( + r"^## Table of contents\s*$\n(.*?)(?=^##\s|\Z)", + text, + re.MULTILINE | re.DOTALL, + ) + if toc_match is None: + continue + toc = toc_match.group(1) + major_headings = [title for level, title in headings if level == "##" and title != "Table of contents"] + for title in major_headings: + anchor = heading_slug(title) + validation.check( + f"](#{anchor})" in toc, + f"{path.relative_to(ROOT)}: table of contents missing #{anchor}", + ) + + +def validate_documentation_sources(validation: Validation) -> None: + validation.check(DOCUMENTATION_SOURCES.is_file(), "missing documentation source catalog") + if not DOCUMENTATION_SOURCES.is_file(): + return + catalog = load_json(DOCUMENTATION_SOURCES) + validation.check(set(catalog) == {"schema_version", "sources"}, "documentation source catalog fields drifted") + validation.check(catalog.get("schema_version") == 1, "documentation source catalog schema_version must be 1") + sources = catalog.get("sources") + validation.check(isinstance(sources, list) and bool(sources), "documentation source catalog must contain sources") + if not isinstance(sources, list): + return + identifiers: set[str] = set() + for index, source in enumerate(sources, 1): + validation.check(isinstance(source, dict), f"documentation source {index} must be an object") + if not isinstance(source, dict): + continue + validation.check( + set(source) == {"id", "url", "kind", "affected_skills", "last_verified"}, + f"documentation source {index} fields drifted", + ) + identifier = source.get("id") + validation.check(isinstance(identifier, str) and bool(identifier), f"documentation source {index} id is required") + if isinstance(identifier, str): + validation.check(identifier not in identifiers, f"duplicate documentation source id {identifier}") + identifiers.add(identifier) + validation.check( + urlparse(str(source.get("url", ""))).scheme == "https", + f"documentation source {identifier} must use HTTPS", + ) + affected = source.get("affected_skills") + validation.check( + isinstance(affected, list) + and bool(affected) + and all(isinstance(name, str) for name in affected) + and set(affected) <= EXPECTED_SKILLS, + f"documentation source {identifier} has invalid affected_skills", + ) + last_verified = source.get("last_verified") + try: + datetime.date.fromisoformat(last_verified) if isinstance(last_verified, str) else None + valid_date = isinstance(last_verified, str) + except ValueError: + valid_date = False + validation.check(valid_date, f"documentation source {identifier} requires a valid last_verified date") def validate_skills(validation: Validation) -> None: @@ -884,6 +994,8 @@ def main() -> None: validate_root_discovery(validation) validate_containment(validation) validate_skills(validation) + validate_reference_hygiene(validation) + validate_documentation_sources(validation) validate_skill_ui_metadata(validation) validate_marketplace_content(validation) validate_public_content(validation) diff --git a/skills/messaging-performance-analyzer/SKILL.md b/skills/messaging-performance-analyzer/SKILL.md index 3898887..6776c59 100644 --- a/skills/messaging-performance-analyzer/SKILL.md +++ b/skills/messaging-performance-analyzer/SKILL.md @@ -140,8 +140,3 @@ Use the `sent` skill for shared Sent terminology and routing. | `scripts/analyze_mdr_funnel.py` | Validation script | Reads an MDR export (CSV or JSON), groups channel × direction outcomes, separates delivery transitions from engagement, and retains malformed/unknown rows. Run from the skill root: `python scripts/analyze_mdr_funnel.py path/to/mdr.csv` (use `--threshold N`, `--show-errors`, or `--format json`). Exit `0` means no observed transition breach, `2` means bad input/no usable cohort, and `3` means an observed breach. JSON uses `null` where a denominator is unavailable; text uses `N/A`. | | `scripts/fixtures/good.json` | Fixture | Synthetic healthy-funnel MDR export. | | `scripts/fixtures/bad.json` | Fixture | Synthetic MDR export with deliberate >50% SENT→DELIVERED drop. | - -## Unverified claims to confirm or remove - -- Any fixed cohort-size threshold such as “1,000 messages minimum” is an analyst heuristic, not a documented Sent API rule. -- External provider identifiers such as carrier message IDs, WhatsApp `wamid`, and RCS message IDs are not in the v3 docs as join keys; use Sent `message_id` and treat provider IDs as escalation-only context. diff --git a/skills/messaging-performance-analyzer/references/mdr-status-codes.md b/skills/messaging-performance-analyzer/references/mdr-status-codes.md index 44d8175..5d18234 100644 --- a/skills/messaging-performance-analyzer/references/mdr-status-codes.md +++ b/skills/messaging-performance-analyzer/references/mdr-status-codes.md @@ -3,10 +3,22 @@ Supporting reference for `messaging-performance-analyzer`. "MDR" is the human term for Sent's per-message status stream; the v3 surfaces are `GET /v3/messages/{id}` and `GET /v3/messages/{id}/activities`. The codes below are Sent's own normalized catalog as documented at docs.sent.dm — not raw provider codes. Authoritative upstream sources (for the downstream provider codes that may appear in `error.details` after Sent normalization): + - WhatsApp: [Cloud API Error Codes](https://developers.facebook.com/docs/whatsapp/cloud-api/support/error-codes) - SMS: TCR + carrier-specific reject reasons (T-Mobile, AT&T, Verizon each publish their own list) - RCS: [RBM API errors](https://developers.google.com/business-communications/rcs-business-messaging/reference/rest) +## Table of contents + +- [Message status lifecycle](#message-status-lifecycle) +- [Synchronous errors](#synchronous-errors-http-response-body) +- [Send-time per-message errors](#send-time-per-message-errors) +- [Webhook payload shape](#webhook-payload-shape) +- [Webhook event lifecycle](#webhook-event-lifecycle) +- [Provider-level codes](#provider-level-codes-referential-not-sent-normalized) +- [Counting rules](#counting-rules) +- [Source notes](#source-notes) + ## Message status lifecycle Sent normalizes channels into a shared delivery state machine. The latest status per `message_id` determines its current outcome, while an activity history supplies the evidence for transition counts. Do not backfill earlier stages from a latest-only record. @@ -214,3 +226,10 @@ Carriers don't share an enum; the categories you actually need to triage on: - **Stop SMS at `DELIVERED`.** Calculate `READ` engagement only for WhatsApp and RCS. - **Separate channel fan-out.** `POST /v3/messages` with `"channel": ["sms","whatsapp","rcs"]` creates one message per channel; each has its own `message_id` and its own lifecycle. Don't double-count at the recipient level unless the user explicitly asks for recipient-level rollup. - **Honor a minimum cohort size** before drawing conclusions about small rate shifts. A working heuristic is ≥1,000 messages per cohort; below that, noise dominates. This is an analyst rule of thumb, not a Sent API rule. + +## Source notes + +- Sent lifecycle, error-envelope, template, message-activity, and webhook claims were last checked on 2026-08-09 against the [Sent v3 OpenAPI](https://api.sent.dm/swagger/v3/swagger.json), [message status guide](https://docs.sent.dm/llms/start/guides/message-status-tracking.txt), and [webhook event reference](https://docs.sent.dm/llms/start/webhooks/event-types.txt). +- Provider-code tables are referential aids sourced from the linked provider documentation; they are not Sent-normalized enums. SMS carriers do not publish one shared reject enum. +- The 1,000-message cohort threshold is an analyst heuristic, not a documented Sent API requirement. +- Provider identifiers such as carrier message IDs, WhatsApp `wamid`, and RCS message IDs are not documented Sent v3 join keys. Use Sent `message_id` and treat provider IDs as escalation context. diff --git a/skills/messaging-performance-analyzer/references/performance-diagnosis-playbook.md b/skills/messaging-performance-analyzer/references/performance-diagnosis-playbook.md index a8b5787..1e5a7ba 100644 --- a/skills/messaging-performance-analyzer/references/performance-diagnosis-playbook.md +++ b/skills/messaging-performance-analyzer/references/performance-diagnosis-playbook.md @@ -4,6 +4,14 @@ Supporting reference for `messaging-performance-analyzer`. The SKILL.md tells yo For the full catalog of codes referenced below, see `references/mdr-status-codes.md`. +## Table of contents + +- [Symptom-driven diagnosis](#symptom-driven-diagnosis) +- [Cross-skill handoff matrix](#cross-skill-handoff-matrix) +- [When to escalate to Sent support](#when-to-escalate-to-sent-support) +- [Diagnostic loop](#diagnostic-loop) +- [Source notes](#source-notes) + ## Symptom-driven diagnosis Every entry follows the same pattern: **observable symptom -> where the failure code lives -> what to check first -> handoff if rooted elsewhere.** @@ -137,3 +145,9 @@ Repeat until the symptom is explained or scoped: 7. If the symptom is missing customer-side data: prove webhook health via `is_active`, `consecutive_failures`, and `/v3/webhooks/{id}/events` before blaming delivery. 8. Hand off via the matrix above, or escalate to Sent support with the required evidence. 9. Quantify the diagnosis — never "looks better now" without a recomputed funnel. + +## Source notes + +- This playbook is operational guidance synthesized from the [Sent v3 OpenAPI](https://api.sent.dm/swagger/v3/swagger.json), [message status guide](https://docs.sent.dm/llms/start/guides/message-status-tracking.txt), [webhook event reference](https://docs.sent.dm/llms/start/webhooks/event-types.txt), and [channel-routing reference](https://docs.sent.dm/llms/reference/channel-routing.txt), last checked on 2026-08-09. +- Dominance thresholds, cohort-size guidance, comparison windows, and escalation timing are analyst heuristics unless a cited Sent source states otherwise. +- Re-check the repository documentation source catalog before relying on exact endpoints, enums, or limits after its `last_verified` date.