Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
128 changes: 128 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -236,6 +236,16 @@ Triggers: PR to main, **push to main** (separate from pages.yml E2E step)

Runs a full build + Playwright test on every push to main and every PR. Uploads `playwright-report/` artifact on failure. Uses `github.token` only. Does NOT deploy.

### `update-hive-cache.yml` — Hive history pipeline

Triggers: schedule (every 2h), workflow_dispatch

Steps: checkout → setup-node → restore hive cache (incremental, key: `github-data-hive-`) → `npm run fetch-hive-history` → save cache → commit updated `static/data/hive-history.json`

Permissions: `contents:write` — commits history file back to main.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Verify actual permissions in the workflow
rg -A5 "permissions:" .github/workflows/update-hive-cache.yml

Repository: projectbluefin/documentation

Length of output: 171


Line 245: Fix permissions documentation mismatch

The workflow has permissions: contents:read only, but line 245 documents contents:write with the intent to "commit updated hive-history.json" back to main. A read-only workflow cannot write commits to the repository. Either:

  1. Correct line 245 to reflect that the workflow caches the file but does not commit it, or
  2. Update the workflow to include contents:write if commits are required

Verify the intended behavior and align documentation with implementation.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@AGENTS.md` at line 245, The documentation at line 245 in AGENTS.md claims the
workflow has `contents:write` permissions to commit the hive-history.json file
back to main, but the workflow itself only has `contents:read` permissions,
which cannot perform write operations. Verify the intended behavior: if the
workflow should only cache the file without committing it, update line 245 to
accurately reflect this read-only behavior. If commits to hive-history.json are
actually required, update the workflow permissions configuration to include
`contents:write`. Ensure the documentation and workflow permissions are aligned
with the actual intended behavior.


Fetches: Hive snapshot HTML, org contributor counts (all-time), contributor weekly stats (GitHub `/stats/contributors`, daily TTL), per-repo contributor breakdown. Appends snapshot entry every 2h (max 2016 entries / ~6 months rolling).

### `update-sbom-cache.yml` — Nightly SBOM fetch

Triggers: schedule (04:00 UTC nightly), workflow_dispatch
Expand Down Expand Up @@ -284,6 +294,7 @@ Runs `renovate-config-validator --strict`.
| `MusicPlaylist.tsx` | `docs/music.md` | `static/data/playlist-metadata.json` |
| `PageContributors.tsx` | DocItem/Footer (all doc pages) | `static/data/file-contributors.json` |
| `GiscusComments/` | Blog posts | GitHub Discussions via Giscus |
| `HiveFactoryDashboard.tsx` | `src/pages/hive.tsx` (`/hive/`) | Client-side: snapshot HTML, queue API, GitHub API, `hive-history.json` |

### Changelog package tracking

Expand Down Expand Up @@ -344,6 +355,123 @@ npm run start # preview at http://localhost:3000/reports

---

## Hive Factory Dashboard

The Bluefin OS Factory live dashboard lives at `docs.projectbluefin.io/hive/`.

- Source: `src/components/HiveFactoryDashboard.tsx` (~3200 lines) + `HiveFactoryDashboard.module.css`
- Page wrapper: `src/pages/hive.tsx`
- Title: **Bluefin Operating System Factory** (no "The") — subtitle unchanged
- Tagline at page bottom: "Enslaving the Oppressors since 2026" (Destiny lore)

### Data sources

All data is fetched **client-side at runtime** — no build-time fetch for the hive snapshot.

| Source | What it provides |
|---|---|
| Snapshot HTML at `raw.githubusercontent.com/kubestellar/docs/main/public/live/hive/bluefin/index.html` | Live hive state — governor mode, ACMM level, agent status, victories, advisories |
| `queue.projectbluefin.io/data.json` | Guardians/Ghosts PR queue (regenerates every 10 min) |
| GitHub API (`api.github.com`) | PR search (merged/open), org stats, commit activity |
| `static/data/hive-history.json` | Historical snapshot entries, org-wide contributor stats |

**`hive-history.json`** is a committed file (gitignore exception) updated every 2h by `.github/workflows/update-hive-cache.yml`. Script: `scripts/fetch-hive-history.js`.

### Snapshot parsing

The snapshot HTML contains multiple `render(` calls. To find the real JSON payload:
- Scan all `render(` occurrences; pick the one where the next non-whitespace char is `{`
- Top-level `acmmLevel` field is at the root — NOT `agentMetrics.outreach.acmm`

### Panels

| Panel | Data source | Notes |
|---|---|---|
| Hero / ACMM level | snapshot root `acmmLevel` | Mission timer, velocity sparkline |
| Governor panel | snapshot `governor` | Mode, queue depth, thresholds |
| Governor Timeline | snapshot `timeline[]` | 24h colored mode strip (surge/busy/quiet/idle) |
| Token Budget | snapshot `governor.budgetPct` etc. | Progress bar, green/amber/red thresholds |
| Nous / Strategy Lab | snapshot `nous` | Active experiment, snapshot progress, principle count |
| Frame Formation | snapshot `agents[]` | Per-Frame status, ACMM level, sparklines |
| Victory Log | snapshot `victories[]` | Recent hive wins |
| Velocity Panel | GitHub PR search | 7/30-day merge rate sparklines |
| Issue Queue | GitHub issues API | P0/P1/agent-ready bucketed |
| Recently Merged | GitHub PR search (30 PRs) | Guardian/Ghost split, conventional commit badges |
| Contributor Wall | `hive-history.json` contributors | Unique human contributors across all 15 repos |
| Factory Trends | `hive-history.json` entries (last 72) | 6 sparklines: ACMM, budget, queue, advisories, merged/cycle, merge time |
| Contributor Leaderboard | `hive-history.json` contributorStats | Tabs: All Time / This Month / This Week; top 25; milestone badges |

### Destiny lore two-column layout

| Column | Title | Color | Contents |
|---|---|---|---|
| Left | Guardians | Warm gold `#d97706` | Human contributor PRs — the Light we carry |
| Right | Ghosts | Sapphire `#2563eb` / `#93c5fd` | Agent-driven issues — the machines at work |

### Contributor Leaderboard

Tabs: All Time / This Month / This Week. Top 25 per tab. Shows rank, avatar, commit count, top repos, milestone badges.

Milestone badge tiers (highest earned shown):

| Badge | Threshold | Color |
|---|---|---|
| Mythic | 5000+ lifetime commits | `#ff7b72` |
| Legend | 1000+ lifetime commits | `#f0883e` |
| Elite | 500+ lifetime commits | `#bc8cff` |
| Veteran | 100+ lifetime commits | `#58a6ff` |
| Contributor | 10+ lifetime commits | `#3fb950` |
| Sprint | 10+ commits this week | `#d29922` |
| Rising | This week > 1.5x 4-week avg | `#3fb950` |
| Multi-repo | Active in 3+ repos | `#a371f7` |

Weekly/monthly data (from GitHub `/stats/contributors`) accumulates after CI warms the stats cache. Falls back to all-time view during warmup.

### hive-history.json schema

```json
{
"entries": [{ "t": ms, "acmmLevel": 3, "govMode": "busy", "queue": 175, "agents": 6,
"runningAgents": 5, "advisories": 0, "budgetPct": null,
"mergedToday": null, "mergedThisWeek": null, "medianMergeMins": 801 }],
"contributors": { "login": totalCommits },
"contributorsByRepo": { "repo": { "login": commits } },
"lastContributorFetch": "ISO",
"contributorStats": {
"login": { "total": N, "lastWeek": N, "lastMonth": N, "last3Months": N, "byRepo": {"repo": commits} }
},
"lastWeeklyStatsFetch": "ISO"
}
```

`contributorStats` is populated by `fetchContributorWeeklyStats()` in `fetch-hive-history.js` (daily TTL). GitHub's `/stats/contributors` returns HTTP 202 on cold cache — script retries with backoff.

### Factory repos tracked (15)

`bluefin`, `common`, `documentation`, `actions`, `bluefin-lts`, `dakota`, `bonedigger`, `bootc-installer`, `knuckle`, `testsuite`, `website`, `brew`, `iso`, `wolfictl`, `fisherman`

### Theme constants

- Background: `#0d1117`
- Guardians accent: `#d97706` (amber)
- Ghosts accent: `#2563eb` / `#93c5fd` (sapphire)
- Governor modes: surge=`#f85149` busy=`#d97706` quiet=`#3b82f6` idle=`#21262d`

### Adding a new data panel

1. Add interface fields to `HiveSnapshot` (types block ~L36-100)
2. Extract in `parseSnapshotJson` (~L260-380)
3. Write the component (follow `GovernorTimeline` or `TokenBudgetPanel` as template)
4. Add CSS to `HiveFactoryDashboard.module.css`
5. Wire into JSX render (conditionally on data presence)
6. `npm run typecheck && npm run lint && npm run build` — all must pass

**Never** add `fetch-sbom` or `fetch-data` to the hive pipeline — the snapshot is fetched client-side at runtime.

**CSP:** `connect-src` in `docusaurus.config.ts` must include `raw.githubusercontent.com`, `queue.projectbluefin.io`, `api.github.com`.

---

## ProjectCard component

`src/components/ProjectCard.tsx` — used on `docs/donations/projects.mdx`.
Expand Down
150 changes: 141 additions & 9 deletions scripts/fetch-hive-history.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,31 @@
*
* Runs every 2 hours via update-hive-cache.yml.
* Appends a snapshot of key Hive metrics to static/data/hive-history.json.
* Also refreshes all-time contributor stats from every active factory repo.
* Refreshes all-time contributor counts once per day (contributors endpoint).
* Refreshes weekly contributor stats once per day (stats/contributors endpoint).
*
* History file format:
* {
* "entries": [ { t, acmmLevel, govMode, budgetPct, queue, agents, advisories,
* mergedToday, mergedWeek, runningAgents } ... ],
*
* // All-time totals from /contributors endpoint
* "contributors": { "login": totalCommits, ... },
* "contributorsByRepo": { "repo": { "login": commits } },
* "lastContributorFetch": "ISO timestamp"
* "lastContributorFetch": "ISO timestamp",
*
* // Weekly breakdown from /stats/contributors endpoint
* // Enables monthly/weekly leaderboard windows
* "contributorStats": {
* "login": {
* "total": 5680,
* "lastWeek": 12, // commits in last 7 days
* "lastMonth": 45, // commits in last 28 days (4 weeks)
* "last3Months": 150, // commits in last 91 days (13 weeks)
* "byRepo": { "repo": commits }
* }
* },
* "lastWeeklyStatsFetch": "ISO timestamp"
* }
*/

Expand All @@ -23,11 +39,14 @@ const SNAPSHOT_HTML_URL =
"https://raw.githubusercontent.com/kubestellar/docs/main/public/live/hive/bluefin/index.html";
const OUTPUT_FILE = path.join(__dirname, "../static/data/hive-history.json");

// 14 days at one entry per 2h = 168 entries
const MAX_ENTRIES = 168;
// ~6 months at one entry per 2h = 2016 entries
const MAX_ENTRIES = 2016;

// Refresh all-time contributor counts once per day
const CONTRIBUTOR_TTL_MS = 24 * 60 * 60 * 1000;

// Refresh contributor stats at most once every 6 hours
const CONTRIBUTOR_TTL_MS = 6 * 60 * 60 * 1000;
// Refresh weekly stats once per day (stats/contributors is expensive: 1 req/repo)
const WEEKLY_STATS_TTL_MS = 24 * 60 * 60 * 1000;

// All active factory repos
const FACTORY_REPOS = [
Expand Down Expand Up @@ -224,6 +243,92 @@ async function fetchContributors() {
return { totals, byRepo };
}

/**
* Fetch weekly contributor stats via /repos/{owner}/{repo}/stats/contributors.
* Returns lastWeek / lastMonth / last3Months windows per contributor.
*
* The endpoint may return 202 while GitHub computes stats. We retry up to 3 times
* with a 2-second back-off per repo.
*
* Returns: { [login]: { total, lastWeek, lastMonth, last3Months, byRepo: { repo: commits } } }
*/
async function fetchContributorWeeklyStats() {
const nowSec = Math.floor(Date.now() / 1000);
const weekAgo = nowSec - 7 * 86400;
const monthAgo = nowSec - 28 * 86400; // 4 weeks
const threeMonthsAgo = nowSec - 91 * 86400; // 13 weeks

// stats[login] = { total, lastWeek, lastMonth, last3Months, byRepo }
const stats = {};

await Promise.allSettled(
FACTORY_REPOS.map(async (repo) => {
const url = `${GH_API}/repos/projectbluefin/${repo}/stats/contributors`;
let attempts = 0;
let data = null;
while (attempts < 4) {
attempts++;
let res;
try {
res = await fetch(url, { headers: ghHeaders() });
} catch {
break;
}
if (res.status === 404 || res.status === 403 || res.status === 204) break;
if (res.status === 202) {
// GitHub is computing stats — wait and retry
await new Promise((r) => setTimeout(r, 2000 * attempts));
continue;
}
if (!res.ok) break;
try {
data = await res.json();
} catch {
break;
}
break;
}
if (!Array.isArray(data)) return;

for (const entry of data) {
const login = entry?.author?.login;
if (!login) continue;
if (login.endsWith("[bot]") || BOT_LOGINS.has(login)) continue;
const total = typeof entry.total === "number" ? entry.total : 0;
if (total === 0) continue;

// Sum weekly commit counts for each time window
let lastWeek = 0;
let lastMonth = 0;
let last3Months = 0;
if (Array.isArray(entry.weeks)) {
for (const w of entry.weeks) {
const wt = typeof w.w === "number" ? w.w : 0;
const wc = typeof w.c === "number" ? w.c : 0;
if (wc === 0) continue;
if (wt >= weekAgo) lastWeek += wc;
if (wt >= monthAgo) lastMonth += wc;
if (wt >= threeMonthsAgo) last3Months += wc;
}
}

if (!stats[login]) {
stats[login] = { total: 0, lastWeek: 0, lastMonth: 0, last3Months: 0, byRepo: {} };
}
stats[login].total += total;
stats[login].lastWeek += lastWeek;
stats[login].lastMonth += lastMonth;
stats[login].last3Months += last3Months;
if (total > 0) {
stats[login].byRepo[repo] = (stats[login].byRepo[repo] || 0) + total;
}
}
})
);

return stats;
}

function loadHistory() {
try {
if (fs.existsSync(OUTPUT_FILE)) {
Expand All @@ -236,7 +341,9 @@ function loadHistory() {
entries: [],
contributors: {},
contributorsByRepo: {},
contributorStats: {},
lastContributorFetch: null,
lastWeeklyStatsFetch: null,
};
}

Expand All @@ -247,6 +354,7 @@ async function main() {
if (!Array.isArray(history.entries)) history.entries = [];
if (!history.contributors) history.contributors = {};
if (!history.contributorsByRepo) history.contributorsByRepo = {};
if (!history.contributorStats) history.contributorStats = {};

// ── Fetch hive snapshot ──────────────────────────────────────────────────
let metrics = null;
Expand Down Expand Up @@ -279,14 +387,14 @@ async function main() {
);
}

// ── Refresh contributor stats if stale ───────────────────────────────────
// ── Refresh all-time contributor counts (daily) ───────────────────────────
const lastFetch = history.lastContributorFetch
? new Date(history.lastContributorFetch).getTime()
: 0;
const needsContributorRefresh = Date.now() - lastFetch > CONTRIBUTOR_TTL_MS;

if (needsContributorRefresh) {
console.log("[hive-history] Fetching contributor stats from factory repos...");
console.log("[hive-history] Fetching all-time contributor counts from factory repos...");
try {
const { totals, byRepo } = await fetchContributors();
history.contributors = totals;
Expand All @@ -301,7 +409,31 @@ async function main() {
console.warn(`[hive-history] Contributor fetch failed: ${err.message}`);
}
} else {
console.log("[hive-history] Contributor stats still fresh, skipping fetch");
console.log("[hive-history] All-time contributor counts still fresh, skipping");
}

// ── Refresh weekly contributor stats (daily) ─────────────────────────────
const lastWeeklyFetch = history.lastWeeklyStatsFetch
? new Date(history.lastWeeklyStatsFetch).getTime()
: 0;
const needsWeeklyRefresh = Date.now() - lastWeeklyFetch > WEEKLY_STATS_TTL_MS;

if (needsWeeklyRefresh) {
console.log("[hive-history] Fetching weekly contributor stats (stats/contributors)...");
try {
const stats = await fetchContributorWeeklyStats();
history.contributorStats = stats;
history.lastWeeklyStatsFetch = new Date().toISOString();
const count = Object.keys(stats).length;
const activeThisWeek = Object.values(stats).filter((s) => s.lastWeek > 0).length;
console.log(
`[hive-history] Weekly stats: ${count} contributors, ${activeThisWeek} active this week`,
);
} catch (err) {
console.warn(`[hive-history] Weekly stats fetch failed: ${err.message}`);
}
} else {
console.log("[hive-history] Weekly contributor stats still fresh, skipping");
}

// ── Write output ─────────────────────────────────────────────────────────
Expand Down
Loading