docs: add optimizer docs hidden from docs build

[jeremysilva1098]

Summary by CodeRabbit

• Documentation
  • Added comprehensive guidance for Insight-Driven Optimization, including agent roles, prerequisites, setup, workflows, CLI usage, scheduling, configuration, artifacts, and troubleshooting.
  • Documented telemetry-driven analysis, experimentation, validation, evaluation, persistence options, and draft pull request workflows.
  • Added gated navigation entries and publication guidance for the new Agents documentation, making the content easier to discover when released.

[github-actions]

🌿 Preview your docs: https://nvidia-preview-pr-1014-optimizer-docs-jesilva.docs.buildwithfern.com/nemo-platform

[coderabbitai]

📝 Walkthrough

Walkthrough

Adds a comprehensive Insight-Driven Optimization guide and navigation entries for gated and latest documentation versions.

Changes

Insight-Driven Optimization documentation

Layer / File(s)	Summary
Insight-Driven Optimization guide docs/agents/insight-driven-optimization.mdx	Documents the Analyst, Experimenter, and Eval Author agents, setup, workflows, data persistence, CLI commands, scheduling, evaluation, outputs, and troubleshooting.
Documentation navigation wiring docs/fern/gated-nav.yml, docs/fern/versions/latest.yml	Adds gated navigation placement and commented guidance for publishing the Agents page in the latest documentation version.

Possibly related PRs

• NVIDIA-NeMo/nemo-platform#896: Covers related Experimentalist plugin workflows documented in this change.
• NVIDIA-NeMo/nemo-platform#975: Covers related Insight-Driven Optimization agents and plugins.

Suggested labels: docs

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly describes the addition of optimizer documentation and its exclusion from the docs build.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch optimizer-docs/jesilva

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/agents/insight-driven-optimization.mdx`:
- Around line 356-357: Update both Studio links in the optimization setup
documentation to construct their URLs from the exported NMP_BASE_URL and
WORKSPACE values instead of hard-coded localhost:8080 and default values, so
they work for remote deployments and non-default workspaces.
- Around line 348-353: Update the “Confirm it worked” section near the API curl
command to distinguish persistence targets by workflow: for profile mode,
instruct users to verify entries in .nemo-optimizer/insights.yaml, and for API
mode, retain the API query command. Ensure the confirmation guidance reflects
the optimizer.yaml discovery behavior described earlier.
- Around line 348-353: Update the Insights API curl example under “Confirm it
worked” to include the Bearer Authorization header using NMP_ACCESS_TOKEN and
the --fail-with-body option, while preserving the existing URL and query
parameters.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 3e654248-c341-4a48-9d89-6cdf5fc63dc2

📥 Commits

Reviewing files that changed from the base of the PR and between af0e372 and 6ccf340.

⛔ Files ignored due to path filters (2)

• docs/agents/_images/optimizer-experiment-comparison.png is excluded by !**/*.png
• docs/agents/_images/optimizer-insight-detail.png is excluded by !**/*.png

📒 Files selected for processing (3)

• docs/agents/insight-driven-optimization.mdx
• docs/fern/gated-nav.yml
• docs/fern/versions/latest.yml

[github-actions]

Suite	Lines Covered	Line Rate	Branch Rate
Unit Tests	29440/37448	78.6%	63.2%
Integration Tests	17379/36166	48.0%	20.6%