Improve documentation organization, accuracy, and readability

[wangyinz]

Summary

• reorganize the documentation around clearer desktop, cluster, data, processing, and reference paths
• review and update user-manual and API pages while preserving important existing material
• simplify the README and onboarding guides so the short Docker happy path remains prominent
• correct the Spark Compose database service reference while documenting that mspass-dbmanager remains valid for the sharded configuration
• fix generated API-reference markup, including the ambiguous elog cross-reference

Why

The documentation had outdated commands, broken or incomplete API coverage, inconsistent navigation, and some deployment guidance that no longer matched the shipped configuration. A preservation/readability pass also found that several initial rewrites made first-run instructions unnecessarily intimidating, so those pages were simplified and the original approachable commands and list structure restored.

User impact

Users get clearer navigation, current deployment and data-management guidance, more complete API references, and shorter onboarding flows. Existing conceptual explanations, diagrams, examples, and important caveats are retained.

Validation

• full Sphinx HTML build with the project mspass Conda environment
• all documentation pages rendered successfully
• ambiguous generated elog cross-reference warning eliminated
• only two expected offline intersphinx inventory warnings remained
• shipped Dask and Spark Compose files validated with docker compose config --quiet
• git diff --check

Local HTML output used for final review: /tmp/mspass-docs-html-final-20260711/index.html.

[codecov]

Codecov Report

❌ Patch coverage is 88.67925% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 64.45%. Comparing base (09d914f) to head (6f86190).
⚠️ Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
python/mspasspy/db/normalize.py	88.67%	6 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #754      +/-   ##
==========================================
+ Coverage   64.29%   64.45%   +0.16%     
==========================================
  Files         170      170              
  Lines       30655    30663       +8     
  Branches     2038     2038              
==========================================
+ Hits        19709    19765      +56     
+ Misses      10193    10145      -48     
  Partials      753      753              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[wangyinz]

I completed a preservation-first corrective review after identifying that the earlier readability pass removed too much beginner-oriented material.

Corrective commits:

• 5630a2e2 restores detailed user guidance in the high-churn manual and Docker/Compose pages.
• c5623686 restores onboarding explanations, the original quick-start teaching flow, and historical authorial content.

The review covered every changed documentation file. Material was restored unless it was demonstrably obsolete, incorrect, an unfinished placeholder, or replaced by a code-verified correction. Examples include restoring the full 913-line parallel-I/O chapter, the beginner error-handling philosophy, CRUD explanations, Docker port/mount/Jupyter guidance, virtual-cluster concepts and diagrams, GUI configuration explanations, and the original quick-start startup-output walkthrough.

Final validation: a clean forced Sphinx HTML build rendered every page successfully. The only warnings were the two offline external intersphinx inventories; the ambiguous generated elog warning remains fixed.

Review build: /tmp/mspass-docs-html-preservation-final/index.html.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review is ineligible. To be eligible to request a review, you need a paid Copilot license, or your organization must enable Copilot code review.

[pavlis]

I have some minor things I think should be resolved before we merge this, but the big picture is this fixes a lot more problems and mistakes than it creates. i.e. there are likely mistakes I missed, but it nothing compared to what it fixes.

@wangyinz look at the comments I left and consider implementing some of the minor changes I suggested. One thing I think I did but can't tell for sure is remove a file called "research_computing_essay.rst". That was something I started and decided to not include. It must have gotten pushed to the repository in error. Make sure it is removed with this pull request.

[Copilot]

Pull request overview

Copilot reviewed 57 out of 57 changed files in this pull request and generated 1 comment.

[wangyinz]

Addressed the human review feedback in d8caa6f6, the ObsPy follow-up in 29d18d7c, and the Copilot review in 9b52e18d. All eleven inline review threads now have responses and are resolved.

• Fixed the duplicated wording and replaced the dead ANF link with the final archive DOI in both affected pages.
• Re-reviewed memory_management.rst and corrected the concrete Dask/Spark, RSS, spill, worker/thread, heuristic, skew, and ensemble guidance issues.
• Corrected the ObsPy example to retain the Stream: native Stream.filter() processes every contained Trace, and the MsPASS signals.filter wrapper supports ensemble inputs through Stream conversion decorators.
• Corrected the matcher input contract to accept Metadata, added an unambiguous Sphinx type link, and aligned the documented list return contract with implementation.
• Corrected rota to rtoa in both locations and repaired the nearby ensemble-processing example so it processes each member and uses the current APIs.
• Confirmed removal of the unfinished essays and verified the RMS and peak equations and their LaTeX.
• Audited for similar documentation problems and fixed additional grammar, duplicate words, stale links, API names, type contracts, and code-example errors.

Validation: Python compilation and a runtime plain-Metadata matcher check pass; git diff --check passes; and a clean forced Sphinx HTML build succeeds. The only two warnings are unreachable external ObsPy and PyMongo intersphinx inventories.

[Copilot]

Pull request overview

Copilot reviewed 57 out of 57 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

python/mspasspy/db/normalize.py:677

• The docstring currently says mspass_object must be one of the waveform types (TimeSeries/Seismogram/Ensembles), but the actual check is isinstance(mspass_object, Metadata), which also allows plain Metadata (and works because the waveform types inherit Metadata). Clarifying this in the docstring would make the contract consistent with the implementation.

[Copilot]

Pull request overview

Copilot reviewed 57 out of 57 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 58 out of 58 changed files in this pull request and generated no new comments.