Skip to content

Docs: clarify equal-weighted portfolio vs cap-weighted benchmark comp… - #2316

Open
taro0915 wants to merge 1 commit into
microsoft:mainfrom
taro0915:docs/benchmark-weighting-note
Open

Docs: clarify equal-weighted portfolio vs cap-weighted benchmark comp…#2316
taro0915 wants to merge 1 commit into
microsoft:mainfrom
taro0915:docs/benchmark-weighting-note

Conversation

@taro0915

Copy link
Copy Markdown

Description

Documentation-only follow-up to #2315. No behavior change.

  • qlib/contrib/strategy/signal_strategy.py — add a class docstring to TopkDropoutStrategy stating that positions are equal-weighted, and noting the implication for benchmark choice. (The class previously had no docstring, only a # TODO: block.)
  • qlib/workflow/record_temp.py — add a note to the PortAnaRecord docstring describing how excess_return_with_cost / excess_return_without_cost are computed, and that no weighting adjustment is applied.
  • docs/component/strategy.rst — add the same note to the TopkDropoutStrategy section.

No code paths, defaults, or outputs are modified.

Motivation and Context

Related issue: #2315

TopkDropoutStrategy sizes new positions by splitting the available cash evenly across the stocks to be bought (qlib/contrib/strategy/signal_strategy.py:266), so the realized portfolio is close to equal-weighted. PortAnaRecord reports excess return as a plain arithmetic difference against a single benchmark instrument (qlib/workflow/record_temp.py:507-512), and the default benchmark is SH000300 (CSI 300), which is capitalization-weighted.

These two defaults are used together in the default PortAnaRecord config and throughout examples/benchmarks. Because no adjustment is made for the difference in weighting scheme, the reported excess return contains the equal- versus cap-weighting spread of the universe in addition to the contribution of the signal. #2315 has the measurements: with a zero-alpha signal the offset is material and consistently signed, and it shifts the reported numbers for a real LightGBM + Alpha158 run as well.

This is not a defect in the arithmetic. Excess return against a named index is a legitimate quantity, and it is the right one if the question is "did this beat the CSI 300." It is a gap in what the documentation lets a reader assume, because the same number is routinely read as the contribution of the model. This PR makes the distinction explicit at the three places a user is most likely to be standing when they form that assumption.

Why documentation first

#2315 sketched two possible directions: changing or parameterizing the comparison, or documenting the current behavior. The second is separable and carries no compatibility risk, so it is offered on its own here rather than bundled. If maintainers prefer the first, an equal-weighted benchmark option is a natural follow-up — qlib/contrib/report/analysis_position/profit_attribution.py:238 already carries a TODO for exactly that — and I am happy to open it.

How Has This Been Tested?

  • Pass the test by running: pytest qlib/tests/test_all_pipeline.py under upper directory of qlib.
  • If you are adding a new feature, test on your own test scripts.

Docstrings and reStructuredText only; no executable code paths are touched, so there are no test coverage implications. The change adds lines only (+40 / -0) and does not modify any existing line, so black formatting of existing code is unaffected.

Types of changes

  • Fix bugs
  • Add new feature
  • Update documentation

@taro0915

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants